Test Analysis

UReport provides three view modes for browsing test execution results:

In addition to the three view modes, a Comparison toggle is available in the Launches toolbar. Enabling it opens a split-panel overlay that shows the same test results side-by-side across multiple product lanes — it works alongside any of the three views above.

The toolbar above the test list gives you several ways to narrow down results:

The Display Settings button (sliders icon) controls history mode, sort order, and visible columns. The Filter button (funnel icon) controls quarantine visibility, speed, group-by, and pinned-only mode.

Quarantine filter modes

The quarantine filter has three modes:

Click any test to open the detail panel on the right. It contains four tabs (or five when multiple lanes are selected in comparison mode):

At the top of the panel, the ⋮ button opens the test actions menu with options to triage, link as outage, link as custom state, assign, pin, and more.

If the test has been triaged and has a tracking ticket, a tracking number button appears in the header — click it to open the linked issue directly in your issue tracker.

When a test fails and you know why, you triage it. This records the root cause (issue type + tracking ticket) and attaches it to the failure pattern. Any future test run that fails with the same pattern is automatically matched to the same triage record — so you never have to re-triage the same known issue twice.

Triaged tests are excluded from the "new failures" count and are visually distinguished so the team can focus on genuinely unknown failures.

When you triage a test, UReport stores a matching strategy on the record. Every subsequent failing test is compared against all triage records using that strategy to decide if it is already a known issue.

By default, the Compare By selector is not shown in the modal — it is an opt-in feature. An admin must enable it under Settings → Advanced Analysis → Allow Compare By Selection. Until then, every triage record uses the default automatic cascade described below.

Default behavior — automatic cascade

When no explicit strategy is stored, UReport tries each rule in order and stops at the first match:

Explicit strategies (admin opt-in)

When "Allow Compare By Selection" is enabled, a Compare By selector appears in the triage modal. Users can override the cascade per triage record:

The token field

The token is an optional string you include in the test failure payload. UReport stores and compares it exactly as provided.

Similarity matching lets you back-fill investigations onto historical failures in previous builds. This is separate from the current-build "similar tests" checkboxes.

• Toggle on Enable Similarity Matching
• Adjust the percentage slider (60–100%) — higher means stricter match
• The dialog shows all history tests that meet the threshold, with a percentage score and highlighted diff of the error message
• On submit, those history tests are also triaged with the same tracking ticket

To update a test that is already triaged, open the ⋮ menu — the "Triage" submenu is still available. Choosing a different issue type re-opens the dialog pre-filled with the existing data.

To remove the triage, open the ⋮ menu and click "Remove Triage". This deletes the triage record immediately.

An outage is for failures caused by an infrastructure or environment problem — not a bug in the application or test code. Examples: database down, network timeout, CI agent misconfiguration, third-party service unavailable.

When you link a test as an outage, UReport creates an Outage record with the failure message as the match pattern. Every test in that build that produces the same failure message is automatically marked as outage-affected — even tests you did not open individually.

Outage-affected tests are excluded from pass rate calculations, so your quality metrics stay clean during infrastructure incidents.

This is the key difference from "Triage": triaged tests persist globally and automatically match future failing tests across all subsequent builds until removed. Outages must be re-applied each time the infrastructure issue recurs.

Use outages for one-off incidents (a CI environment was down during build #123). Use triage for recurring known failures that need to be suppressed across many builds.

Open the ⋮ menu on the outage-affected test and click "Remove Outage". This does not delete the outage record — it adds the test to the exception list so only that test is no longer affected. Other tests matched by the same outage pattern remain unaffected.

Custom states are product-lane-specific categories you can apply to failing tests when neither "Triaged" nor "Outage" fits your workflow. They are useful for temporary or workflow-specific classifications.

Custom states are configured per product lane in Settings → Product Lanes → Edit Lane → Custom States:

Assigning a test marks a specific person as responsible for investigating or fixing a failure. This is separate from investigating — you can assign without knowing the root cause yet.

Test relations are metadata labels you attach to tests — teams, components, tags, and custom key-value pairs. Once attached, relations become a first-class dimension you can slice data by across the entire platform:

• Filter tests in Launches — narrow down results to tests owned by a specific team or covering a specific component
• Dashboard widgets — build pass-rate charts, failure trend charts, and heat maps scoped to a team, component, or tag
• Comparison analysis — compare quality across teams or components side by side
• Accountability — assign ownership so the right team sees their failures immediately

Without relations, dashboards and filters can only work at the product lane level. With relations, you get granular visibility into which part of the system is failing and who owns it.

All official reporters let you attach relation metadata directly in test code. How those relations are saved to UReport differs by reporter:

There are two ways to add relations:

Option 1: Inline with test results

{
  "tests": [
    {
      "build": "65abc123def456789",
      "uid": "checkout-payment-001",
      "name": "CheckoutTest.testPaymentFlow",
      "status": "PASS",
      "info": {
        "file": "tests/checkout/payment.spec.ts",
        "path": "tests/checkout",
        "components": ["Checkout", "Payment"],
        "teams": ["E-Commerce"],
        "tags": ["smoke", "critical"],
        "priority": "P1"
      }
    }
  ]
}

Option 2: Pre-configured relations

{
  "uid": "checkout-payment-001",
  "product": "ECommerce",
  "type": "E2E",
  "file": "tests/checkout/payment.spec.ts",
  "path": "tests/checkout",
  "components": [{"name": "Checkout"}, {"name": "Payment"}],
  "teams": [{"name": "E-Commerce"}],
  "tags": [{"name": "smoke"}, {"name": "critical"}],
  "customs": {
    "priority": "P1",
    "feature": "Payment Processing"
  }
}

UReport automatically analyzes test execution history to surface patterns. Behavior tags appear as clickable chips in the filter bar — click one to filter to only those tests.

UReport automatically categorizes test speed based on duration. Thresholds differ between API and UI tests:

API Test Thresholds

UI Test Thresholds

Comparison View shows the same test side-by-side across multiple product lanes. Use it to isolate environment-, browser-, or version-specific failures.

In the test detail panel, the Comparison tab (visible when multiple lanes are selected) shows per-lane status, metadata, and a step-by-step diff against the base run.

Pinning marks a test so you can easily find it again. Pin status is personal and saved to your account on the server — it persists permanently until you manually unpin, surviving logout, browser refresh, and device changes.