Auto-Triage
Auto-Triage automatically propagates known failure investigations to new failing tests with matching tokens, error messages, or stack traces — without any manual triage needed.
What It Does
When a build completes with failing tests, auto-triage checks whether any of those failures match a known investigation — identified by failure token, error message, or stack trace.
If a match is found, UReport automatically creates an investigation record for the new failing test, copying the cause and tracking number from the source. The new record is tagged as auto-triaged and has a configurable expiry (TTL) — after which the match is considered stale and the test will be re-evaluated on the next build.
Auto-triage covers tests that share the same failure pattern across different test IDs, suites, or builds. Once a root cause has been investigated once, all future tests with the same failure are labelled automatically.
Key properties:
- Fires automatically after every build completion — no action needed
- Copies cause and tracking info from existing investigations
- Message and stack trace matches expire after a configurable TTL (default 5 days); token matches and promoted matches are permanent
Auto-Triage vs Quarantine
| Auto-Triage | Quarantine | |
|---|---|---|
| Purpose | Label a test with a known root cause | Exclude a persistently failing test from the pass rate |
| Trigger | Failure pattern match (token / message / stack trace) | Failure frequency threshold (total / consecutive / rate) |
| Effect on pass rate | None — the test still counts as failed | Test is excluded from the pass rate calculation |
| Expiry | Yes — auto-triage matches have a TTL | No — quarantine is active until resolved or exempted |
| Input required | At least one manual investigation with a matching failure | No manual input — fully rule-driven |
Auto-triage and quarantine can both apply to the same test at the same time. They are independent features.
Enable and Configure
Auto-triage is configured per product/type from the Launches page. Click the wand icon (✨) in the toolbar to open the auto-triage settings panel, then toggle it on.
| Setting | Default | Description |
|---|---|---|
| Enabled | Off | Master switch. Auto-triage does not run unless this is on. |
| Review TTL (days) | 5 | How long a message or stack trace match remains valid. After this many days, the auto-triage record expires and the test becomes eligible for re-triage. Range: 1–365. |
| Source age limit (days) | 90 | Investigations older than this are not used as match sources. Prevents very old patterns from silently matching new failures. Range: 1–180. |
TTL by Match Type
| Match type | Expires? | Default behaviour |
|---|---|---|
Token (token) | No — permanent | Never expires. Token matches are highly reliable — based on a deterministic identifier embedded in the test failure. |
Error message (exact) | Yes | Expires after Review TTL days. |
Stack trace (stack) | Yes | Expires after Review TTL days. |
| Promoted match | No — permanent | A message or stack match promoted to permanent. Never expires. |
Set Review TTL to a low value (e.g. 3–5 days) for suites with dynamic error messages. Use a higher value (e.g. 14–30 days) for stable messages like assertion errors that rarely change wording.
Matching Logic
For each failing test, auto-triage tries three lookups in priority order. The first match wins.
Step 1 — Token Match (highest priority)
If the test has a failure token, it is looked up in the token map. Token matches are the most reliable — a token is a deterministic identifier generated by the test framework (e.g. a hash of the test ID + assertion location). TTL: permanent.
Step 2 — Error Message Match (exact)
If no token match, the full error message is compared exactly against the message map. Punctuation, whitespace, and casing must match exactly. TTL: expires after Review TTL.
Step 3 — Stack Trace Match (exact)
If no message match, the full stack trace is compared exactly against the stack map. Used for investigations configured with COMPARE_BY_MIXED or COMPARE_BY_STACK_TRACE. TTL: expires after Review TTL.
If no match is found at any step, the test is left uninvestigated — it will be checked again on the next build.
Source Investigation Validity
Auto-triage builds its match maps from existing InvestigatedTest records for the same product/type, created within the source age limit window.
| Source type | Valid as source? | Notes |
|---|---|---|
| Manual investigation | Always | A human-created investigation is always valid regardless of age (within source age limit). |
| Auto-triage, token match | Always | Token matches are permanent — the source record never expires. |
| Auto-triage, message/stack match | Only if not expired | If the source auto-triage record has expired (TTL elapsed), it is excluded from the match pool. |
| Fuzzy match (similarity < 100%) | Never | Fuzzy matches cannot be safely replicated by exact lookup — always excluded from the source pool. |
| TTL = 0 or unset on auto-triage record | Never | Treated as expired immediately. |
Expired Source Handling
An expired auto-triage record means the match confidence window has closed. Using it as a source would silently re-triage tests that should be reviewed manually — so it is excluded from the match pool.
Expiry applies only to the source pool. A test that has an active (non-expired) auto-triage record is still considered investigated — it won't be re-triaged until its own TTL expires.
Automatic Trigger Sequence
Runs automatically after every build's status calculation:
- Build completes — status is calculated
- Failing tests identified — all tests with
status = FAILare collected - Rerun exclusion — tests that passed on retry (
status = PASS, is_rerun = true) are excluded - Coverage check — tests that already have a valid (non-expired) investigation are skipped
- Match and create — eligible tests are matched against the source pool; matches result in auto-triage records being created or refreshed
Auto-triage is fire-and-forget — it never blocks the build result.
Manual Trigger
From the Launches page, click the wand icon (✨) in the toolbar to open the auto-triage panel. The panel shows how many failing tests are not yet triaged.
Step 1 — Preview: Click Preview to scan failing tests against the source pool. The results table shows each match — test name, matched investigation, cause, match type, and a Permanent checkbox. No records are created yet.
Step 2 — Apply: Review the matches. Check Permanent on any message or stack match you want to promote immediately. Click Apply (or Apply (N) to apply only selected rows) to create the auto-triage records.
Manual trigger is useful when you just created a new investigation and want to immediately propagate it to other failing tests, or when you want to preview and selectively apply matches before enabling automatic triggering.
Promoting a Match
Promotion converts a time-limited message or stack match (TTL > 0) into a permanent one (TTL = -1).
| When | How |
|---|---|
| At apply time | In the manual trigger preview table, check the Permanent checkbox before clicking Apply. Token matches are always permanent and show a lock icon instead. |
| After creation | On the Investigated Tests page, filter to auto-triaged records. Each message/stack match shows a Promote to permanent button. A Promote All button is also available per source group. |
A promoted record:
- Never expires (TTL = -1)
- Can serve as a permanent source for future auto-triage matches
- Is treated identically to a token match in terms of expiry
Promote matches for failures caused by long-lived infrastructure issues or known external dependencies. This prevents the same investigation from needing to be re-created after the Review TTL expires.
Reruns Exclusion
Auto-triage skips tests with status = PASS, is_rerun = true. If a test failed but passed on retry, there is no need to propagate a prior failure investigation. Only tests that are still failing at the end of all retry attempts are candidates.
Worked Example
Settings: Enabled, Review TTL = 5 days, Source age limit = 90 days.
Existing investigations (sources):
| Source UID | Type | Token | Error message | Caused by | Tracking | TTL |
|---|---|---|---|---|---|---|
| checkout/pay-btn | Manual | — | Expected element .pay-btn to be visible | UI regression | PROJ-1234 | n/a (permanent) |
| auth/login-flow | Auto-triage | tok_abc123 | — | Infra: auth service down | OPS-777 | -1 (permanent) |
| search/results-count | Auto-triage | — | AssertionError: expected 10 but got 0 | Data seed missing | QA-42 | Expires in 3 days |
Build B20 fails. Failing tests after rerun exclusion:
| Test UID | Token | Error message | Already investigated? |
|---|---|---|---|
| cart/add-item | — | Expected element .pay-btn to be visible | No |
| checkout/confirm | tok_abc123 | — | No |
| search/pagination | — | AssertionError: expected 10 but got 0 | No |
| auth/logout | — | Timeout after 5000ms | Yes (active manual) |
Matching:
- cart/add-item — no token; message matches
checkout/pay-btn. Type:exact. Auto-triage created: cause = "UI regression", tracking = PROJ-1234, TTL = 5 days. - checkout/confirm — token
tok_abc123matchesauth/login-flow. Type:token. Auto-triage created: cause = "Infra: auth service down", tracking = OPS-777, TTL = -1 (permanent). - search/pagination — no token; message matches
search/results-count. Source is auto-triage with 3 days remaining — not expired. Type:exact. Auto-triage created: cause = "Data seed missing", tracking = QA-42, TTL = 5 days. - auth/logout — already investigated (active manual record). Skipped entirely.
After 5 days: The cart/add-item and search/pagination auto-triage records expire. On the next build where these tests fail, they are re-evaluated. If checkout/pay-btn's manual investigation still exists and the error message is the same, they will be re-triaged automatically.
checkout/confirm is not affected — its token match is permanent.
Next: AI Analysis →