Skip to content

Regressions and failures

The Failures area of the app is where uploaded regressions show up. It lists every regression in the current project, and for each one shows the pass/fail picture, the signatures its failures fell into, the failing jobs, and their logs. This page walks through those screens and explains every number on them.

To get there, select Failures in the sidebar. Everything you see belongs to the project selected in the project switcher.

The regressions list

The list shows the project's 50 most recent regressions, newest first.

Column What it shows
Regression The name you passed with --name. Below it: the branch and the first 10 characters of the revision, or the free-text project label of a pasted regression. Select the name to open the regression.
Status live, complete, or aborted (see below).
Jobs All jobs in the regression: passing, failing, and skipped.
Pass rate Passing jobs as a share of all jobs. Skipped jobs count toward the total.
Failures Failing jobs that have been analyzed and grouped into a signature.
Signatures Distinct signatures hit by this regression.
Started The regression's start time: the earliest JUnit <testsuite timestamp>, or the time it was opened.

A job counts as failing when its status is fail, error, timeout, or killed.

Note

The app shows only the 50 most recent regressions. To reach older ones, use the API: GET /v1/regressions accepts limit (up to 200) and offset.

Regression statuses

stateDiagram-v2
    [*] --> live: upload opens the regression
    live --> complete: upload finishes
    live --> aborted: marked aborted (API)
    complete --> live: same external id uploaded again
    aborted --> live: same external id uploaded again
Status Meaning
live Jobs may still arrive. The regression page updates by itself as batches are processed.
complete The upload finished.
aborted Someone marked the run as abandoned through the API (POST /v1/ingest/regressions/{id}/complete with {"status": "aborted"}). The CLI never does this.

Uploading again with the same --external-id reopens a complete or aborted regression, adds the new jobs, and completes it again.

The regression page

The header shows the regression's name and status, then its branch, the first 12 characters of the revision, the start time, and a CI build link if the upload included --ci-url.

While the regression is live, the page refreshes itself as each batch is processed, so you can watch the counts climb during a long upload.

Summary cards

Card What it shows
Jobs All jobs in the regression.
Pass rate Passing jobs as a share of all jobs.
Failing jobs Jobs with status fail, error, timeout, or killed.
Signatures Distinct signatures hit by this regression.

By component and failing tests

Two tables break the regression down:

  • By component groups jobs by their component property, worst first. Jobs without a component appear under โ€”.
  • Failing tests lists tests with at least one failing job, worst first (up to 50).

Both show Jobs, Failed, and Pass rate per row. Set the component property in your JUnit to get a useful component breakdown; see Properties.

The summary cards and these tables appear only for regressions uploaded with the CLI. A regression created with the paste form has failures but no jobs.

Signatures

Each row is a signature that at least one of this regression's failures fell into, with the most hits first. A signature groups failures with the same fingerprint across the whole project, not only this regression. See Signatures.

Column What it shows
Signature The signature's title. Select it to open the signature. Badges below the title: new (first seen in this regression) and recurred (came back after an absence, see below).
Class The signature's classification.
Hits here Failing jobs in this regression that fell into the signature.
Tests Distinct tests in this regression that hit it.
All-time Hits across every regression in the project.
Status The signature's triage status. See Working a signature.
Shortest failing job Links to the quickest failure to reproduce: fewest cycles (from the job's cycles property) and fastest (from its runtime). When one job is both, a single shortest log link appears.
Root cause The state of the latest root-cause run and a view run โ†’ link, if one exists.

A signature counts as recurred in a regression when it had disappeared: at least one complete regression on the same branch ran between its previous sighting and this one without hitting it. Upload every run, including green ones, and pass --branch, so this badge is reliable.

While a regression is live and nothing has been grouped yet, the table reads "No failures bucketed yet." A finished regression without failures reads "No failing signatures."

Failing jobs

This table lists the first 50 failing jobs, with a note such as "First 50 of 212." when there are more.

Column What it shows
Test The test name, with a log link if a log was uploaded for the job.
Status fail, error, timeout, or killed.
Kind The failure kind found in the log, such as assertion or timeout. See Failure kinds.
Seed The job's seed.
Signature The signature the job fell into. It reads pending until the job's batch has been processed.

The full job list, including passing jobs, is available through the API:

curl -H "Authorization: Bearer $DVFORGE_TOKEN" \
  "https://api.daruur.cloud/v1/regressions/<regression id>/jobs?failing=true&limit=200&offset=0"

Use job_status=pass (or any other status) instead of failing=true to filter by one status. limit can be at most 200. The regression id is the last part of the regression page's address.

The log viewer

Selecting a log link, or one of the shortest-job links, opens the job's log at /logs/<job id>. The header shows the job id, the log's SHA-256, and where the log came from:

  • stored by DV Forge: the project uploads directly to DV Forge, which keeps the log.
  • fetched from your data plane ยท not stored by DV Forge: the log lives in your data plane. DV Forge fetched it just now to show it to you and doesn't keep a copy.

The viewer shows up to the first 2,097,152 characters of the log. Longer logs are marked truncated.

Each log is fetched from wherever it was stored at upload time. If a project moves to a data plane later, its older logs are still shown from DV Forge.

When the data plane doesn't answer

If the log lives in a data plane that is offline, the viewer shows:

This log lives in your data plane, which didn't answer. Check that it is running and can reach DV Forge, then reload.

The message appears at once when the data plane hasn't sent a heartbeat in the last 90 seconds, or has been revoked. Otherwise DV Forge waits up to 30 seconds for the data plane to send the log before showing it. Ask your administrator to check the data plane's status on the Data planes page.

A job that has no log opens a "not found" page.

Deleting a regression

Deleting a regression is available through the API only. It requires the member role or higher; a project token with the ingest:write scope can also do it.

curl -X DELETE -H "Authorization: Bearer $DVFORGE_TOKEN" \
  "https://api.daruur.cloud/v1/regressions/<regression id>"

A successful delete returns HTTP 204. The regression's jobs and failures are removed. The signatures they hit stay, because signatures are project-wide, but their counts are recalculated. A signature left with no hits at all is hidden from lists.

Warning

Deleting can't be undone. To replace a bad upload, delete the regression and upload again, or upload under a new --external-id.

Pasting logs with Ingest regression

The Ingest regression button on the regressions list opens a form for pasting failing logs by hand. It's a quick way to see how DV Forge groups a handful of failures, without setting up the CLI.

  1. Enter a Name and, optionally, a Project label. The label is free text shown in the list; it doesn't select a project.
  2. Paste one failing log into each Failure card. Select Add failure for more cards, up to 2,000. The optional Scenario hint field links a failure to one of the project's Scenarios; leave it empty for your own logs.
  3. Select Ingest + cluster.

Load HMAC sample fills the form with a sample set of failing logs.

A pasted regression is created as complete. It has failures and signatures, but no jobs: it has no summary cards, no breakdowns, no failing jobs table, and no pass rate.

Not available for projects on a data plane

If the current project is bound to a data plane, the form is refused with "this project ingests through its data plane; raw logs stay there โ€” point the uploader at it instead of pasting logs". Pasting would send raw logs to DV Forge, which a data plane exists to prevent. Upload with the CLI through the data plane instead.

For real regressions, use the dvforge CLI: it records every job, pass and fail, which the paste form can't.