Skip to content

Sending results

DV Forge learns about your regressions from your CI. After each run, a single dvforge upload command sends the JUnit results and the matching simulation logs. DV Forge records every test as a job, parses the failing ones, and groups them into signatures. This page explains how an upload works and what DV Forge does with it. The rest of this section covers the CLI, CI setup, supported log formats, and the Failures screens where the results show up.

Before you start

You need three things:

  • The dvforge CLI on the machine that runs your regression. Your DV Forge contact provides it, either as a Python package or as a standalone binary for RHEL, Rocky, or Alma Linux 8 and 9. See The dvforge CLI.
  • A project token with the ingest:write scope. It looks like dvf_1a2b3c4d_… and belongs to exactly one project. A project admin creates it. See Projects, roles and tokens.
  • The upload URL. This is https://api.daruur.cloud, unless your project sends results through a data plane.

How an upload works

One dvforge upload sends one regression:

  1. Open the regression. The CLI sends the regression's name, its external id (usually your CI build id), and optional branch and revision details. DV Forge creates the regression, or finds the one that already has this external id. The regression is now live.
  2. Send the jobs in batches. Every JUnit <testcase> becomes one simulation job. That includes passing and skipped tests, not only failures, so DV Forge can report pass rates and know when a failure stops happening. Jobs go up in batches of up to 5,000.
  3. Upload the logs each batch needs. Before each batch, the CLI uploads the log files it matched to that batch's tests. Each log is stored under the SHA-256 hash of its uncompressed content. A log that DV Forge already has is skipped.
  4. Complete the regression. The regression moves from live to complete.
sequenceDiagram
    participant CI as CI job (dvforge upload)
    participant DV as DV Forge
    CI->>DV: Open regression (name, external id)
    DV-->>CI: Regression id, status live
    loop Each batch of up to 5,000 jobs
        CI->>DV: Upload the batch's logs (skipped if already stored)
        CI->>DV: Send the job batch
        DV-->>CI: Accepted
    end
    CI->>DV: Complete the regression
    Note over DV: Failing jobs are parsed<br/>and grouped into signatures

DV Forge processes each batch in the background. It parses the log of every failing job (fail, error, timeout, or killed), picks the line that explains the failure, and assigns the job to a project-wide signature. While a regression is live, its page in the app updates as batches finish. To make the CLI wait for that processing, pass --wait.

Upload only the logs you need

DV Forge parses only the logs of failing jobs, and the app links only to those. Add --only-failing-logs to skip uploading logs of passing tests. The passing jobs themselves are still recorded.

Re-running is always safe

Every step of an upload can be repeated without creating duplicates:

  • Logs are stored by content hash, so uploading the same file twice stores it once.
  • The regression is matched by its external id. Running the upload again with the same --external-id adds to the same regression.
  • Each job has a stable id derived from the regression's external id, the test name, and the seed. Each batch carries a key derived from its content. A batch that already arrived is recognized and not counted twice.

So if an upload is interrupted (a network outage, a killed CI job, Ctrl-C), run the same command again. It picks up where it stopped. Use your CI's build id as the external id, not the attempt number, so that a retried CI job lands in the same regression. For details, see Resuming an interrupted upload.

Regression statuses

A regression is live while jobs may still arrive, complete once the upload finishes, or aborted if someone marks it abandoned through the API. Uploading again with the same external id reopens it. See Regression statuses for the details, and Uploading one regression in parts to split one regression across several uploads.

If your project uses a data plane

Some organizations run a DV Forge data plane inside their own network. When a project is bound to a data plane, CI uploads go to the data plane's URL instead of https://api.daruur.cloud. Nothing else about the command changes:

export DVFORGE_URL=http://dvforge-dp.example.internal:8100   # your data plane
export DVFORGE_TOKEN=dvf_...                                 # the same project token
dvforge upload --name "nightly 1234" --external-id ci-1234 --junit out/results.xml \
  --logs 'out/logs/**/*.log' --only-failing-logs

The data plane stores the raw logs in your network and parses the failures there. It forwards only job results and a redacted analysis of each failure. Raw logs never leave your network. When someone opens a log in the app, DV Forge fetches it from the data plane on demand and doesn't store it.

Warning

A project bound to a data plane accepts uploads only through that data plane. If you point the CLI at https://api.daruur.cloud instead, the upload is refused with the message "this project ingests through its data plane; point the uploader at it" (exit code 3).