Skip to content

The dvforge CLI

dvforge upload sends one regression to DV Forge: your JUnit results become jobs, and the simulation logs it can match to those jobs are uploaded alongside them. This page covers installing and configuring the CLI, how it reads JUnit and finds logs, and how to control batching, waiting, and resuming. For a flag-by-flag list, see the CLI reference.

Getting the CLI

Your DV Forge contact provides the CLI in one of two forms:

  • A Python package for Python 3.11 or later. Its only dependency is the httpx HTTP library. Install it into a virtual environment or with pipx:

    pipx install /path/to/the-dvforge-cli-package
    # or
    python3.11 -m venv ~/.dvforge && ~/.dvforge/bin/pip install /path/to/the-dvforge-cli-package
    
  • A standalone binary for RHEL, Rocky, and Alma Linux 8 and 9 on x86_64, for CI agents that don't have Python 3.11. It needs nothing else installed:

    install -m 0755 dvforge-linux-x86_64 /usr/local/bin/dvforge
    

Check the installation:

dvforge --version

Agents that mount /tmp noexec

At start-up the standalone binary unpacks itself into $TMPDIR (default /tmp). If /tmp is mounted noexec, set TMPDIR to a directory that allows execution.

Configuration

The CLI reads three settings, from flags or environment variables. A flag overrides its environment variable.

Setting Environment variable Flag Default
Upload URL DVFORGE_URL --url http://localhost:8000
Project token DVFORGE_TOKEN --token none
Project DVFORGE_PROJECT --project ORG/SLUG the token's project
  • Set DVFORGE_URL to https://api.daruur.cloud, or to your data plane's URL if your project uses one. The default only suits a local test server.
  • Set DVFORGE_TOKEN to a project token with the ingest:write scope. Tokens look like dvf_<8 hex characters>_<secret>. A token with only the read scope is refused (exit code 3). See Projects, roles and tokens.
  • You rarely need DVFORGE_PROJECT: a project token already names its project. If you do set it, it must name the token's project.

Prefer environment variables for the token

Command-line flags are visible to other users in ps. Put the token in DVFORGE_TOKEN, ideally from your CI's secret store. The CLI sends it as an Authorization: Bearer header and never prints it.

A worked example

Suppose your regression writes out/results.xml and one log per test under out/logs/. After the run:

export DVFORGE_URL=https://api.daruur.cloud
export DVFORGE_TOKEN=dvf_1a2b3c4d_...

dvforge upload \
  --name "nightly 1234" \
  --external-id "ci-1234" \
  --junit out/results.xml \
  --logs 'out/logs/**/*.log' \
  --only-failing-logs \
  --branch main \
  --rev "$(git rev-parse HEAD)" \
  --scm git

The CLI writes progress and warnings to stderr and a summary to stdout:

dvforge: batch 1/1: 1000 jobs, 42 logs
regression 7c0d… (created): nightly 1234 [ci-1234]
jobs sent: 1000 in 1 batch(es) - pass 958, fail 33, error 4, timeout 5
logs: 42 uploaded (42 new, 0 already stored) (958 passing logs skipped)
status: complete

On a re-run the first line says (updated) instead of (created), batches that already arrived are reported as already ingested, and logs that are already stored count as already stored.

To check what the CLI would send without sending anything, add --dry-run. It needs no token and makes no network calls, but it still reads the JUnit, matches and hashes the logs, and prints every request it would make:

dry run: nothing will be sent
POST /v1/ingest/regressions
  {"branch": "main", "external_id": "ci-1234", "name": "nightly 1234", ...}
PUT /v1/blobs/787ed16a…  <- out/logs/crc_test__77.log (60 B, gzip)
POST /v1/ingest/regressions/<id>/jobs  batch 1/1: 3 jobs  Idempotency-Key: 2695a43d…
  first job: {"status": "pass", "test": "fifo.fifo_smoke_test", ...}
POST /v1/ingest/regressions/<id>/complete  {"status": "complete"}
would send 3 jobs in 1 batch(es) (pass 1, fail 1, timeout 1); 2 logs to upload (1 passing logs skipped)

How JUnit becomes jobs

Pass one or more JUnit XML files with --junit (repeat the flag for several files; they all go into the same regression). The root element must be <testsuites>, <testsuite>, or <testcase>. Suites may be nested. Every <testcase> becomes one job.

Test name

The job's test name is classname.name. If the testcase has no classname (the class attribute is also accepted), if the classname equals the name, or if the name already starts with classname., the test name is just name. Test names are clipped to 255 characters. A testcase with neither a name nor a classname is skipped.

Status

In the testcase Job status
An <error> element (it outranks <failure>) error
A <failure> element fail
A <skipped> element skipped
None of these pass
A <failure> or <error> whose type contains timeout or timed out timeout
A <failure> or <error> whose type contains killed, or is oom or sigkill killed
A status property set to pass, fail, error, timeout, killed, or skipped that status (overrides everything above)

For a job that didn't pass, the failure message is the first line of the <failure> or <error> element's message attribute, or else the first line of its text, up to 2,000 characters. If a failing job has no message and no log, the element's type is used as its message.

Properties

DV-specific facts come from <property> elements. Names are case-insensitive. Properties set on a <testsuite> apply to every testcase inside it, including nested suites. A property on the testcase overrides the suite's value.

Property Job field Notes
seed, random_seed, sv_seed, ntb_random_seed Seed Checked in that order. Also read from a seed attribute on the testcase, or from an explicit seed_1234, seed=1234, or seed-1234 in the test name. Up to 255 characters.
component Component Used for the By component breakdown. Up to 255 characters.
simulator Simulator Falls back to --simulator. Up to 64 characters. Recorded only; it doesn't choose the log parser.
simulator_version Simulator version Up to 64 characters.
parser Parser hint Names the log parser to use, for example questa. See Simulators and log formats.
build_target Build target Falls back to the regression's --build-target. Up to 255 characters.
config Configuration Up to 1,024 characters.
host Host Falls back to the <testsuite hostname> attribute. Up to 255 characters.
plusargs Plusargs Split on whitespace, up to 200 entries.
cycles Simulated cycles Integer, 0 to 2⁶³−1. Used to pick a signature's fewest cycles log.
sim_time_fs Simulation time Integer femtoseconds, 0 to 2⁶³−1.
exit_code Exit code Signed 32-bit integer. Unsigned forms such as 4294967295 or 0xC0000005 are converted to their signed value.
status Status Overrides the status from the JUnit elements (see above).
log, logfile, log_file, log_path, logpath, sim_log Log file Points at this test's log. Read from the testcase only, never inherited from a suite.

Integer properties accept decimal, 0x hexadecimal, and _ digit separators. A value that isn't a number, or is out of range, is dropped rather than failing the upload.

The CLI also reads these testcase attributes:

Attribute Job field
time Runtime in seconds. Dropped unless it is a finite number between 0 and 10⁹.
timestamp Start time (ISO 8601).

The regression's start time is the earliest <testsuite timestamp> across all the JUnit files. If there is none, DV Forge uses the time the regression was opened.

How logs are matched to tests

The CLI attaches at most one log file to each job, and uses each log file at most once. For each testcase, the first rule that finds a file wins.

1. An explicit pointer in the JUnit. One of the log properties above, a [[ATTACHMENT|path]] marker in <system-out> or <system-err>, or a line in those elements that is nothing but a path ending in .log or .log.gz (optionally prefixed log:). Relative paths are resolved against the JUnit file's directory first, then the current directory. Pointers work without --logs.

2. File names under --logs. Pass one or more --logs patterns. Quote them so the shell doesn't expand them; ** matches any depth, and a directory means every file under it. The CLI compares the testcase's name attribute (not the classname) with each file's base name, minus .gz and a .log, .txt, or .out extension. The match is case-insensitive and must fall on word boundaries, so test doesn't match contest_x.log. Candidates are ranked:

  1. Exactly the name and the seed, for example fifo_test__1234.log or fifo_test.1234.log.
  2. Contains the name and the seed, for example run_fifo_test_seed1234.log.
  3. Exactly the name (and the seed isn't in the file name, or the test has no seed).
  4. Contains the name.
  5. Otherwise, the last two parent directories contain the name (and the seed), for example regress/fifo_test/seed_1234/sim.log.

Better ranks claim files first, so fifo_test never takes fifo_test_long__1.log from fifo_test_long. If several files tie for a test's best rank, the CLI warns that the match is ambiguous and attaches nothing to that test. Files that no test claims are counted and ignored. A pattern that matches no files produces a warning.

Logs that are already gzipped on disk are sent as they are; others are compressed on the fly. In both cases the stored hash is that of the uncompressed content.

Choosing which logs to send

Flag Effect
--only-failing-logs Upload logs only for fail, error, timeout, and killed jobs. Recommended: DV Forge parses only failing logs.
--no-logs Send results only. Ignores --logs and the JUnit log pointers. Can't be combined with --logs or --only-failing-logs.
--max-log-mb N Skip, with a warning, any log larger than N MiB uncompressed. Default 256, which is also the server's limit.

A log that can't be read is skipped with a warning. A job whose log was skipped is still sent; it is bucketed by its failure message instead.

Batches and concurrency

Jobs go up in batches of --batch-size jobs (default and maximum 5,000). For each batch, the CLI first uploads the logs that batch references, then sends the batch. --concurrency sets how many logs are hashed and uploaded in parallel (default 8, maximum 64).

Network errors, HTTP 5xx responses, and HTTP 429 are retried up to five attempts per request, with exponential backoff and jitter. A Retry-After header from the server is honored, up to 60 seconds.

Waiting for processing

By default the CLI exits as soon as DV Forge has accepted every batch. Processing (parsing failures and assigning signatures) continues in the background. Add --wait to make the CLI poll each batch until it is processed:

  • If every batch finishes, the CLI completes the regression and exits 0.
  • If any batch fails to process, the CLI exits 4 and doesn't complete the regression.
  • If batches are still processing after --wait-timeout seconds (default 600), the CLI exits 4. The batches keep processing.

Use --wait when a later CI step reads the results back, for example through the API.

Uploading one regression in parts

Some flows produce one regression's results in several pieces, for example sharded runs on different machines. Upload each piece with the same --external-id and add --no-complete to every upload except the last. The regression stays live until the final upload completes it.

Jobs that repeat across parts

A job's identity is derived from the regression's external id, the test name, the seed, and how many earlier jobs in the same upload had that test and seed. If two parts both report the same test with the same seed, the second one is treated as a duplicate and dropped. Give each part distinct test and seed combinations.

You don't strictly need --no-complete: uploading to a completed regression reopens it. But leaving it open keeps the status accurate while parts are still arriving.

Resuming an interrupted upload

Rerun the exact same command. Every step is safe to repeat:

  • Before uploading a log, the CLI asks whether DV Forge already has it, and skips it if so.
  • The regression is found again by its external id.
  • Job ids and batch keys depend only on the inputs, never on time, so a resent batch is recognized as already ingested.

If you press Ctrl-C, the CLI stops with exit code 130 and the message "interrupted — rerun the same command to resume". After a server or network failure (exit code 4), it prints "rerun the same command to resume; completed batches are not re-ingested".

Keep these two rules in mind:

  • The first copy of a job wins. If an earlier attempt sent a job without its log, a rerun that now has the log doesn't attach it. To change jobs that already arrived, upload under a new --external-id.
  • Don't change log files mid-upload. If a log changes between hashing and upload, the CLI stops with an error. Rerun once the file is stable.

Uploading through a data plane

If your project is bound to a data plane, set DVFORGE_URL to the data plane's URL. The command, the token, and the output stay the same.

If the data plane can't reach DV Forge while you upload, it keeps your batches and the completion in a durable queue and forwards them in order once the connection is back. The CLI reports this and still exits 0:

dvforge: batch 2/3: 5000 jobs, 12 logs (spooled by the data plane)
dvforge: note: the control plane was unreachable; the data plane queued 2 batch(es) and the completion and forwards them in order once it is back
...
status: complete (spooled on the data plane)

Two limits apply:

  • --wait can't wait for queued batches, because they don't have a batch id yet. The CLI warns about this.
  • Opening the regression can't be queued, because the CLI needs the regression id it returns. If DV Forge is unreachable at that moment, the CLI retries and then exits 4.

Exit codes

Code Meaning What to do
0 Success, including --dry-run and uploads queued by a data plane. Nothing.
2 Usage error: bad or conflicting flags, a missing or unreadable JUnit file, malformed or unsafe XML, no testcases, or no credentials. Fix the command or the input. Nothing was sent.
3 Authentication or permission error (HTTP 401 or 403). Check DVFORGE_TOKEN and its scope. Also check DVFORGE_URL: a project bound to a data plane refuses direct uploads with a 403, and a data plane refuses projects not bound to it.
4 Server or network error after retries, any other API error, a batch that failed to process, or a --wait timeout. Rerun the same command to resume.
130 Interrupted (Ctrl-C). Rerun the same command to resume.

Limits and clipping

The CLI keeps an upload from being rejected over one oversized value:

What Limit When exceeded
--name, --external-id, --branch, --build-target 255 characters Usage error (exit 2)
--rev 128 characters Usage error (exit 2)
--ci-provider 64 characters Usage error (exit 2)
--ci-url 2,048 characters; must start with http:// or https:// Usage error for length; the server refuses other schemes (exit 4)
Test name, seed, component, build target, host 255 characters Clipped
Simulator, simulator version, parser hint 64 characters Clipped
Configuration 1,024 characters Clipped
Failure message 2,000 characters (first line only) Clipped
Plusargs 200 entries Extra entries dropped
Runtime 0 to 10⁹ seconds Dropped
Cycles, simulation time 0 to 2⁶³−1 Dropped
Exit code Signed 32-bit Dropped
Jobs per batch 5,000 Usage error if --batch-size is larger (exit 2)
Log size 256 MiB uncompressed (--max-log-mb); a pre-gzipped file over 64 MiB on disk Skipped with a warning

JUnit safety

JUnit files are written by the code under test, so the CLI treats them as untrusted. It refuses any file that contains a DOCTYPE or ENTITY declaration (exit code 2). Legitimate JUnit reports never need one. Deeply nested suites are handled without risk of a crash.