Skip to content

Simulators and log formats

When a job fails, DV Forge reads its log, finds the line that best explains the failure, and records a structured failure analysis. That analysis is what groups the failure into a signature and what you see on the signature and regression pages. This page lists the log formats DV Forge understands, how it picks a parser for each log, what the analysis contains, and how redaction works when a data plane does the parsing.

Which logs are analyzed, and where

DV Forge analyzes every job whose status is fail, error, timeout, or killed. Logs of passing and skipped jobs are stored if you send them, but they aren't parsed.

The analysis runs wherever the raw log lives:

  • Projects without a data plane: in DV Forge, right after the batch arrives.
  • Projects bound to a data plane: inside the data plane, in your network. Only the analysis, after redaction, is sent to DV Forge.

If a failing job has no log, DV Forge analyzes its one-line failure message from the JUnit instead.

Supported formats

Parser id Recognizes
questa Questa and ModelSim: vlog, vcom, and vopt compile output, and vsim transcripts (** Error, ** Fatal, # prefixed lines, Time: continuation lines).
xcelium Xcelium and Incisive: xrun, xmvlog, xmelab, xmsim (and the nc* equivalents) messages in the *E,CODE (file,line): text form.
vcs VCS: compile diagnostics (Error-[CODE] blocks), simv runtime output, SVA failures, $error reports, and the simulation report trailer.
verilator Verilator: compile and lint diagnostics (%Error-CODE, %Warning-CODE) and Verilated-model runtime output ([time] %Error, assertion failures, $stop).
systemc SystemC (IEEE 1666) sc_report output: Error:, Warning:, Fatal: reports with their In file: and In process: lines.
uvm UVM report-server output from any simulator, including report servers that put (file:line) after the time, and the UVM report summary.
dvforge_bundle The DV Forge failure export format: field : value header lines (test, seed, exit_status, …) followed by tagged, timestamped log lines.
generic The fallback for everything else: lines with log levels (ERROR, Warning:) or error, fatal, assertion, or timeout wording.

Every simulator parser also understands UVM messages, so a UVM testbench run on Questa is fully read by the questa parser: its UVM messages and summary plus the simulator's own errors.

Across all formats, DV Forge also recognizes common signs that the job's environment failed rather than the design. It records them with these message ids:

Message id Typical wording
KILLED A bare Killed line, exit code 137, SIGKILL, the OOM killer
OOM out of memory, std::bad_alloc, cannot allocate memory
LICENSE License checkout failures, license server unreachable
DISK_FULL No space left on device, Disk quota exceeded
JOB_TIMEOUT Batch-scheduler run limits (TERM_RUNLIMIT, DUE TO TIME LIMIT)
CRASH Segmentation faults, core dumped, stack traces, internal tool errors

The patterns are strict, so testbench text about the design ("bus error response", "killed 3 transactions") isn't mistaken for the simulator dying. A license retry or similar message that appears before the simulation finished normally is kept as a warning, not treated as the cause.

How a parser is picked

  1. A parser hint. If the job carries a parser property in the JUnit (see Properties) whose value is one of the parser ids above, that parser is used.
  2. Detection. Otherwise, every parser scores how confident it is from the first 8 KB of the log, based on tool banners and message shapes. The highest score wins. Simulator parsers outrank the plain uvm parser when they see their own markers.
  3. Fallback. If no parser recognizes the log, the generic parser reads it. A parser that runs into trouble also falls back to generic, so every log yields an analysis.

Tip

The simulator property is recorded with the job, but it doesn't choose the parser. If detection picks the wrong parser (for example, because your wrapper script prints a long preamble before the simulator's banner), set the parser property instead.

What a failure analysis contains

Field What it is
Headline The message the failure is grouped by. This is the first message at the highest severity in the log (error or fatal). After-the-fact summaries, such as Verilator's Verilog $stop or Exiting due to N error(s), Questa's Error loading design, or Xcelium's parse and elaboration summaries, are used only when nothing else is there.
Error line The headline's original log line, or the JUnit failure message when no parser found a headline. Shown with each failure and used when splitting a signature.
Failure kind A category: see the table below. Shown in the Kind column of a regression's failing jobs.
Messages The warnings and errors from the log, in log order, each with severity, text, message id, source file and line, simulation time, and hierarchy where the tool prints them. Up to 50; if there are more, the most severe are kept. Informational messages are never kept.
Simulation time When the failure happened, normalized to femtoseconds.
Parser The parser id and version that read the log.
Excerpt The log lines around the headline: 25 lines before and after it, or the last 50 lines if there is no headline. Up to 64 KiB. Data planes send 16 KiB by default; see Redaction.
Signature key A fingerprint of the headline with run-specific details masked out: values, times, paths, instance indexes, and echoes of the test name and seed. Failures with the same key share a signature. See Bucketing settings.

Failure kinds

Kind Meaning
assertion An assertion (SVA, immediate assert) failed.
scoreboard A scoreboard reported the failure.
mismatch A compare failed outside a scoreboard ("expected … got …").
timeout A testbench timeout, watchdog, deadlock, or scheduler run limit.
timing A timing check ($setup, $hold, …) was violated.
coverage A coverage check failed.
compile Compilation or elaboration failed; the simulation never ran.
crash The simulator or a library crashed.
oom The job ran out of memory or was killed.
infra An environment problem, such as a license or full disk.
unknown The log shows an error that doesn't fit any category above.

When no parser finds a headline

Some logs have no recognizable error message, for example when a job was cut short. DV Forge then groups the failure by, in order:

  1. The failure message from the JUnit, if there is one.
  2. The most telling line in the log, searched for markers such as UVM_FATAL, UVM_ERROR, fatal:, error:, assertion, mismatch, timeout, and deadlock, or GNU make's *** [...] Error line. Run headers such as status : FAILED and echoed simulator command lines are skipped.
  3. The last three lines of the log.

A failing job with neither a log nor a message is grouped as "failed without a log or message".

Give DV Forge a good message

If your logs are often missing or truncated, put a meaningful one-line summary in the JUnit <failure message="…">. It's the next best thing to a parseable log.

Simulation times without a unit

UVM and Verilator often print times without a unit. DV Forge infers the unit from the log itself (a timescale echo, the vsim -t resolution, or the final time). When nothing indicates a unit, it assumes nanoseconds.

Size limits

Logs are untrusted input, so the parsers protect themselves. Lines longer than 16 KiB are truncated before matching, message text is capped at 4 KiB, and error lines at 8 KiB. Logs over 256 MiB uncompressed are refused at upload.

Redaction on a data plane

When a project uses a data plane, the analysis is computed in your network and redacted there, before anything is sent to DV Forge. Your administrator sets the redaction rules on the data plane: DVFORGE_DP_REDACT_PATTERNS, a JSON array of regular expressions whose every match becomes [redacted], and DVFORGE_DP_MAX_EXCERPT_CHARS and DVFORGE_DP_EXCERPT_CONTEXT_LINES, which limit the excerpt (16 KiB and 25 lines on either side of the headline by default; 0 characters sends no log lines). See the configuration reference.

The patterns apply to every free-text part of the analysis: the headline, the error line, the signature title and template, the excerpt, and each message's text, original line, hierarchy, file name, and message id. They also apply to the job's one-line failure message.

Redaction can change grouping

The signature key is recomputed from the redacted headline, because that's the text DV Forge stores. If a pattern erases the only difference between two failures, they land in the same signature. Redact secrets and names you must protect, not details that distinguish one failure from another.

What redaction doesn't cover

Job metadata is forwarded as sent: test names, seeds, component, simulator, build target, configuration, plusargs, and host. If those contain sensitive values, change what your JUnit reports.

For projects without a data plane, logs are stored in DV Forge and no redaction is applied. To keep raw logs in your network, see Data planes.

In-house tools

If your logs come from a tool none of the parsers recognize, the generic parser still produces an analysis from error and warning wording. A data plane can also load additional parsers packaged as Python plugins. Ask your DV Forge contact if you need one.