Simulators and log formats¶
When a job fails, DV Forge reads its log, finds the line that best explains the failure, and records a structured failure analysis. That analysis is what groups the failure into a signature and what you see on the signature and regression pages. This page lists the log formats DV Forge understands, how it picks a parser for each log, what the analysis contains, and how redaction works when a data plane does the parsing.
Which logs are analyzed, and where¶
DV Forge analyzes every job whose status is fail, error, timeout, or
killed. Logs of passing and skipped jobs are stored if you send them, but
they aren't parsed.
The analysis runs wherever the raw log lives:
- Projects without a data plane: in DV Forge, right after the batch arrives.
- Projects bound to a data plane: inside the data plane, in your network. Only the analysis, after redaction, is sent to DV Forge.
If a failing job has no log, DV Forge analyzes its one-line failure message from the JUnit instead.
Supported formats¶
| Parser id | Recognizes |
|---|---|
questa |
Questa and ModelSim: vlog, vcom, and vopt compile output, and vsim transcripts (** Error, ** Fatal, # prefixed lines, Time: continuation lines). |
xcelium |
Xcelium and Incisive: xrun, xmvlog, xmelab, xmsim (and the nc* equivalents) messages in the *E,CODE (file,line): text form. |
vcs |
VCS: compile diagnostics (Error-[CODE] blocks), simv runtime output, SVA failures, $error reports, and the simulation report trailer. |
verilator |
Verilator: compile and lint diagnostics (%Error-CODE, %Warning-CODE) and Verilated-model runtime output ([time] %Error, assertion failures, $stop). |
systemc |
SystemC (IEEE 1666) sc_report output: Error:, Warning:, Fatal: reports with their In file: and In process: lines. |
uvm |
UVM report-server output from any simulator, including report servers that put (file:line) after the time, and the UVM report summary. |
dvforge_bundle |
The DV Forge failure export format: field : value header lines (test, seed, exit_status, …) followed by tagged, timestamped log lines. |
generic |
The fallback for everything else: lines with log levels (ERROR, Warning:) or error, fatal, assertion, or timeout wording. |
Every simulator parser also understands UVM messages, so a UVM testbench run
on Questa is fully read by the questa parser: its UVM messages and summary
plus the simulator's own errors.
Across all formats, DV Forge also recognizes common signs that the job's environment failed rather than the design. It records them with these message ids:
| Message id | Typical wording |
|---|---|
KILLED |
A bare Killed line, exit code 137, SIGKILL, the OOM killer |
OOM |
out of memory, std::bad_alloc, cannot allocate memory |
LICENSE |
License checkout failures, license server unreachable |
DISK_FULL |
No space left on device, Disk quota exceeded |
JOB_TIMEOUT |
Batch-scheduler run limits (TERM_RUNLIMIT, DUE TO TIME LIMIT) |
CRASH |
Segmentation faults, core dumped, stack traces, internal tool errors |
The patterns are strict, so testbench text about the design ("bus error response", "killed 3 transactions") isn't mistaken for the simulator dying. A license retry or similar message that appears before the simulation finished normally is kept as a warning, not treated as the cause.
How a parser is picked¶
- A parser hint. If the job carries a
parserproperty in the JUnit (see Properties) whose value is one of the parser ids above, that parser is used. - Detection. Otherwise, every parser scores how confident it is from the
first 8 KB of the log, based on tool banners and message shapes. The
highest score wins. Simulator parsers outrank the plain
uvmparser when they see their own markers. - Fallback. If no parser recognizes the log, the
genericparser reads it. A parser that runs into trouble also falls back togeneric, so every log yields an analysis.
Tip
The simulator property is recorded with the job, but it doesn't choose
the parser. If detection picks the wrong parser (for example, because your
wrapper script prints a long preamble before the simulator's banner), set
the parser property instead.
What a failure analysis contains¶
| Field | What it is |
|---|---|
| Headline | The message the failure is grouped by. This is the first message at the highest severity in the log (error or fatal). After-the-fact summaries, such as Verilator's Verilog $stop or Exiting due to N error(s), Questa's Error loading design, or Xcelium's parse and elaboration summaries, are used only when nothing else is there. |
| Error line | The headline's original log line, or the JUnit failure message when no parser found a headline. Shown with each failure and used when splitting a signature. |
| Failure kind | A category: see the table below. Shown in the Kind column of a regression's failing jobs. |
| Messages | The warnings and errors from the log, in log order, each with severity, text, message id, source file and line, simulation time, and hierarchy where the tool prints them. Up to 50; if there are more, the most severe are kept. Informational messages are never kept. |
| Simulation time | When the failure happened, normalized to femtoseconds. |
| Parser | The parser id and version that read the log. |
| Excerpt | The log lines around the headline: 25 lines before and after it, or the last 50 lines if there is no headline. Up to 64 KiB. Data planes send 16 KiB by default; see Redaction. |
| Signature key | A fingerprint of the headline with run-specific details masked out: values, times, paths, instance indexes, and echoes of the test name and seed. Failures with the same key share a signature. See Bucketing settings. |
Failure kinds¶
| Kind | Meaning |
|---|---|
assertion |
An assertion (SVA, immediate assert) failed. |
scoreboard |
A scoreboard reported the failure. |
mismatch |
A compare failed outside a scoreboard ("expected … got …"). |
timeout |
A testbench timeout, watchdog, deadlock, or scheduler run limit. |
timing |
A timing check ($setup, $hold, …) was violated. |
coverage |
A coverage check failed. |
compile |
Compilation or elaboration failed; the simulation never ran. |
crash |
The simulator or a library crashed. |
oom |
The job ran out of memory or was killed. |
infra |
An environment problem, such as a license or full disk. |
unknown |
The log shows an error that doesn't fit any category above. |
When no parser finds a headline¶
Some logs have no recognizable error message, for example when a job was cut short. DV Forge then groups the failure by, in order:
- The failure message from the JUnit, if there is one.
- The most telling line in the log, searched for markers such as
UVM_FATAL,UVM_ERROR,fatal:,error:,assertion,mismatch,timeout, anddeadlock, or GNU make's*** [...] Errorline. Run headers such asstatus : FAILEDand echoed simulator command lines are skipped. - The last three lines of the log.
A failing job with neither a log nor a message is grouped as "failed without a log or message".
Give DV Forge a good message
If your logs are often missing or truncated, put a meaningful one-line
summary in the JUnit <failure message="…">. It's the next best thing to
a parseable log.
Simulation times without a unit¶
UVM and Verilator often print times without a unit. DV Forge infers the unit
from the log itself (a timescale echo, the vsim -t resolution, or the final
time). When nothing indicates a unit, it assumes nanoseconds.
Size limits¶
Logs are untrusted input, so the parsers protect themselves. Lines longer than 16 KiB are truncated before matching, message text is capped at 4 KiB, and error lines at 8 KiB. Logs over 256 MiB uncompressed are refused at upload.
Redaction on a data plane¶
When a project uses a data plane, the analysis is computed in your network
and redacted there, before anything is sent to DV Forge. Your administrator
sets the redaction rules on the data plane: DVFORGE_DP_REDACT_PATTERNS, a
JSON array of regular expressions whose every match becomes [redacted], and
DVFORGE_DP_MAX_EXCERPT_CHARS and DVFORGE_DP_EXCERPT_CONTEXT_LINES, which
limit the excerpt (16 KiB and 25 lines on either side of the headline by
default; 0 characters sends no log lines). See the
configuration reference.
The patterns apply to every free-text part of the analysis: the headline, the error line, the signature title and template, the excerpt, and each message's text, original line, hierarchy, file name, and message id. They also apply to the job's one-line failure message.
Redaction can change grouping
The signature key is recomputed from the redacted headline, because that's the text DV Forge stores. If a pattern erases the only difference between two failures, they land in the same signature. Redact secrets and names you must protect, not details that distinguish one failure from another.
What redaction doesn't cover
Job metadata is forwarded as sent: test names, seeds, component, simulator, build target, configuration, plusargs, and host. If those contain sensitive values, change what your JUnit reports.
For projects without a data plane, logs are stored in DV Forge and no redaction is applied. To keep raw logs in your network, see Data planes.
In-house tools¶
If your logs come from a tool none of the parsers recognize, the generic
parser still produces an analysis from error and warning wording. A data plane
can also load additional parsers packaged as Python plugins. Ask your DV Forge
contact if you need one.