System Ground Truth: What Agents Must Know About Production Systems

An agent can explain every function in a repository and still misunderstand the product running in production.

That is because a production system is not a pile of artifacts. It is the behavior that emerges when code, configuration, data contracts, metrics, control flow, and operational practice run together. A recommendation policy may live partly in a guard clause, partly in a feature-store convention, and partly in a dashboard definition. Each artifact can be read correctly while the system-level conclusion is wrong.

This creates a problem underneath almost every serious agent evaluation. Before we can ask whether an agent can audit, troubleshoot, or improve a deployed system, we need a reference that says what that system actually does.

In our paper, System Ground Truth: Defining, Generating, and Evaluating What Agents Must Know About Deployed Systems, we define that reference, test two ways to generate it automatically, and measure how closely automated ground truth behaves like expert-authored ground truth across two real production recommendation systems.

The missing unit in agent evaluation

Most benchmarks start with a reference already in hand. Repository QA grades facts about files. Tool-use benchmarks grade whether an agent reaches a known goal state. LLM-as-judge systems compare an answer with an existing rubric. All of them assume someone has already decided what is true.

For a deployed production system, that reference is usually written manually by subject-matter experts. In our survey of forward-deployed engineers, writing, verifying, and anchoring one vetted claim took roughly 20 to 60 minutes. A useful benchmark can require hundreds of claims, must be rebuilt for every system, and starts drifting as soon as code, configurations, tables, or metrics change.

Automation is attractive, but only after choosing the right object. If extraction targets individual files, it can produce a precise benchmark for the wrong thing. The object an agent must understand is the running system those files compose.

A three-part theory of system ground truth

We organize system ground truth into three connected parts. The first says where a system sits. The second says which truths matter for a system in that position. The third says what qualifies as a valid, auditable answer.

1. Where the system sits

An Ontology of Industrial Systems locates a system along four dimensions. These coordinates determine what evidence exists and which failures matter.

• Problem shape: rank or retrieve, detect or classify, allocate or match, optimize or price, trigger or engage, forecast or estimate.
• Feedback loop: immediate behavior, session outcomes, delayed outcomes, transaction resolution, or no natural feedback.
• Consequence profile: reversible decisions through to irreversible ones.
• Stakeholder surface: single-user, two-sided, multi-stakeholder, or internal operations.

A feed ranker and a fraud detector should not be interrogated with the same template. The ranker needs truths about candidate coverage, objective composition, exploration, and latency. The fraud detector needs truths about class imbalance, label lag, thresholds, appeals, and asymmetric error costs. System shape changes which truths are load-bearing.

2. Which truths matter

The Essential Truths schema defines seven categories that recur across production systems, then adds shape-specific and role-specific truths.

1. What it does: purpose, business outcome, surface, and place in the wider system.
2. How it is evaluated: primary metrics, guardrails, north star, and experimentation methodology.
3. Current architecture: components, models, features, serving path, and tunable configuration.
4. Data flows: sources, schemas, freshness, and known quality gaps.
5. What has been tried: experiments, rejected ideas, results, and settled constraints.
6. Performance and gaps: current performance, weak slices, and distance from the practical optimum.
7. How it is operated: deployment, monitoring, incident response, and ownership.

The categories are also role-conditioned. An engineering lead needs almost the whole map. A product manager emphasizes outcomes, evaluation, past experiments, and performance gaps. An SRE needs architecture and operations. Ground truth is not one flat document for every reader.

3. What counts as truth

We separate What, How, and Why. What is the portable truth-content: the recommendation slate is capped at 150 items. How is its evidence and provenance: a module-level constant and a guard in the scoring path. Why is the rationale: perhaps the cap was chosen because of storage cost.

What and How belong in ground truth. Why does not. The deployed system can prove its current behavior and show where that behavior is enforced; it cannot deterministically recover a decision that may live in a PRD, a chat thread, or nobody's memory. Why remains a useful interrogation prompt because it often uncovers more Whats, but the irreducible rationale is not authoritative system truth.

Two additional filters keep the reference honest: only production-reachable code is eligible, and every artifact class gets a recency rule. Dead code and stale dashboards should never define the current system.

From theory to a system benchmark

We built a reference implementation called AutoGT. It reads production-scope code, configuration, and available observability surfaces, then returns a gold claim set in the same schema used by expert authors. Every claim is atomic, carries an evidence anchor, and includes a support rubric for an automatic judge.

The evaluation holds everything else fixed. Expert-authored and auto-generated gold score the same candidate knowledge books through one unchanged LLM-as-judge harness. The diagnostic question is simple: do the two gold sources rank descriptions of the same deployed system in a similar order?

Two automation strategies

LLM extraction with an ontology prior

The first generator reads parsed code structure and verbatim text chunks. It emits candidate claims only when the evidence excerpt maps back to source, canonicalizes duplicate statements, counts support across extraction passes, and abstains on weak candidates.

Its key design variable is the ontology prior: tell the extractor what kind of system it is reading, then increase selection weight for the Essential-Truths categories that system shape makes important. The prior does not alter the evidence requirement or let the generator see expert gold. It changes what the extractor looks for.

The deterministic System-as-Oracle

The second generator, the System-as-Oracle (SaO), puts no LLM in the truth-recovery path. It reads named constants and SQL values; metric, table, and dashboard definitions; and algorithmic control flow such as cascades, guards, filters, branches, and error paths. When code and dashboards disagree, that disagreement becomes a claim instead of being silently discarded.

The trade-off is deliberate. The LLM extractor reaches for breadth. The deterministic oracle reaches for authority, reproducibility, and auditability.

What the study found

We ran the same pipeline on two independent production video-recommendation backends. Both systems occupy the same ontology cell: rank/retrieve, immediate-feedback, reversible, and single-user. Company A contributed 27 candidate knowledge books and a metrics dashboard; Company B contributed nine books and no dashboard. This tests cross-customer transfer within one problem shape, not transfer across every kind of production system.

Generated gold ranked candidate books similarly

On the larger system, the unconditioned LLM produced fragile agreement with expert gold: Spearman ρ = 0.36, with a confidence interval crossing zero. Adding the ontology prior lifted agreement to ρ = 0.73. The deterministic oracle reached ρ = 0.78.

On the smaller system, the no-prior LLM, prior-equipped LLM, and deterministic oracle reached ρ = 0.84, 0.92, and 0.80 respectively. The nine-book confidence intervals are wide, so those point estimates should be read as proof of concept rather than settled cross-system performance.

But coverage remained narrow

Ranking agreement is only half the picture. Every automated method concentrated on architecture (U3) and data flows (U4), the truths directly visible in code and dashboards. The human experts were the only source with meaningful breadth across what has been tried (U5), performance gaps (U6), operations (U7), and the recommendation-specific categories.

The coverage entropy makes the difference visible. Human-authored claims reached 2.59 and 2.37 bits across the two systems. The automated methods ranged from 0.86 to 1.40. The prior improved which candidate descriptions were ranked well, but it did not close the breadth gap.

Automation knows architecture. Experts know experience.

This is the most important result. Automated ground truth can already carry part of the expensive, repetitive work of benchmark construction. It can recover code-grounded claims at machine speed, keep evidence attached, and re-run as the system changes.

It cannot yet recover the system's lived history: why an experiment was abandoned, which cohort remains fragile, how an incident changed an operating procedure, or which constraint has become a sacred cow. Those truths live across experiments, runbooks, dashboards, review threads, and human memory. Experts remain the coverage frontier.

The right operating model is therefore not human or automation. It is automation for the stable, evidence-grounded half of the benchmark, with scarce expert attention reserved for experiential and operational truths.

What this changes in practice

1. Benchmark the system, not the repository. Compose evidence across code, configuration, data, metrics, and control flow before writing claims.
2. Condition extraction on system shape. A ranker, fraud detector, pricing engine, and notification system require different truth templates.
3. Keep evidence first-class. Every What needs a How: a symbol, span, query, metric, or structural witness that makes the claim auditable.
4. Separate truth from rationale. Preserve current behavior and provenance as gold; treat Why as a probe that can uncover more facts, not as deterministic truth.
5. Spend expert time where machines are weakest. Ask SMEs for experiments, performance gaps, operational knowledge, and shape-specific failure modes instead of making them re-document constants and data flows.
6. Re-run and diff. A regenerable reference turns benchmark maintenance into a change feed, making drift visible instead of waiting for periodic re-authoring.

The research agenda

The current study establishes feasibility, not a universal solution. The next tests must cross ontology cells, starting with a shape such as detect/classify, and verify that the truth template changes as predicted. Ground-truth representations also need first-class support for branching control flow and multi-hop claims, not only atomic statements.

Coverage should expand by admitting more production evidence: warehouse schemas, scheduling DAGs, runbooks, alert configurations, experiment logs, and incident records. Evaluation itself should report separability, judge reliability, and confidence intervals as standard. A single correlation is not enough.

The bottom line

Production agents need more than repository understanding. They need a current, auditable account of the system they are expected to change: what it does, how that truth is evidenced, which facts matter for this kind of system, and where human experience must complete the picture.