A Grounded Documentation Agent for ERP Implementations

ERP consultants assemble workflow documentation by hand from requirement documents, configuration screens, execution logs and product knowledge. A general-purpose assistant can describe how the software usually behaves, but not how a specific client instance has been configured.

That is where documentation goes wrong, and it fails in a predictable direction: stating that the system requires a quality inspection before packing, when no such control exists in the instance, is worse than saying nothing. It misleads the people signing off the implementation and the operators who depend on it afterwards.

• A single stateful agent — a fifteen-stage state machine with four terminal actions: document, clarify, escalate, or reject as out of scope.
• Two external tools: a deterministic configuration-inspection tool over a frozen sanitised snapshot, and evidence retrieval over a controlled corpus ranked by TF-IDF in pure Python. No vector store, because retrieval scope did not require one.
• A seven-status evidence model. Three statuses can support a claim about the deployment; the other four — demo narrative, generic product knowledge, contradiction, absence — cannot, and belong in the disclosure section instead.
• A nineteen-rule deterministic validator that checks structure, secret patterns, case isolation, citation resolvability and enforcement honesty, and decides whether a failure is repairable or must reach a human.
• One bounded repair attempt, which can reconcile and revalidate but structurally cannot re-call the tools — a failed draft is a reasoning problem, not an evidence-gathering problem.

The design decision that matters is where the guarantees live. Grounding, schema compliance, secret exclusion and case isolation are enforced in deterministic code, not in the language model, so they hold regardless of which model sits behind the interface. The model is confined to language tasks: decomposing a request, normalising business phrasing, refining queries, drafting from supported evidence.

Across a twenty-case set (eight normal, six edge, six adversarial) on the deterministic reasoner, every case executed without error: 20/20 expected actions, grounding rate 1.0, 104 of 104 implementation claims supported, no hallucinated citations, 70.9 ms mean run time. The bounded repair path was used in 7 of 20 runs.

Stated plainly, because the report states it plainly: the evidence is synthetic and hand-authored, the cases were written alongside the implementation, and no hosted model completed the full twenty-case set. That result measures internal consistency, not external validity. The zero-hallucination figure is weak evidence by construction, since the deterministic reasoner builds citations from the evidence registry.

• Validate against a real sanitised export rather than a synthetic fixture. The tool was designed for that swap — it has not been made.
• Have the evaluation cases authored by someone who did not write the implementation.
• Complete a hosted-model run across the full case set before claiming anything about model-side behaviour.