Why Darkmoon builds its reports server-side, deterministically, from proof

The most important design decision in Darkmoon’s reporting is what we do not do: we do not let the model write the report. A language model asked to summarize a long run produces a non-deterministic, sometimes condensed, sometimes incomplete document. For a security report, that is unacceptable, so the report body is assembled server-side from the findings the agent pushed, complete by construction. This article walks through how.

The rule: the body is generated from the database, always

When a campaign is finalized, the agent may pass a report reference, or even a full markdown body. The platform ignores the body and regenerates it from the stored findings anyway. The code is blunt about it:

# The report BODY is ALWAYS generated server-side from the stored findings,
# complete by construction. We never trust an LLM-provided report_markdown
# for the body — it is non-deterministic and has produced incomplete reports.
_is_reference = True  # always regenerate from DB, ignore any LLM-provided body

The one thing taken from the model is the executive summary prose, which is narrative by nature. Every number, every finding, every table is computed from the findings store. Two runs with the same findings produce the same report.

What the report contains

The generated report is a full ISO 27001-style pentest report, ordered from the boardroom down to the payloads:

• Management summary. Plain language for decision-makers: an overall posture (from critical down to medium), the top critical issues in prose, and a required-actions table keyed to timeframes.
• Executive summary. The overall risk level and a findings-summary table breaking each severity into total, exploited and confirmed counts.
• Findings table. Every finding with its id, title, severity, CVSS, status and endpoint, sorted by severity and then by CVSS.
• Vulnerability details. Per finding: description, technical analysis, exploitation commands, raw request and response, evidence logs, and remediation, plus a metadata block carrying the MITRE ATT&CK technique and the ISO 27001 Annex A control where the agent established them.
• Remediation roadmap. Findings grouped into P0 to P3 by severity, each with a timeframe from "immediate, under 24h" to "long term, under a month".

The whole document is stamped CONFIDENTIAL and carries the methodology line ISO 27001 / NIST SP 800-115 / MITRE ATT&CK that every campaign records.

Defensive by construction

Because the generator runs server-side on data an agent pushed, it is written to survive imperfect input. A CVSS score that arrives as "9.8 (High)" or "N/A" instead of a number would once have raised a ValueError and returned a 500 for the entire report; the generator now parses the leading number if present and falls back to zero, so one malformed field never sinks the document. Sorting, severity labels and status labels all degrade gracefully on unexpected values.

The privacy step: rehydration at a single point

This is where reporting meets the privacy gateway. Findings are stored with placeholders, because the model only ever sees placeholders: a real internal host is stored as something like HOST_INTERNAL_001, an internal IP as IP_PRIVATE_001. A report generated straight from the database is therefore full of tokens. The real values are restored by a rehydration function applied at the single point where the CONFIDENTIAL report is written to disk, covering the whole body and the filename in one place.

# Findings are stored with placeholders (the model only ever sees those).
# Rehydrate at the single write point so the local, CONFIDENTIAL report
# carries the real values while the model context never did.
if rehydrate is not None and report_markdown:
    report_markdown = rehydrate(report_markdown)

This is conditional defence in depth, not an absolute. It depends on the gateway being active and on values having been tokenized in the first place; the caveats are in how to run an AI pentest without sending your data to the LLM. What it buys the report is a clean separation: the model’s context is placeholders, the report on your disk is real.

Not overwriting a good report with a partial one

A run dispatches many agents, and more than one may try to finalize. The platform guards the report: if a substantial report already exists and a later call arrives with shorter content, the existing report is preserved rather than replaced by a sub-agent’s partial mini-report. The orchestrator’s full report wins.

Output formats

The run can adapt its report to where the findings are going, selected by a format option:

The canonical stored artifact is Markdown, and the client can export it to a branded PDF whose header carries the same methodology line. A report is durable, portable and readable by both a board and an assessor.

Where this fits

The report is the durable end of a run that you watch live on the dashboard. Its per-finding mappings are what make it useful for demonstrating obligations like NIS2, and its proof-first discipline is the argument in proof of exploitation over AI vulnerability scores.

Darkmoon is our open source project (GPL-3.0): github.com/ASCIT31/Dark-Moon, docs.