Blog··8 min read

From Vulnerability Findings to Attack Paths: DarkMoon's Research Journey

Thirteen months of attack-path research in an autonomous AI pentest system: what was tried, what was measured, and what has not been demonstrated.

DarkMoon research series·A. Architecture & attack paths·Article 1 of 23·All 23 articles

Introduction

DarkMoon is an autonomous AI penetration-testing system. An orchestrator and specialist LLM agents run tools against an authorised target and publish findings: an exposed service, an injectable parameter, a leaked credential. For most of its life, DarkMoon produced these findings as a list, ordered by severity.

A list is not how an attacker thinks. An attacker chains: the leaked credential opens the database, the database holds a key, the key opens a shell. Security teams want to know which chain is real, and which single fix would break it.

This article is the overview of a research series. It follows, in order, the approaches DarkMoon tried between October 2025 and November 2026 to turn findings into attack paths, and states what each one actually measured. The full record, with 61 experiment pages and every negative result, is public in the attack-paths research repository.

The problem

Linking findings sounds easy until you ask what a link is supposed to mean.

  • A topological link says two assets are reachable from each other.
  • A plausibility link says a technique usually precedes another.
  • An execution link says that, in this campaign, the exploitation of B actually used a result obtained in A.

Only the third is an attack path in the sense a defender cares about, and it is the hardest to establish. The research question the repository settled on is narrow:

When LLM agents state that one finding enabled another, how can a system decide, without trusting the agent and without extra model calls, which of those statements are supported by what the system itself recorded, and how often are they right?

This is a different problem from classic attack-graph generation (deriving potential paths from a network model) and from provenance-based intrusion reconstruction (OS audit logs). It sits between them.

Methodology

The repository is a research record reconstructed read-only from code, commits and internal reports. Every statement carries one evidence label:

LabelMeaning
VERIFIEDchecked in code or in a primary report cross-checked with a second artefact
OBSERVEDstated in one primary source, not re-executed
SIMULATEDdeterministic harness, replay, synthetic corpus or proxy model
HYPOTHESISan interpretation no source tested
SUPERSEDEDreplaced by later work, kept for history
UNKNOWNthe evidence was not available

Experiments are classed as successful, failed, inconclusive or adversarial. Two reading rules matter throughout this series: finding counts are never comparable across targets, and a harness or proxy-model score is never an agent-declaration precision.

Real results: the journey in seven steps

1. Isolated findings (Oct 2025 to Sep 2026). Findings went into markdown reports, then into a dashboard store. There was no relation between findings beyond the report's narrative. Prioritisation was by reported severity and proof status, which survives today as the fallback "by reported severity × proof".

2. Topology (Sep 2026). Paths were computed as breadth-first routes over the infrastructure graph. On 18 real production campaigns the server returned 43 topological paths, and the interface read none of them; 22 % of findings were attached to nodes through a fallback, up to 31 of 35 on a single node (VERIFIED). Topology was kept for the map and for a clearly labelled inferred layer, not as a source of attack paths.

3. Structured claims and observed rules (late Oct to early Nov 2026). Agents were given four optional "chain data" fields, and the server matched three evidence rules: R1 (a secret produced by A is used as a credential by B), R2 (open registration) and R3 (an exploited finding's response proves access to another node). The diagnosis on 18 campaigns and 340 findings (EXP-010) was stark: 0 of 340 findings carried any causal field, 5 proven paths in total, all from R3, and ranking allowed on 0 of 18 campaigns (VERIFIED).

4. A preregistered causal programme (Nov 8 to 11, 2026). Counterfactual replay, a passive evidence engine, an expert engine, a capability graph and rules on finding text were each tested against preregistered criteria. Every causal-verification approach failed its criterion. Causal-edge recall was 0 of 14 on every run of an internal 15-service lab (EXP-015), and no multi-finding chain was proven on real data. One durable rule came out of it: declared causal fields never make a proven path without server corroboration.

5. Server-side reconstruction from logs (Nov 9 to 17, 2026). A run ledger now records salted fingerprints of what each command produced and consumed. Attribution rules on top of it ("own command", endpoint observation, execution ids) either produced false proven links or cost more tokens than the +1 % budget allowed. Article 5 of this series covers these failures in detail.

6. Agent-native declarations and architecture D (Nov 16 to 18, 2026). The four structured fields were replaced by one optional list, enabled_by, in which the agent names the earlier findings it used. The server shows each valid declaration at once as declared_by_agent and promotes it only by exact value attribution on its own execution records. This is the current architecture, released on 18 November 2026.

7. Community port (Nov 19, 2026). The engine, enabled_by and a Terminal Explorer were ported to the open-source edition in Dark-Moon PR #59. Community has no execution ledger, so proven_paths = 0 by construction.

What the current state measures, stated as the repository states it (almost every real-run figure is n = 1, one target, one model):

  • 0 server-qualified links on real data: c5/c6 replay 0 of 25, c7 0 of 19, final lab run 0 of 19, DVGA Community 0 of 16 by construction.
  • Declared-link strict precision 0.6 / 0.556 / 0.316, single runs (c5 with only 5 decidable links; c6 interval 0.27 to 0.81; c7 interval 0.15 to 0.54). The intervals overlap.
  • No multi-step chain proven on real data under the strict definition. An earlier "2 proven cross-host chains" claim is SUPERSEDED: those chains were agent-declared.
  • Non-promotion holds on synthetic scenarios: 0 false promotions on 100 scenarios and 117 edges, mutation score 95.2 % (SIMULATED).

Limits

DarkMoon does not have a universally reliable causal model, and a graph plus an LLM does not demonstrate validated causal intelligence. The oracles and harnesses were built by the same team. Targets are self-built labs or well-known vulnerable applications that are likely present in model training data. No run was made on a public benchmark. Two product strings overstate how independent the R1/R3 evidence is; they are flagged for correction in the repository's limitations page.

Figures

Diagram. Figure 1. Approach history, adapted from the repository's approach-history diagram.
Figure 1. Approach history, adapted from the repository's approach-history diagram.
DarkMoon Terminal Explorer in the Community edition on synthetic demo data: the Attack paths tab lists 17 paths, all labelled POSSIBLE, with 0 evidence-backed; the selected path is a 3-step chain from an HTTP exposure to a confirmed finding, each link marked 'possible, not observed', with a note that the server inferred it from topology and the campaign evidence does not show the chain end to end.
Figure 2. What the honest end state looks like in the open-source edition: every path is a hypothesis. Synthetic demo data.Original capture: ASCIT31/darkmoon-attack-paths-research

Scientific sources

  • Sheyner, Haines, Jha, Lippmann, Wing. Automated generation and analysis of attack graphs. IEEE S&P 2002. doi:10.1109/SECPRI.2002.1004377
  • Ou, Govindavajhala, Appel. MulVAL: A Logic-based Network Security Analyzer. USENIX Security 2005. usenix.org
  • King, Chen. Backtracking intrusions. SOSP 2003. doi:10.1145/945445.945467
  • W3C. PROV-DM: The PROV Data Model. 2013. w3.org
  • Deng et al. PentestGPT: Evaluating and Harnessing Large Language Models for Automated Penetration Testing. USENIX Security 2024. usenix.org

All entries come from the repository's verified bibliography.

Research, source code & reproducibility


The rest of this series takes each step apart. If you want to check a number before reading on, every figure above links back to an experiment page you can read in full.

Next
Run it against your own lab

Darkmoon is open source (AGPL-3.0) and self hosted. Clone it, point it at a target you own, and read every line.