DarkMoon research series·A. Architecture & attack paths·Article 3 of 23·All 23 articles
Introduction
"Attack graph", "security knowledge graph" and "causal attack chain" are often used as if they were three names for one picture: boxes for vulnerabilities, arrows for "leads to". They are not the same object. They differ in where the arrows come from and in what an arrow claims.
DarkMoon, an autonomous AI pentest system, tried all three while trying to link its findings into attack paths. This article uses that record to make the differences concrete. The underlying literature review is public in the attack-paths research repository.
The problem
Suppose a pentest produced two findings: an unauthenticated Redis instance (A) and remote command execution through Redis EVAL (B). Should the report draw A → B?
- An attack graph says: given this network model and these pre/post-conditions, an attacker could go from A to B.
- A knowledge graph says: in general, the technique behind A is often followed by the technique behind B.
- A causal claim says: in this campaign, B's exploitation used something obtained in A.
The three can disagree. In DarkMoon's own campaigns, "SLAVEOF / CONFIG SET / EVAL ← unauthenticated Redis" was a recurring false link: the three findings share one root cause (no authentication) but none enabled the others. A plausibility graph happily accepts that edge. A causal reading does not.
Methodology
This comparison reads each family through three questions: who creates the edge, what the edge asserts, and what would make it wrong. Each family is then tied to an approach DarkMoon actually tested, with the evidence label the repository gives it.
Attack graphs: deriving what could happen
Classic attack graphs (Phillips and Swiler, Sheyner et al., Ammann et al., Topological Vulnerability Analysis, MulVAL) derive edges from a model of the network and of exploit pre/post-conditions. MulVAL encodes this as Datalog rules over scanner facts and produces logical attack graphs polynomial in network size. The edges are potential: correct relative to the model, silent about whether anyone executed them.
DarkMoon tested a MulVAL-like capability graph built from CWE, CAPEC and ATT&CK pre/post-conditions over 51 real findings (EXP-039): 10 of 78 edges correct (12.8 %), 9 of 87 paths fully correct, recall 10 of 22 (SIMULATED, SUPERSEDED). The graph was not wrong as a model of possibilities; it was wrong as a model of what happened.
Knowledge graphs: encoding what usually happens
Security knowledge graphs (MITRE ATT&CK, SEPSES, BRON, AttacKG and its LLM successor) link techniques, weaknesses and vulnerabilities across many incidents and reports. Their edges are type-level: "technique X often precedes Y". ATT&CK's own design document describes a curated knowledge base, not a causal model or a complete enumeration.
DarkMoon used ATT&CK tactic order as a filter on agent-declared pairs. An internal review found that it accepted 74 % of pairs and 10 of 10 false chains (VERIFIED). It was downgraded to context warnings, never a verdict. Type-level priors can rule out the implausible; they cannot separate a direct edge from a common cause in one campaign.
Causal reasoning: deciding what happened here
Causal theory (Spirtes, Glymour and Scheines; Pearl) makes one point that matters here: observational co-occurrence cannot separate A → B from A ← C → B without further assumptions or interventions. In a pentest, hidden common causes are the normal case, because the underlying misconfiguration is often not itself a finding. And token-level ("actual") causation in a single run is harder than type-level structure.
Recent work on LLMs and causality points both ways. Kıcıman et al. report strong results on pairwise causal discovery benchmarks while noting unpredictable failure modes; Jin et al. (Corr2Cause) report that LLMs perform close to chance at inferring causation from correlation statements.
DarkMoon's attempts at genuine interventions, counterfactual replay on lab targets, were accurate where they applied but had low recall: wire flows 20 of 20 correct, recall 4 of 29, and 2 false proven links in the first round (EXP-019, SIMULATED).
Real results
| Family | Edge author | What an edge claims | DarkMoon test | Result |
|---|---|---|---|---|
| Attack graph (MulVAL-like) | rules over pre/post-conditions | could happen | capability graph on 51 real findings | 10/78 edges correct (SIMULATED) |
| Knowledge graph (ATT&CK order) | curated type-level knowledge | usually happens | plausibility filter on declared pairs | accepted 10/10 false chains (VERIFIED) |
| Causal claim by an LLM agent | acting agent (enabled_by) | happened here | real campaigns with oracles | strict precision 0.6 / 0.556 / 0.316, single runs (VERIFIED) |
| Server lineage check | server, on its own records | a value flowed from A to B | architecture D on real campaigns | 0 qualified links on real data (VERIFIED) |
The repository's interpretation (HYPOTHESIS, consistent with the causal literature): the roughly one-third precision of agent-declared links and the dominance of "same root cause" errors are what you would expect when an agent states edges from co-occurrence and narrative. Architecture D's answer is mechanistic evidence, a value that must flow from A's output into B's input, which is a design choice that follows from causal theory. It is not a new causal-discovery method.
Limits
- DarkMoon does not have a reliable causal model of attack paths, and none of these families gave it one.
- The comparison mixes measurement types: a replay of a rule graph, a review of a filter and real campaigns scored by oracles. Samples are small and not on the same targets.
- The mapping of DarkMoon link states to causal vocabulary (an observed flow is not necessity; necessity for one request is not an attack chain) is the repository's own and is labelled
HYPOTHESISwhere the code does not state it. - Knowledge graphs and attack graphs remain useful as context. The point is only that they cannot be counted as evidence that a link was executed.
Figures
Scientific sources
- Sheyner, Haines, Jha, Lippmann, Wing. Automated generation and analysis of attack graphs. IEEE S&P 2002. doi:10.1109/SECPRI.2002.1004377
- Ou, Boyer, McQueen. A scalable approach to attack graph generation. ACM CCS 2006. doi:10.1145/1180405.1180446
- Strom et al. MITRE ATT&CK: Design and Philosophy. MITRE, 2018. mitre.org
- Hemberg et al. Linking Threat Tactics, Techniques, and Patterns with Defensive Weaknesses, Vulnerabilities and Affected Platform Configurations for Cyber Hunting (BRON). 2020. arXiv:2010.00533
- Li, Zeng, Chen, Liang. AttacKG: Constructing Technique Knowledge Graph from Cyber Threat Intelligence Reports. ESORICS 2022. doi:10.1007/978-3-031-17140-6_29
- Spirtes, Glymour, Scheines. Causation, Prediction, and Search. MIT Press, 2000. doi:10.7551/mitpress/1754.001.0001
- Pearl. Causality: Models, Reasoning, and Inference. 2009. doi:10.1017/CBO9780511803161
- Kıcıman, Ness, Sharma, Tan. Causal Reasoning and Large Language Models: Opening a New Frontier for Causality. 2023. arXiv:2305.00050
- Jin et al. Can Large Language Models Infer Causation from Correlation? ICLR 2024. arXiv:2306.05836
Research, source code & reproducibility
- Main research repository: ASCIT31/darkmoon-attack-paths-research, see scientific positioning and causality and evidence
- Source code (open source): ASCIT31/Dark-Moon, engine modules in
mcp/api/ - Benchmarks: ASCIT31/Darkmoon-Benchmarks (detection only; not an edge-level benchmark)
- Official documentation: docs.dark-moon.org
- Scientific references: literature review §3.1 to §3.4 and BibTeX
- Reproducible experiments: EXP-039 capability graph, EXP-019 counterfactual replay, EXP-050 c7
Related articles
- From Vulnerability Findings to Attack Paths: DarkMoon's Research Journey
- Building an Expert-System-Inspired Architecture with LLM Agents
- Declared vs Proven Attack Paths: A Practical Evidence Model
- Cross-Agent Dependencies and the Common-Root-Cause Problem
If you build attack graphs or security knowledge graphs and have a way to tell a direct edge from a common cause in a single run, we would like to compare notes in the research repository's issues.
See the proof, not just the write-up: the Pro remediation benchmark (fixes retested against the exploit) · how Darkmoon compares to other AI pentest tools.
← All articles