Blog··8 min read

Designing Explainable Security Graphs Without Misrepresenting Evidence

How DarkMoon draws attack paths so a hypothesis never looks like proof: weakest-link labels, line style plus word plus icon, and the limits we still have.

DarkMoon research series·B. Data visualization & UX·Article 12 of 23·All 23 articles

Introduction

Attack graphs have been studied for more than two decades, from model-based generation (Sheyner et al., 2002; Ammann et al., 2002) to reviews of their visual syntax (Lallie et al., 2020). DarkMoon's situation is a little different: its paths are built from findings produced by LLM agents during an authorised penetration test, and those findings carry very different levels of certainty. Some links were observed by the server, some were declared by an agent, and some were inferred from topology.

A graph flattens those differences unless the design deliberately keeps them. A solid arrow between two boxes reads as « this led to that », whatever the data behind it. This article explains how we tried to keep the graph honest, using two public research records: darkmoon-attack-paths-research for the engine and darkmoon-visualization-research for the views.

Problem

Two facts from the engine research set the constraint.

  1. Agent declarations are useful but often wrong. On real single runs, the strict precision of agent-declared links was 0.6 (5 decidable links), 0.556 [0.27–0.81] and 0.316 [0.15–0.54] (VERIFIED, n = 1 each, intervals overlap). The dominant error was linking two facets of the same root cause.
  2. Server qualification has not succeeded on real data. No declared link has reached the server's observed_flow qualification on any measured real campaign (0/25 in replay, 0/19, 0/19; VERIFIED). Agents mask credentials and the Privacy Gateway tokenises them, so the values needed for exact attribution are absent.

So on most real campaigns, almost everything on the graph is a hypothesis. The interface has to show that without becoming unreadable.

Methodology

The retained engine design (« architecture D ») stores each agent declaration as declared_by_agent, shows it immediately as a hypothesis, and promotes a link only by exact value attribution on the server's own execution records. Otherwise it abstains with one of 14 closed reasons. A path is Evidence-Backed only if every step is supported by a server rule or qualification; everything else is a Possible Attack Path and is never ranked (VERIFIED, code and ADRs). In DarkMoon Community there is no execution ledger and agent-written evidence text is withheld from the engine, so proven_paths = 0 by construction.

Diagram. Figure 1. From link provenance to what the user sees. The weakest link sets the label of the whole path.
Figure 1. From link provenance to what the user sees. The weakest link sets the label of the whole path.

Interface side: four rules

RuleImplementationStatus
Proof level is encoded by line style + word + icon, never by colour alone and never by severity colourEvidence-Backed = solid + shield; Possible = dashed + dashed circle; Treatment in progress = dotted + wrenchVERIFIED (commit)
Exposure is not attackWithout a proven path, the 3D map offers an « exposure path » with the fixed line « a reading of the inventory, not a demonstrated attack »VERIFIED (code)
No ranking without proofPath-based prioritisation is refused with fewer than two proven paths sharing a step; the UI says so instead of showing an empty rankingVERIFIED (capture)
Same words everywhereOne legend across 2D map, path rail, 3D legend, report and terminal; word parity tested against the serverVERIFIED (commit) / OBSERVED (tests)

The first rule follows a standard visualisation argument: line style is a separable channel from hue, and it survives greyscale printing and colour-vision deficiencies (Munzner, 2014; WCAG 2.2 SC 1.4.1). The others are domain rules that came from owner reviews and became decision records (UXDR-05, -11, -13).

Real results

Before and after on the attack-paths view

Attack-paths view before the change: an 'Observed' group and choke-point styling on the 2D map, dark theme, synthetic demo data.
Attack-paths view before the change: an 'Observed' group and choke-point styling on the 2D map, dark theme, synthetic demo data.Original capture: ASCIT31/darkmoon-visualization-research
Attack-paths view after the change: the rail groups paths as Possible, Inferred or Likely, the selected path is drawn dashed, and a banner reads 'Inferred attack path, not proven. A reading of the inventory, not a demonstrated attack.'
Attack-paths view after the change: the rail groups paths as Possible, Inferred or Likely, the selected path is drawn dashed, and a banner reads 'Inferred attack path, not proven. A reading of the inventory, not a demonstrated attack.'Original capture: ASCIT31/darkmoon-visualization-research

The v1 capture grouped paths as « Observed » and drew choke points prominently. The v2 capture groups them as Possible / Inferred / Likely, draws the selected path dashed, and states in plain words that it is not a demonstrated attack (VERIFIED, captures; the demo seed also differs between v1 and v2, see metadata).

The 3D map tells a story only for a proven path

The orbital three.js/WebXR map was reworked in October and November 2026. Its legacy « attack » mode was renamed exposure path; every exposure step carries the eyebrow « Exposure path · not proven » and is drawn dashed; the guided story (« Trace the attack path ») exists only for a server-proven path (VERIFIED, commits and code).

Orbital 3D infrastructure map, dark theme: assets on rings by exposure zone, with an intro card offering to walk the exposure path because no proven path exists.
Orbital 3D infrastructure map, dark theme: assets on rings by exposure zone, with an intro card offering to walk the exposure path because no proven path exists.Original capture: ASCIT31/darkmoon-visualization-research

Operator-declared paths are kept apart

An administrator can declare a path by hand. Those paths are labelled « Declared by operator · not proven », drawn as thin dashed lines, and never counted with observed paths, in the treatment plan or in the report (VERIFIED, code).

The same campaign in two editions

On the same synthetic campaign, the Community terminal shows 0 evidence-backed · 17 possible paths, while Pro shows 2 evidence-backed · 17 possible (VERIFIED, captures). The difference comes from the editions' evidence rules, not from the drawing.

Community Terminal Explorer at 160×45: path list headed '0 evidence-backed · 17 possible', a 3-step Possible Attack Path with links labelled 'possible · not observed', and a node panel explaining the path kind.
Community Terminal Explorer at 160×45: path list headed '0 evidence-backed · 17 possible', a 3-step Possible Attack Path with links labelled 'possible · not observed', and a node panel explaining the path kind.Original capture: ASCIT31/darkmoon-visualization-research

Limits

We list these because a design that claims honesty should be checked against its own wording.

  • Our own wording overstates in two places. The server pattern rules R1 and R3 read agent-written evidence text, not an independent capture. Yet the Pro Terminal Explorer describes an Evidence-Backed path as « every step backed by evidence the server recorded », and a Community docstring says Pro evidence « comes from the server's own recording ». Both strings are flagged for correction in the engine repository's limitations (VERIFIED).
  • Two counters can disagree. The executive legend reads every non-observed provenance as Possible, while the path rail treats a corroborated derived path as Evidence-Backed. The counts can differ; on the demo data the difference is 0 (OBSERVED).
  • Comprehension is a hypothesis. No user study has tested whether « Evidence-Backed vs Possible » is understood without training (not measured).
  • No layout comparison. ELK layered graph, Atlas 2D map and orbital 3D map coexist; none was compared with the others on the same task (not measured). No frame-rate or ELK layout-time measurement exists (not measured).
  • The orbital map's severity colours differ from the application tokens (VERIFIED inconsistency; presumed unintended).

Figures

  • Figure 1 (Mermaid): provenance → path label.
  • Captures: real UI on synthetic demo data, from the visualization research repository; metadata in each folder's metadata.json.
  • More engine diagrams, checked against Community commit 10eaab0: diagrams/.

Scientific sources (verified)

DOIs checked against Crossref and URLs fetched on 2026-11-20. Author lists follow the BibTeX file of the attack-paths research repository.

Research, source code & reproducibility

Related reading: Declared vs Proven Attack Paths: A Practical Evidence Model · Visualizing Cyber Attack Paths: From Node-Link Diagrams to Orbital Graphs · Visualizing Attack Paths in ASCII and Unicode · Progressive Disclosure in Vulnerability Management Interfaces


If you find a place where a DarkMoon view states more than the evidence supports, please report it on the attack-paths research repository; we treat those as defects.

Next
Run it against your own lab

Darkmoon is open source (AGPL-3.0) and self hosted. Clone it, point it at a target you own, and read every line.