· 9 min read
A penetration test that only proves impact leaves you with the harder half of the work: writing the fix, checking it actually closes the hole, and making sure it did not break anything else. Most of the autonomous pentesting story stops at the finding. Darkmoon's specialists prove exploitation, and a separate defensive agent picks up from there, turning each confirmed finding into a minimal code fix that carries its own before-and-after proof. The one rule that shapes the whole design: it opens a pull request for a human to review and merge, and never merges on its own.
This is a Pro-edition capability. It is worth being precise about what it does, because remediation is exactly the place where an over-eager automation would do the most damage. A wrong patch that lands unreviewed is worse than no patch at all.
The gap between a proven exploit and a merged fix
Static remediation advice is generic by nature. A scanner tells you a parameter is injectable and points at a rule number; turning that into the right change in your codebase is still manual. The advantage Darkmoon starts from is that a finding is not a guess. It is an exploit that already ran end to end, with the exact request and the raw response. That gives the defensive agent something a pure-analysis tool never has: a concrete way to reproduce the vulnerability against a fresh build, apply a fix, and re-run the same exploit to see it fail.
The methodology is not invented from scratch. It reuses openly published practice, the three-phase secure-review prompt from Anthropic's claude-code-security-review (understand the code's secure pattern, trace tainted data from sink to source, make the minimal root-cause change) and the detect, validate-in-a-sandbox, bounded-patch, human-PR shape published for agentic security fixing. Darkmoon's addition is the reproduced exploit at both ends of the patch.
The pipeline: reproduce, patch, re-run, open a PR
For each confirmed finding the agent walks a fixed loop. It maps candidate source locations, builds an ephemeral sandbox from the repository's own Dockerfile, reproduces the exploit, asks an injected model for the smallest correct fix, rebuilds, then runs the gates. It iterates until it is confident or gives up honestly.
| Stage | What happens | Why it matters |
|---|---|---|
| Code map | Rank candidate sink locations from the finding's category and tokens, several per file | A decoy line cannot hide the real sink |
| Sandbox | Ephemeral Docker container from the repo Dockerfile, bound to 127.0.0.1, removed on exit | The exploit runs somewhere disposable, not against production |
| Reproduce | Replay the proven exploit against a fresh build before any change | Confirms the agent is fixing the real bug |
| Patch | An injected LLM returns structured search/replace edits, applied deterministically; an overreach judge reviews scope | Root-cause change, not a broad rewrite |
| Gates | Exploit and variants must fail after the patch; known-good behaviour and the repo tests must still pass | Proves the fix closes the hole without regressing |
| Pull request | Branch pushed, PR opened with the before/after evidence, linked to the finding(s) | A human reviews and merges |
The model that writes the patch is the same class of provider the pentest ran on: Anthropic if a key is present, an explicit OpenAI-compatible endpoint, or the platform's OpenRouter provider. That means a run driven by a local or self-hosted model can keep the remediation step on the same footing, rather than forcing your source through a different third party.
The gates are where the honesty lives
A fix is only ever labelled by what was actually demonstrated. Full sandbox validation needs a buildable Dockerfile and, ideally, a small repro plan the repository can ship, a .darkmoon/repro.json that says how to build the target and how to tell exploited from safe. When that is present the agent has hard before-and-after evidence. When it is not, it does not pretend.
| Disposition | Meaning |
|---|---|
| patched | Gates passed, judge approved, confidence at or above threshold. PR opened. |
| proposed | No push target or credential, or a dry run. Branch and commit prepared locally, recorded for the operator, nothing pushed. |
| draft | A fix was produced but not fully validated, for example no sandbox. Draft PR at lower confidence, evidence attached. |
| failed / skipped | No usable patch, or no code location mapped. Never a fabricated or unproven fix. |
The last row is the important one. If the agent cannot map the finding to a location or cannot produce a fix that survives the gates, it says so and moves on. There is no synthetic diff dressed up as a fix.
Credentials stay local, and the PR channel is the only egress
Opening a pull request needs a forge credential, and a defensive agent that reaches out to a code host is a sensitive thing to run. Two design choices keep it in bounds. First, operators save a provider credential once and it is sealed at rest, encrypted with a persistent key, never returned in cleartext by the API, which only ever hands back whether a secret is set plus a masked hint. Second, the launch line carries an opaque reference such as cred_ab12cd, never the token itself; the pipeline resolves the real secret locally at push time, and that path is never exposed to the model or the API.
for each confirmed finding:
codemap -> rank candidate sinks (several per file)
sandbox -> ephemeral Docker container, bound to 127.0.0.1
reproduce -> replay proven exploit on a fresh build
patch (LLM) -> structured edits, applied deterministically
gates -> exploit + variants FAIL after patch, tests PASS
open PR -> push via opaque CREDENTIAL_REF, link to finding(s)
(human reviews and merges, never auto-merge)Authorization and human review
Remediation is dispatched only when it is explicitly enabled, a credential reference is present, and a repository target is set. The pull request is a proposal for a human reviewer, on GitHub, GitLab (as a merge request), Gitea and the other supported forges alike. It is never merged automatically. Providing a forge credential is an operational control, not authorization to change a system you do not own, and the fix still needs a person to read the diff, the before/after evidence and the linked findings before it lands.
What we have actually validated
We are careful to separate what is proven from what is designed. The backend units and integration (the sealed credential store with its masking and auth gate, the PR-to-finding link store, the prompt builders and provider selection) pass, and so do the toolkit units for the code map, the deterministic patch apply, the forge payloads and the validators. Beyond the unit level, a full end-to-end run against a deliberately SQL-injectable application: the exploit reproduced against a fresh build, the model produced a parameterized-query fix, the exploit and its variants failed after the patch while a valid login still worked, the blended confidence came out at 0.99, and a real fix branch and commit were prepared.
That is one validated engagement on a target built to be fixable, not a fleet statistic, and we are not going to dress it up as one. It shows the loop closes end to end when the target can be built and reproduced. Where it cannot, the disposition table above is what you get instead.
Honest limitations
- Full sandbox validation needs a buildable Dockerfile and ideally a repro plan. Without them the agent still proposes a fix, but opens a draft PR at lower confidence and says so.
- GitLab exposes merge requests rather than pull requests. They are the same thing here: proposals for review, never auto-merged.
- The credentials router is authenticated and encrypted at rest, but the platform's other data routers are unauthenticated today, a pre-existing gap tracked in our design docs, so the dashboard should be deployed behind its documented single sign-on.
- The GitHub, GitLab and Gitea paths are exercised most. Bitbucket and Azure DevOps PR creation is implemented but has seen less use.
Where remediation sits in a Darkmoon run
The remediation agent runs alongside the report, not instead of it. The report body is still generated deterministically by the server from the pushed findings; the defensive agent never touches it and never finalizes the campaign. So a run can end with two artefacts: the proof report you would get anyway, and a set of reviewable pull requests, each linked back to the findings it addresses. If you want the upstream half of that story, the offensive architecture that produces the findings in the first place, read multi-agent AI pentesting without giving the model a shell, and for wiring the whole loop into a build, autonomous penetration testing in your CI/CD pipeline.
Darkmoon is GPL-3.0 and self hosted: source, docs. The remediation agent is a Pro-edition capability.