#Audit log

Every pass made over this codebase looking for defects, what each one found, and what changed as a result. It is kept because the shape of the mistakes is more useful than the list of them — several were found only after the thing meant to catch them turned out to be broken itself.

The threat model and the known limitations live in SECURITY.md.

#Findings from the audit, and what changed

An adversarial pass over the codebase found these. All are fixed, each with a regression test.

#Second pass

Rollback is worth being precise about: this detection cannot prevent the attack, only make it impossible to perform quietly. Branch protection on the repository holding the vault is the actual control.

#Third pass

#Fourth pass — crypto review and the hardware path

Two things were previously listed as "cannot be verified here". Both had an achievable form that was simply skipped.

Crypto properties are now pinned by adversarial tests, including ones the code does not itself enforce and would lose silently if the backend changed:

• A low-order X25519 public key cannot yield a predictable shared secret. This matters: if ECDH returned all zeros for those points, an admin tricked into running hush team add mallory <crafted-key> would make the vault readable by anyone holding the file, no private key needed. OpenSSL rejects all seven classic points — but that is inherited, not enforced by hush, so it is now asserted.
• Every seal uses a fresh IV (2000 samples), data-key generations are independent, tampering with any field of a value or a wrap is detected, a wrap cannot be transplanted to another recipient, and the KDF genuinely depends on both public keys.

The hardware path is now tested without hardware. The age plugin protocol is age's code, not hush's; what hush owns is detecting a plugin identity, invoking age correctly, and reporting failures. A stub plugin and a stub age cover all of it — and doing so found two real bugs:

#Fifth pass — conformance, the ladder's top rung, and the untested surfaces

The scheme is now validated against an independent implementation. test/scheme-conformance.test.ts reimplements hush/v1 in WebCrypto, written against the documented spec rather than against src/crypto.ts, and checks it opens what hush produced. That shows two things self-testing cannot: the scheme is specified precisely enough for someone else to implement, and hush matches that specification rather than merely being self-consistent. Every primitive is also differentially tested — node:crypto and WebCrypto must agree on HKDF-SHA256, AES-256-GCM with AAD, and X25519 — which is much of what a reviewer checks for primitive misuse.

#Sixth pass — the hardware path, driven for real

The remaining gap was "we cannot test the hardware path without hardware". That was only half true. test/fixtures/age-plugin-mock implements the actual C2SP age-plugin protocol, so the real age binary drives it exactly as it drives age-plugin-yubikey — recipient-v1, identity-v1, and the -> msg stanza a hardware key uses to ask for a touch.

The complete upgrade is now exercised end to end in CI: hush detects the plugin identity as hardware, adds a plugin-backed recipient through real age, credits the hardware rung, retires the software key, re-keys the vault, and then opens it only through the plugin.

What that leaves unproven is one thing: whether age-plugin-yubikey correctly speaks USB to the device. That is that project's responsibility and its own test suite — nothing in hush sits between them.

Independent human review of the scheme is still worth having. The conformance suite raises the floor; it does not replace a reviewer.

#Seventh pass — fuzzing and mutation testing

Two methods not used before. Fuzzing checks invariants over inputs nobody chose; mutation testing checks whether the tests would notice if a fix were removed.

Fuzzing (test/fuzz.test.ts, seeded so failures reproduce) asserts that no secret survives any chunking of any output, that values round-trip through export and import unchanged, and that the vault agrees with a plain model over random operation sequences. Every regex was also checked for catastrophic backtracking; all are linear.

Mutation testing deliberately reintroduced ~25 past bugs to see which the suite would catch. Seven survived, meaning seven real fixes had no test holding them in place — including the atomic save, the vault lock, the allowEnvs account check, and the command deny list. All are now pinned.

And it found a live bug:

That bug was found by refusing to dismiss a flaky test. Making the failure diagnosable — printing what the losing writer said — turned "sometimes 7 of 8" into "PAR_1 reported success and is not in the vault", which named the cause immediately.

Two mutants survive deliberately. The constant-time token comparison is functionally identical to ===, and no functional test can distinguish them. The rollback watermark's write condition is idempotent, so relaxing it changes nothing observable. Both are equivalent mutants rather than coverage gaps.

#Eighth pass — the .env dropzone

A new feature is new surface, so it got the same treatment. Four issues in the code that had just been written:

Plus client-side guards the server could not provide: a file-size check before the tab reads a multi-gigabyte drop into memory, a file-count cap, and a double-submit guard (a second click re-sent spent stages and reported them as expired).

A usability trap worth naming, because it would have cost someone an afternoon: filing a key into a service account the project does not use is accepted, but hush run then silently does not inject it. Import now reports which accounts are not in use, with a one-click fix.

The page script had no coverage at all. The whole UI is a template string, so TypeScript never looks at it — a syntax error there compiles cleanly, passes every server test, and ships a blank page. There is now a test that parses it, runs its esc() against hostile input, executes its row-key function to prove two files stay apart, and checks that every data-bearing interpolation into markup is escaped. That last one found s.id reaching an attribute raw; a hardcoded catalogue key today, so not exploitable, but escaped now.

#Ninth pass — drift, dead config, and coverage holes

Documentation and configuration rot silently; no behavioural test notices. A consistency checker comparing code against everything that describes it found:

Five surfaces had no tests at all: the UI's /api/account and /api/team endpoints (the latter re-keys the vault on member removal), and the hush_list_accounts, hush_check_repo and hush_provision MCP tools — including the one that resolves "use my acme fal key". Adding them found that hush_provision reported "Ready" for an account holding only some of a service's variables, so an agent would start a deploy that failed halfway with a confusing auth error.

Validation failures also answered 500 rather than 400, reading as server faults. There is now a ValidationError type thrown by the name, size and key checks, so bad input is distinguishable from a real fault — replacing a brittle message-matching regex that had already gone stale once.

#Tenth pass — hush doctor had gone stale

The "is my setup right?" command was still describing the tool as it was five passes earlier. It never mentioned the security ladder, the policy actually in force, the agent skill, the age bridge, or service accounts — and it repeated a bug already fixed in hush ls, printing fal/personal under "envs" as though an account were an environment.

That last one is the interesting kind: not a crash, just two surfaces telling the user different things about the same setting.

#Eleventh pass — attacking staleness at the source

The last several passes all found the same shape of defect: one part of the tool stops agreeing with another as features land around it. Rather than keep finding instances, this pass went after the cause — every fact stated in more than one place.

The tests added here are the point: not "this file currently says 120", but "no comment may claim a line count at all", and "the default policy may not be retyped outside mcp.ts". A check that a duplicate is correct has to be maintained; a check that a duplicate does not exist does not.

One test in this pass was written as a regex over the source, asserting how the code was written rather than what it does. That is the same brittleness that had already gone stale once in the UI's error handling, so it was rewritten to generate a policy and read it.

#Twelfth pass — the mutation harness was lying

This pass began by re-running mutation testing across every module rather than only the recently-touched ones. It found more in the harness than in the code.

Two harness bugs, in opposite directions. The first shell version passed a space-separated list of test files as one unquoted variable — which zsh does not word-split — so node --test got a single nonexistent filename every time, the # fail line never appeared, and every mutation was reported as surviving. The rewrite in Node fixed that and introduced the opposite fault: it copied only the directories it thought were needed, so the worker's copy was missing LICENSE and RESEARCH.md, consistency.test.ts failed on a broken documentation link before any mutation was applied, and every mutation was reported as killed. Five batches of results — 87 mutations — were worthless, and they were worthless in the flattering direction.

A third fault showed up later: the harness re-read each source file from the live repository on every job, so an edit made while a run was in flight changed what the remaining mutations were applied to. The run then measured a mix of two trees and reported it as one number.

The fixes are not better copy lists. The harness now runs the unmutated tree first and refuses to start unless the baseline is green; it snapshots every source file once at the start; and it checks at the end that nothing moved underneath it. A mutation run without a green baseline measures nothing, a measurement that can only come back "good" is not a measurement, and a run spanning two versions of a file describes neither of them.

With a trustworthy harness, the honest first number was 89 of 137 killed.

The worst of these was found by reading rather than by mutation. Mutation testing asks "would anything notice if this line changed?"; it cannot ask "is what this line does safe?". toShellExports had a test, the test passed, and the test asserted the vulnerable output format verbatim — so every mutation of that line was dutifully killed by a test that was pinning the bug in place.

| Low | findExampleFile was exported and called by nothing. The same shape as the gen field: dead code reads as a feature, and someone eventually extends it or trusts it is doing something | Removed, with a check that refuses any export nothing uses — a test counts as a use, which is the right bar for anything other people are meant to call | | Low | Three of this pass's own test edits corrupted the files they were editing. A patch script whose replacement text contained a backtick terminated its own template literal and spliced a whole file into the middle of itself: src/run.ts ended up containing two complete copies, and SECURITY.md two copies with the stale one first. TypeScript caught the source; nothing would have caught the document | Checks that no document repeats a heading and no source file declares a top-level symbol twice | | Low | A hand-edited JSON example in the README did not parse — a trailing comma, in a config block someone would copy | Every fenced json block in the docs is parsed by a test |

The three worst findings here came from reading and from trying the thing, not from mutation. Mutation testing asks "would anything notice if this line changed?"; it cannot ask "is what this line does safe?", and it cannot ask "does any of this work once it is installed".

toShellExports had a test, the test passed, and the test asserted the vulnerable output format verbatim — so every mutation of that line was dutifully killed by a test that was pinning the bug in place. And the whole package was unusable after npm install while 354 tests passed, because every one of them ran from the checkout. The lesson is the same one the harness taught: a check has to be able to fail for the reason you care about, or it is decoration.

Tests, not fixes, were most of the work. Forty-three properties had no test at all — among them: an expired approval grant being honoured, a biometric denial being ignored, allowCommands not enforced, the deny list matched against the whole path rather than the command name, the UI binding every interface, the identity file's permissions, hush get printing a credential without confirmation, --with not overriding a pinned account, and the vault being written in place rather than replaced.

Three tests were passing vacuously, which is the same failure as an untested property but harder to see:

• A CLI helper wrapped execFileSync in a try/catch that turned any exception into { code: 1 }. A ReferenceError from a missing import was therefore indistinguishable from a command that ran and failed, and a test asserting a non-zero exit passed without the CLI ever starting. There is now a drift check that refuses the pattern outright.
• The low-order-point test asserted only "something threw". A change to the JWK shape would have satisfied it just as well. It now runs a legitimate key through the same path first, so "it threw" means something.
• A control-character test asserted over JSON.stringify of the output, which escapes every control character — so the regex could never match and the test passed with the sanitiser removed.

Two mutations were checked individually and are genuinely equivalent; they are listed below rather than chased with contorted tests.

#Mutants that survive on purpose

Four mutations are known to be behaviourally identical, each checked individually rather than assumed:

• The constant-time token comparison versus ===. No functional test can see the difference.
• The ladder's rung filter (c.rung <= r versus === r). The loop breaks on the first failing rung, so by iteration r every earlier rung has already passed — the two are provably equivalent.
• The rung === 5 half of the nudge guard. Reaching rung 5 means every check passed, and next is the first failing check — so at rung 5 next is always null and the !p.next half has already returned.
• The fsync before the rename. Removing it is invisible to every in-process test, because what it buys is durability across a power cut rather than any behaviour a running program can observe. It stays because the cost is one syscall per save and the thing it protects is the whole vault.

One more was equivalent only because of a defence that was already there: the data-key cache is keyed by fingerprint, and mis-keying it changes nothing because membership is re-checked before the cache is consulted. Removing the re-check as well is caught immediately — which is the useful way to state it.

The rollback watermark's write condition used to be on this list. It is not equivalent any more: the condition now compares the member list by content rather than by length, and a test covers the case that distinguishes them.

Reaching the temp-file cleanup took some care, and is worth recording because the first attempt did not. Every obvious way to break a save — a directory where the vault should be, an unwritable parent — throws before the temp file is created, so the cleanup never runs and a test built on one of those passes without exercising anything. The test that does work makes the destination immutable with chflags, so the temp file is written and fsynced and only then is the rename refused.