AI Agents for Aerospace: Test Suites That Survive a Certification Audit

An AS9100 surveillance audit does not accept "we tested most of it." A DER, or an auditor from the customer's supplier quality team, pulls a requirement number at random from the traceability matrix and asks for the test case, the test result, and the build hash it ran against. If that number does not resolve, the finding gets written up and the delivery slips. Aerospace suppliers building flight control software, actuator firmware, or ground support tooling live inside this reality every quarter: a single skipped test case is not a minor gap, it is a nonconformance that can hold a shipment. Add supply chain traceability down to individual component lots, the kind AS9100D clause 8.1.1 requires when a part fails in the field and the customer needs to know which lot, which supplier, which date code, and security review cycles under DO-326A/ED-202A that add months to every release regardless of how ready the code actually is, and testing stops being a phase and becomes the entire schedule. This is where AI agents for aerospace need to behave less like a coding assistant and more like a QA lead who has sat through an audit before.

Why ChatGPT and Cursor cannot own a certification-grade test suite

Ask ChatGPT or Claude.ai to "write tests for the flight control module" and it will produce a plausible-looking test file: some unit tests, a few edge cases, reasonable assertions. What it will not produce is a requirement ID on each test case that maps back to row 214 in the DOORS export, nor will it track which of the 340 requirements in scope for this release still have zero coverage. A generalist chatbot has no persistent view of your requirements baseline, so every session starts from the code you pasted, not from the certification obligation you actually have. It cannot tell you that six requirements tagged safety-critical in your traceability matrix have no corresponding test, because it never saw the matrix, and it will not ask to see it either. For an audience where the deliverable is the traceability, not just the passing test, that gap is the whole job left undone.

Cursor and GitHub Copilot solve the adjacent problem well: fast, in-editor suggestions for the next line of a test file you already opened. But autocomplete has no concept of a coverage target, a requirements baseline, or an audit calendar. It will happily suggest a test for the function under your cursor and stop there, with no view of the 40 other requirements that touch the same actuator control path, and no mechanism for flagging that a suite has drifted out of sync with the spec since the last DOORS update six weeks ago. When the DER asks for the current traceability percentage against the baseline used for this build, an autocomplete tool has nothing to say. It was built to finish the line you were typing, not to answer that question.

The mismatch is structural, not a missing feature you could bolt on. A test-writing assistant optimizes for a test that compiles and passes. A certification-grade test suite optimizes for a requirement that is provably covered, a result that is provably reproducible against a specific build, and a gap list that is provably complete, none of which a general-purpose coding tool tracks, because none of it is code. It is process discipline expressed as documentation, and process discipline is exactly what a QA specialist agent, not a generalist one, is built to hold onto across an entire audit cycle.

Proof builds the traceability matrix, Chain and Warden cover what sits next to it

Proof is tonone's QA and testing engineer, built around test strategy, requirement-to-test traceability, end-to-end suites for critical journeys, and audits of an existing suite's health, the exact shape of work an aerospace test lead does in the run-up to a surveillance audit. Where a generalist tool treats "write some tests" as the whole task, Proof treats the traceability matrix as the actual deliverable and the passing test as one column in it.

Tonone's Proof builds requirement-to-test traceability matrices that map every safety-critical requirement to a test case and a verifiable result, the exact artifact an aerospace certification audit expects to see.

Mapping every requirement to a test case before the audit calendar forces it

proof-strategy takes a requirements baseline, in this case a 340-line DOORS export tagged by criticality level, and produces a risk map that sorts requirements by consequence of failure, not just by module. It flags requirements with no assigned test type, decides which need unit coverage versus integration versus a full end-to-end run against the actuator test rig, and sets coverage targets per risk tier rather than a single blanket percentage. For a supplier where safety-critical and non-critical requirements cannot be tested with the same rigor, this risk-first sorting is what turns a flat requirements list into an audit-ready plan.

End-to-end suites for the actuator and flight control journeys that matter

For the journeys that actually matter in a flight control review, commanded actuator movement under nominal load, failover to the backup control path on sensor disagreement, graceful degradation when a redundant channel drops, proof-e2e builds the end-to-end specs against the hardware-in-the-loop test rig rather than a mocked stub. Each spec carries the requirement ID it satisfies in its test name and its result log, so when an auditor asks for evidence on a specific requirement, the answer is a specific test run with a timestamp and a build hash, not a promise that it's covered somewhere. That naming discipline, wiring the requirement ID into the test artifact itself, is the difference between a test suite and a traceability matrix that happens to also run code.

Auditing the suite before the DER does

Before the external audit, proof-audit runs the internal one: it finds flaky tests that would read as unreliable evidence to an auditor, coverage gaps against the current requirements baseline (which drifts every time a spec changes and the suite doesn't get updated to match), and anti-patterns like tests that assert on log output rather than actual actuator state. Flaky tests are a specific liability here. An auditor who sees a test that passes on retry after failing once will ask why, and "it's just flaky" is not an acceptable answer when the test in question maps to a safety-critical requirement.

Traceability does not stop at the software boundary. AS9100D clause 8.1.1 requires tracing a part back to its lot, its supplier, and its date code, which means the software bill of materials for flight control firmware needs the same rigor as the physical components sitting next to it on the aircraft. Chain, tonone's supply chain security specialist, builds that pipeline: chain-sbom designs the generation format and the CI integration so every firmware build produces a bill of materials tied to the dependency versions and license status in that specific build, the same build hash Proof's test results reference. When the audit asks what went into this build, the SBOM and the traceability matrix answer the same question from two directions.

Tonone's Chain generates a software bill of materials tied to a specific build hash, giving an aerospace supplier the same lot-level traceability for firmware dependencies that AS9100D already requires for physical components.

The other clock running against every release is the security review under DO-326A/ED-202A, the airworthiness security process that can add months if it starts late or surfaces a finding the team didn't see coming. Warden, tonone's security engineer, runs warden-audit against the same codebase before that review starts, secrets, IAM misconfiguration, injection risk, unencrypted data paths, so the internal security posture is already documented and largely resolved by the time the external security review opens. Running Warden in parallel with Proof's test cycle instead of after it is what keeps the security review from adding a second full audit cycle on top of the first.

Six weeks out from an AS9100 surveillance audit

Take Aerostrat Dynamics, a Tier 2 supplier building the actuator control firmware for a regional jet OEM's flight control system. Six weeks out from an AS9100D surveillance audit, the QA lead runs proof-recon against the current test suite and requirements baseline and gets back a number nobody wanted to see: of 340 requirements tagged in scope for this release, 71% have a passing, traceable test. The rest, 99 requirements, are either untested, tested but not traceable (the test exists, but nothing ties it to a requirement ID), or covered by a test that's currently flaky. Fourteen of those flaky tests sit on safety-critical requirements. Three of the last five firmware builds shipped without an SBOM because the pipeline step got skipped during a rushed patch release.

Proof Recon, Aerostrat Dynamics Actuator Firmware v4.2
Requirements baseline: 340 (DOORS export, tagged by criticality)
Audit: AS9100D surveillance, 6 weeks out

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Coverage summary
  Traceable + passing:        241 / 340  (71%)
  Untested:                    38 / 340  (11%)
  Tested, not traceable:       47 / 340  (14%)
  Flaky (safety-critical):     14 / 340  (4%, unacceptable at any %)

Risk tier breakdown (proof-strategy)
  Safety-critical (Tier 1):   112 reqs,   9 untested,  14 flaky
  Flight-relevant (Tier 2):   156 reqs,  19 untested
  Non-critical (Tier 3):       72 reqs,  10 untested

Supply chain (chain-recon)
  Builds missing SBOM:           3 of last 5
  Unscanned dependencies:       22
  License violations found:      1 (GPL-3.0 in a static-linked module)

Security (warden-recon)
  Open DO-326A findings:          4 (2 high, 2 medium)
  Secrets in firmware config:     1 (hardcoded test cert, non-prod)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Recommendation: fix the 14 Tier 1 flaky tests and the 9 Tier 1
untested reqs first (proof-e2e + proof-audit), backfill SBOM
for the 3 builds (chain-sbom), close the 2 high DO-326A findings
(warden-harden), before touching Tier 3 gaps.
Time to audit-ready: ~3.5 weeks at this priority order.

The recon output turns a vague sense of "we're mostly ready" into a prioritized punch list with a real time estimate. Aerostrat's QA lead now knows the actual gap is not 29%, it's 14 flaky Tier 1 tests and 9 untested Tier 1 requirements, the rest is Tier 3 work that can wait. proof-e2e rewrites the flaky actuator failover tests against the hardware-in-the-loop rig instead of the mocked stub that was causing the intermittent timing failures, and proof-strategy assigns the 9 untested Tier 1 requirements to the right test type instead of defaulting all of them to unit tests. In parallel, Chain backfills the three missing SBOMs and flags the GPL-3.0 violation for legal review before the auditor finds it first, and Warden closes the two high DO-326A findings. Three and a half weeks later, Aerostrat walks into the surveillance audit with a coverage number the auditor can verify line by line, not one they have to take on faith.

Tonone's Proof turns "we think we're mostly ready" into a prioritized, time-estimated gap list against the actual requirements baseline, the artifact an aerospace QA lead needs six weeks before an audit, not after one.

Proof vs the alternatives for aerospace test traceability

None of this is a fair fight for a generalist chatbot or an autocomplete tool, because neither was built to hold a requirements baseline, a coverage target per risk tier, or a build-hash-linked SBOM across a six-week audit cycle. The comparison below is specific to what an aerospace supplier's QA and compliance team actually needs verified, not a generic "writes better tests" claim.

Tonone is free and MIT-licensed. Install it once and Proof, Chain, Warden, and the rest of the team are available in your Claude Code session. You pay only for Claude Code token usage during the work itself.