What a Successful Code Scan Does Not Prove

Quick answer: A successful scan proves the document was rebuilt from routes the scan could resolve, and nothing more. It does not prove the implementation conforms to the spec, behaves correctly at runtime, or is free of defects. Treat unresolved findings as unknown rather than inferred facts, and verify runtime behavior separately through tests and scenario runs. The scan does not run the app.

What the scan actually establishes

The output of a scan is a validated OpenAPI 3.2 document with routes proven through framework tracing and AST analysis, and contracts classified by the completeness gate. That is a real result, but it is a statement about the document and its provenance, not a statement about runtime behavior.

Confusing the two leads to over-trust. A route that was detected and emitted is a route the code mounts, not a behavior that was verified.

Consider a handler that returns 200 on the happy path but throws on an input the scanner never exercises. The scan resolves the route, reads the declared response, and emits the document with a clean status, because every step it performed succeeded. The defect is still there, waiting for the input that triggers it. The scan cannot distinguish a handler that always returns the declared shape from one that only does so for the common inputs, since it never sends a request. That difference only shows up when a test or a scenario actually calls the route and asserts the response.

Claims a scan does not make

Read the gate labels alongside the green status, because a document can be internally valid while carrying gaps that name unresolved behavior.

A successful scan can sit alongside real defects, because none of its checks require the system to run.

How to treat unresolved findings

The honest move is to keep unknowns visible rather than rounding them into facts.

• A gap stays an unknown, not an inferred value.
• A route reached through dynamic loading may be missed and should be checked another way.
• A model-only note remains a hypothesis until verified.
• An empty result is inconclusive, not a pass.

When you meet an unresolved finding, mark it for a dynamic test or a scenario run rather than assuming the scan's silence means the path is safe.

What actually closes the gap

The scan is a starting point that feeds reliable later verification.

1. Review the detected routes and the open gaps.
2. Run tests and smoke checks to cover runtime behavior.
3. Build scenario tests from the scanned spec to verify connected flows.
4. Confirm contract evidence against the declared spec version.
5. Keep the scan result separate from the runtime verdict.

This is the same boundary that keeps static signals from being written as guarantees. A deterministic verdict from real evidence can fail or pass behavior, but the scan itself does not produce that verdict.

Powerduck states the limit rather than hiding it, keeping the scan's provenance and the later runtime evidence as distinct things.

Frequently asked questions

Does a green scan mean the API conforms to its spec? No. It means the document was rebuilt from resolved routes. Conformance needs runtime and contract checks.

Can a successful scan still miss routes? Yes. Dynamic loading and framework entry points can be outside the static view, so verify those separately.

Should I treat gaps as harmless? No. A gap is an unknown, not a confirmed empty. Keep it open until a test or a reviewed fill resolves it.

The bottom line

A successful scan is a rebuilt, provenance-backed document, not a proof of conformance or correctness. It cannot certify runtime behavior it never executes, and it can miss routes reached through dynamic loading. Keep gaps and model-only notes as explicit unknowns, and close the real verification with tests and scenario runs. Over-reading a green scan is how unverified behavior gets mistaken for confirmed behavior. Continue in the code-to-OpenAPI guide.