API Testing in CI: Hoppscotch vs Powerduck
Quick answer
Short answer: Both can run in CI, but the source of the checks differs. A collection-based runner executes the requests you authored; a spec-first runner derives the checks from the contract and verifies conformance. Use the first to run a fixed set of calls; use the second to prove the responses match the API.
Short answer: Both can run in CI, but the source of the checks differs. A collection-based runner executes the requests you authored; a spec-first runner derives the checks from the contract and verifies conformance. Use the first to run a fixed set of calls; use the second to prove the responses match the API.
What a collection runner does in CI
It executes the saved requests and their assertions, which is a reliable way to run a known set of checks. It is only as current as the collection, and it verifies the assertions rather than the whole contract.
What a spec-first runner does
| Property | Collection runner | Spec-first |
|---|---|---|
| Checks come from | Authored requests | The contract |
| Coverage | What was saved | Declared operations |
| Conformance | Assertions only | Full schema check |
| On spec change | Update collection | Reflects the spec |
The failure that CI should catch
A response that violates the contract but passes the authored assertions is the gap. A spec-first runner checks the full response, so the violation fails the build even if no one wrote an assertion for it.
How to combine them
Run the fixed behavioral calls and the contract checks as two stages. The first proves the flows work; the second proves the responses are valid.
This is part of the Powerduck vs Hoppscotch comparison. Next, read OpenAPI-first vs collection-first.
Frequently asked questions
What does a collection runner do in ci?
It executes the saved requests and their assertions, which is a reliable way to run a known set of checks. It is only as current as the collection, and it verifies the assertions rather than the whole contract.
What does a spec-first runner do?
PropertyCollection runnerSpec-first Checks come fromAuthored requestsThe contract CoverageWhat was savedDeclared operations ConformanceAssertions onlyFull schema check On spec changeUpdate collectionReflects the spec
Why does the failure that ci should catch?
A response that violates the contract but passes the authored assertions is the gap. A spec-first runner checks the full response, so the violation fails the build even if no one wrote an assertion for it.
How to combine them?
Run the fixed behavioral calls and the contract checks as two stages. The first proves the flows work; the second proves the responses are valid. This is part of the Powerduck vs Hoppscotch comparison. Next, read OpenAPI-first vs collection-first.