Two teams argue in a pull request. One wants a Pact broker, the other wants the OpenAPI document run against staging, and neither can say what the other would miss. The provider ships a field rename the following week, and both proposals would have caught it, which settles nothing about the change after that.
What you get
You will end up able to say which style catches which change, and whether one of them is enough for your services. This is for you if contract testing is a decision your team has not made yet.
Short answer
Run consumer-driven tests when the provider is a team who will execute your expectations in their build, and the question is whether their change breaks a known caller. Run spec-driven tests when a published document is the contract and the question is whether the service still matches it. They fail on different changes, which is why many services run both.
You will need
A provider and at least one consumer, or a published API description. The choice below depends most on whether the provider will run somebody else’s tests, which is an organizational fact rather than a technical one. Answer that first, because it removes one of the two options before any tooling is compared.
Approaches compared
| Approach | When it fits | What it costs you | When to pick something else |
|---|---|---|---|
| Consumer-driven contracts | Services inside one organization, where providers will run consumer expectations | A broker to run, and every consumer has to write and publish its expectations | The provider is a vendor who will never execute your tests |
| Schema validation | You want responses checked continuously rather than at release time | It checks shape alone, so a correct shape carrying wrong values passes | The question is whether a specific consumer still works |
| Spec-driven tests | A published OpenAPI document is the contract with callers you cannot enumerate | Coverage is only as good as the document, and a thin document tests very little | There is no document, and nobody will maintain one |
| No contract tests | A single team owning both sides, deploying them together | Every cross-service change is verified by hand, or in production | The two sides can be deployed independently |
The two styles answer different questions. Consumer-driven asks whether a change breaks a caller you know about. Spec-driven asks whether the service still matches what it published. A provider with three internal consumers and a public document has both questions, and neither style answers the other’s.
Watch them disagree
The clearest way to choose is to run both against the same four upstream changes.
/**
* Consumer-driven: assert only the fields this consumer reads. Everything
* else in the response is somebody else's business.
*/
export function consumerCheck(body) {
const failures = []
if (typeof body.id !== 'string') failures.push('id is not a string')
if (typeof body.amount_cents !== 'number') failures.push('amount_cents is not a number')
return failures
}node compare.mjsunchanged
consumer-driven: passes
spec-driven: passes
field added upstream
consumer-driven: passes
spec-driven: fails (auto_advance is not in the document)
field this consumer reads changed type
consumer-driven: fails (amount_cents is not a number)
spec-driven: fails (amount_cents is string, the document says number)
field another consumer reads removed
consumer-driven: passes
spec-driven: fails (currency is missing)The second and fourth blocks are the whole decision. An added field is a false alarm for a consumer and a real finding for a document that claims to be complete. A removed field that this consumer never read is invisible to it and breaks somebody else.
Read the disagreements as a choice
Take the second block first. If your provider adds fields often and your consumers are few, a spec-driven suite generates work for changes nobody is affected by. If your document is a published promise, the same failure is the point of having the document.
The fourth block is the argument for running both. Consumer-driven tests cover the consumers who wrote expectations, and a caller who wrote none is not represented. That is the gap a spec-driven run over the published document fills, without needing every caller to participate.
The third block is the one both styles catch, and it is worth noticing that they report it differently. The consumer names the field it could not use. The document names the field and what it was supposed to be. For a provider deciding whether to ship, the second message is more useful, because it says which side of the disagreement is wrong.
Cost is the other axis, and it does not follow the same order. Pact needs a broker, a publishing step in every consumer build, and a verification step in the provider’s. A spec-driven run with Schemathesis needs a document and a target, which is a job in one repository. The consumer-driven setup buys something for that price, which is that a provider’s build fails before the change reaches anybody, rather than after it is deployed to an environment somebody tests against.
Check it worked
The property to hold is that each style fails on the changes it is supposed to catch and stays quiet on the rest.
node --test checks.test.mjs1..4
# tests 4
# suites 0
# pass 4
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 112.230272Four tests, one per row of the comparison, and the assertions are that the two styles differ where this page says they do. Writing that assertion down is the part worth copying: a comparison in a document drifts from the tools it describes, and one in a test suite fails when it does.
When it goes wrong
A consumer-driven suite becomes a second copy of the provider’s schema. A consumer that asserts on every field it receives, rather than the ones it reads, produces exactly the false alarms the style exists to avoid. Review contracts for what the consumer uses, and treat a growing contract as a question rather than as thoroughness.
The second failure is a spec-driven run against a document nobody maintains. It passes, because it tests the service against a description that was updated to match whatever the service does. Check the document into the same repository as the service and generate it from the code, or review it in the same pull request as the handler.
When not to do this
Do not adopt a broker before the provider has agreed to run the contracts. A published contract that nobody verifies is a fixture with more infrastructure around it.
Do not run spec-driven tests against production. Generated request cases include values a real service will act on, and the ones that pass are as likely to create data as the ones that fail.
Do not treat either style as a substitute for testing the values. Both check shape, and a currency that arrives in the wrong denomination is correctly shaped and wrong.
Do not run both styles in the same build step. They fail for different reasons and need different responses, and combining them produces a red build whose message does not say which question failed.
Related how-tos
Last verified
Verified 2026-09-06 against Node 22.22.2. Both output blocks are what the preceding command printed, against two small checkers written for the comparison rather than against Pact or Schemathesis.