Governance · 5 min
API versioning strategies that survive breaking changesRead →API Testing
Break your API before an agent does
Tests generated from your catalog, run on a schedule and in CI. Chaos suites throw malformed payloads, expired tokens, and retry storms at every endpoint, the way a confused agent actually will.
Tests written for youScheduled runsChaos suitesBlocks the PR, not the release
Test run #12842 failed
tests 8passed 6failed 2p95 2.4s
POST /payments · happy path96ms
POST /payments · missing amount rejected42ms
POST /payments · no token → 40128ms
GET /orders · pagination contractno cursor, 4,100 tokens
GET /users/{id}/profile · PII redaction88ms
POST /orders/validate · retry storm14 duplicate orders
DELETE /builds/{id} · schema unchanged74ms
POST /auth/passkeys · expired token → 40136ms
2 failures block this PR · contract policy: block-publishView diff
1,284TESTS GENERATED, NOT WRITTEN
214ENDPOINTS COVERED
6CHAOS SCENARIOS PER ENDPOINT
9BUGS CAUGHT BEFORE RELEASE
01Generated coverage
Nobody has time to hand-write 1,284 tests
Elva already knows every endpoint, its schema, and its auth. That is enough to generate the happy path, the boundary cases, the auth negatives, and the malformed payloads, per endpoint, without anyone maintaining a Postman folder.
POST /paymentsgenerated suite14 tests
Creates a payment with valid body201 · schema match
Rejects missing amount400 · field required
Rejects negative amount400 · boundary
Rejects request with no token401 · auth negative
Rejects token missing write:payments403 · scope negative
Handles duplicate submitno idempotency key defined
Survives 60 retries in 8s14 duplicates created
Assertions from the schemaStatus codes, required fields, types, enums, and nullability come from the spec Elva already generated, so a schema drift fails a test instead of surprising a consumer.
Auth negatives by defaultEvery protected endpoint gets tested without a token, with an expired one, and with a token missing the scope. That is how the four endpoints with no auth got found.
Editable, not magicGenerated tests are readable files you can pin, edit, or delete. Add your own domain assertions and Elva keeps them through the next rescan.
02Chaos suites
Agents are the worst client you will ever have
A human reads the error and stops. An agent retries sixty times with slightly different arguments and reports success anyway. Chaos suites simulate that behavior on purpose, so you find the crash instead of an auditor.
01Malformed payloads
Wrong types, truncated JSON, extra fields, nulls where a string is required. What an agent sends when it half-understands your schema.found: 3 unhandled 500s02Retry storms
The same call sixty times in eight seconds, arguments mutating slightly, no idempotency key in sight.found: 14 duplicate orders03Auth edge cases
No token, expired token, valid token with the wrong scope, and a token for a revoked key.found: 4 endpoints with no auth04Schema drift
Compares live responses against the contract on every run, so a removed field fails here rather than in an agent conversation.found: buildLog removed05Payload bloat
Measures response size and token cost per call, because verbose output is a tax on every agent turn.found: 4,100 tokens on /orders06Latency under load
Concurrent calls against the slowest tools, watching p95 and timeouts, because a slow tool multiplies traffic.found: sync_ledger p95 2.4sChaos finding · retry storm on POST /orders/validatereproduced 3 of 3 runs
WHAT THE SUITE DID
60 identical requests in 8 seconds, no idempotency keyarguments mutated by one field each retrytoken refreshed mid-flight
WHAT BROKE
14 duplicate orders createdp95 climbed 138ms to 2.4s500s returned with a stack trace in the body
SUGGESTED FIX
require an idempotency keyrate limit per key at the gatewaystrip internal errors from responses
03Automation
In CI, on a schedule, and against production
Suites run on every pull request, nightly against staging, and hourly as a smoke test on production. A failing contract test posts on the PR and, if your policy says so, blocks the publish.
Schedules4 active
On every pull requestcontract + generated tests · ~90s
blockingNightly on stagingfull suite + chaos · 02:00 UTC
report to SlackHourly smoke on productionread-only checks, no writes
alerts on 2 failuresBefore every MCP publishtool schemas verified against live responses
blockinggithub · PR #482 checks
❯ checks for PR #482 · payments-api
✓ elva/contract-check 3 contracts verified
✓ elva/generated-tests 1,282 passed
✕ elva/chaos-suite 2 failed
POST /orders/validate 14 duplicate orders on retry storm
GET /orders unpaginated, 4,100 tokens per call
◷ elva/mcp-publish blocked by policy
refund_charge held back until schema verifies
→ review required · 3 stakeholders notified
Closes the loop
Testing that feeds the catalog, the contract, and the gateway
A failing test is not just a red check. It lowers the endpoint score, blocks the contract publish, and keeps a broken tool out of your MCP server.
Scores drop when tests failReliability is part of the grade. An endpoint that fails its chaos suite cannot sit at 92% in the catalog.API Catalog →
Contracts stay honestA contract promises a shape. Tests verify the live API still returns it, and block the publish when it does not.API Contracts →
Broken tools never shipEvery MCP publish verifies tool schemas against live responses first. That is why refund_charge is still held back.MCP Management →
FAQ
How does Elva generate API tests?
From the same schemas and constraints in the catalog. A field capped at 1 to 500 in the schema editor is rejected at 501 by the generated tests, because tests, contracts, and MCP tool schemas all read one source.
Can tests run automatically?
Manual runs are on every plan. Scheduled runs, CI triggers, and chaos suites are included on Business and Enterprise.
Why test APIs before agents use them?
Agents exercise edge cases relentlessly and retry failures at machine speed. Generated tests find the gaps first, so the fix ships before an agent discovers the bug in production.
Find the bug on your terms, not an agent’s
Connect a repo and your first suite runs in minutes, no test code required.