Blog

API Idempotency Testing: Safe Retries, Duplicates, and Payment Replays

by Rajesh K | Oct 7, 2026 | API Testing, Blog, Latest Post | 0 comments

A customer submits a payment. The server processes it, but the response disappears before reaching the client. The application sees a timeout and retries. Did the customer pay once, or twice? The timeout alone cannot answer that question. It describes what the client observed, not necessarily what the server committed. Stripe’s error-handling documentation explicitly distinguishes network failures from known outcomes and recommends preserving the same idempotency key and parameters when retrying an uncertain request. API idempotency testing verifies that repeated attempts to perform one logical operation do not produce additional intended business effects. A useful test suite must examine more than repeated HTTP responses. It must verify persistent state, downstream payment activity, recovery behavior, and competing requests.

The central question is not simply, “Did both requests return the same payment ID?”

It is: after every retry, crash, and concurrent attempt has been resolved, did the system perform only the business operation that was intended? Codoid’s API and backend testing services apply this exact discipline when validating payment and workflow endpoints.

Table of Content

• 1. Define the Guarantee You Are Testing
• 2. Write the Idempotency Contract Before Writing Assertions
• 3. Build an Independent Business-State Oracle
• 4. Test Safe Retries at the Actual Failure Boundaries
• 5. Automate Duplicate and Concurrent Requests
• 6. Expand Beyond the Same-Key Happy Path
• 7. Distinguish Duplicate Requests From Concurrency Conflicts
• 8. Follow Payment Replays Across Every Service Boundary
• 9. Verify Retention, Isolation, and Recovery Behavior
• 10. Make the Suite Expose Implementation Defects
• Conclusion

HTTP defines idempotency in terms of the intended server-side effect of repeated identical requests. Safe methods, along with PUT and DELETE, are idempotent by definition. A POST operation needs an endpoint-specific contract to provide retry protection. Idempotency also does not require every response to be identical or prohibit per-attempt logging.

For a payment-capture endpoint, translate that definition into explicit invariants.

    For one logical capture, within the documented scope and retention window:

    successful_capture_count <= 1
    gross_captured_amount <= requested_capture_amount
    

For a test scenario configured to succeed after temporary faults are removed, add a recovery requirement:

    Eventually, within the test's recovery budget:

    successful_capture_count == 1
    gross_captured_amount == requested_capture_amount
    

Both requirements matter. A server that rejects every request can satisfy “at most one capture” while failing to complete any legitimate payment.

This distinction separates safety, meaning nothing happens twice, from recovery, meaning accepted work does not remain unresolved forever.

It also avoids an unrealistic assertion about network traffic. Several HTTP attempts may be necessary. The test should count committed business effects, not demand exactly one network delivery.

An idempotency key is meaningful only within a defined contract. For an API you control, document at least the following decisions.

A useful design model separates lookup from request identity:

    lookup_identity = (authenticated_tenant, operation_namespace, idempotency_key)

    request_fingerprint = semantic_identity_of_the_operation
    

Keep those concepts separate. The lookup identity finds an existing operation. The fingerprint determines whether the new request actually represents that operation.

For a payment, fingerprint candidates include the amount, currency, destination, payment method, and business reference. A fresh tracing identifier should not normally turn a retry into a different business request. Conversely, changing the amount must not silently retrieve an unrelated success.

Provider Behavior Is Not Interchangeable

Stripe retains the first execution’s status and body, including 500 responses. It checks reused keys against the original parameters. Requests rejected before endpoint execution, including certain validation and concurrency failures, do not receive the same result-retention treatment.

PayPal documents a different replay model: PayPal-Request-Id can retrieve the latest status of the previous request. Support and retention depend on the API, and a simultaneous request with the same identifier may fail while the first is processed.

Adyen documents 409 or 422 responses with error code 704 for certain duplicates that arrive before completion. Its transient-error response header provides retry guidance.

Consequently, a test that accepts any “reasonable” response code is too permissive. Assert the documented status, machine-readable error code, and permitted follow-up behavior. A broader treatment of these checks appears in the payment API testing guide and the REST API testing checklist.

A repeated success response is evidence about the response contract. It is not sufficient evidence about payment safety.

For each test operation, collect observations from authoritative systems rather than from the same replay cache being tested. Useful observations include the payment records, successful provider captures, gross captured amount, ledger posting groups, and fulfillment or receipt effects.

For an isolated payment of 2,500 minor units, an example terminal assertion is:

    payment_ids = [the_expected_payment_id]
    successful_captures = 1
    gross_captured_minor = 2500
    capture_posting_groups = 1
    fulfillment_effects = 1  # When required by the business contract
    

Count the logical ledger posting group, not an arbitrary number of ledger rows. Likewise, count gross captures rather than only the final net balance. Two charges followed by a compensating refund should still fail a duplicate-charge test.

Event-driven systems require another distinction. Multiple deliveries of a message do not necessarily mean multiple business effects. AWS’s transactional-outbox guidance explicitly warns that duplicate messages remain possible and recommends idempotent consumers.

Your oracle should therefore distinguish:

    message_delivery_count        # May exceed one

    committed_business_effects    # Must remain within the contract
    

Do not end an asynchronous test at the first successful observation. Drain the scheduled work, wait for a defined completion barrier, or observe through the relevant retry horizon before declaring that no duplicate effect occurred.

“Make the request time out” is not a sufficiently precise test specification.

The same client-visible timeout can occur before execution, during execution, or after the business effect has committed. Treat those as different scenarios.

The Critical Payment Test: Commit, Lose the Response, Retry

Arrange a successful test payment and a unique key. Allow the payment to commit, then deliberately drop the response on its way back to the client. Retry with the original key and original operation parameters.

The passing result is not merely another successful response. Verify the same logical payment, one successful capture, the correct gross captured amount, and no additional ledger or fulfillment effect.

Use a controlled fault point or proxy rule that confirms the relevant commit occurred before discarding the response. An extremely short client timeout does not prove that the server reached the intended failure boundary.

Do Not Classify Every 500 as a Failed Payment

Stripe advises treating certain 500 outcomes as indeterminate because side effects may have occurred. It also warns against switching to a fresh key merely to escape the error: the original operation may already have changed state.

Test the recovery workflow separately from the replay response. Depending on the API, that workflow may involve an operation-status lookup, a provider-resource lookup, or later reconciliation.

Test the Retrying Client, Not Just the Receiving Server

Verify that the client preserves the logical key and request parameters across attempts, including process restarts where required. Give each HTTP attempt a separate tracing identifier without changing the operation identity.

Test retry eligibility, maximum attempts, overall deadlines, and server-provided retry guidance. Use capped backoff with jitter for eligible transient failures. AWS’s analysis of retry behavior shows why adding jitter spreads retry traffic rather than keeping clients synchronized.

Do not apply a blanket “retry every conflict” rule. For example, Adyen distinguishes retryable transient responses through its documented header behavior.

The following example uses Python, pytest, and HTTPX. HTTPX documents that its synchronous Client can be shared between threads. Python’s Barrier provides a way to release a group of waiting threads together.

This is an integration-test example for an illustrative API contract, not a direct Stripe, PayPal, or Adyen test.

Assume POST /payments returns 201 with a completed payment. Completed replays return the original status and JSON body. A changed payload returns 422 with idempotency_payload_mismatch. An overlapping duplicate returns 409 with idempotency_in_progress.

The example also assumes a private test-only endpoint, /__test/payment-audit, that reads authoritative business records. Its settled flag must mean the harness has drained the operation’s scheduled work, not merely observed one success. This endpoint is custom test infrastructure and must not be publicly exposed.

Set API_BASE_URL, API_TOKEN, and TEST_PAYMENT_METHOD for an isolated test deployment, then install pytest and httpx.

    # test_idempotency.py

    import os
    import time
    from concurrent.futures import ThreadPoolExecutor
    from threading import Barrier
    from typing import Any, Iterator
    from uuid import uuid4

    import httpx
    import pytest

    @pytest.fixture
    def api() -> Iterator[httpx.Client]:
        with httpx.Client(
            base_url=os.environ["API_BASE_URL"],
            headers={
                "Authorization": f"Bearer {os.environ['API_TOKEN']}"
            },
            timeout=httpx.Timeout(10.0, connect=3.0),
            limits=httpx.Limits(
                max_connections=16,
                max_keepalive_connections=16,
            ),
        ) as client:
            yield client

    def new_case() -> tuple[str, dict[str, Any]]:
        return str(uuid4()), {
            "reference": f"idem-test-{uuid4()}",
            "amount_minor": 2500,
            "currency": "USD",
            "payment_method": os.environ["TEST_PAYMENT_METHOD"],
        }

    def submit(
        api: httpx.Client,
        key: str,
        payload: dict[str, Any],
    ) -> httpx.Response:
        return api.post(
            "/payments",
            json=payload,
            headers={
                "Idempotency-Key": key,
                "X-Request-ID": str(uuid4()),  # Unique per HTTP attempt.
            },
        )

    def assert_one_capture(
        api: httpx.Client,
        reference: str,
        payment_id: str,
        amount_minor: int,
    ) -> None:
        deadline = time.monotonic() + 15.0
        observed: dict[str, Any] = {}

        while time.monotonic() < deadline:
            response = api.get(
                "/__test/payment-audit",
                params={"reference": reference},
            )
            response.raise_for_status()
            observed = response.json()

            # Fail immediately on an observed duplicate.
            assert observed["successful_captures"] <= 1, observed

            if observed["settled"]:
                assert observed["payment_ids"] == [payment_id], observed
                assert observed["successful_captures"] == 1, observed
                assert (
                    observed["gross_captured_minor"] == amount_minor
                ), observed
                assert observed["capture_posting_groups"] == 1, observed
                return

            time.sleep(0.1)

        pytest.fail(f"Operation did not settle: {observed}")

    def test_sequential_replay_and_payload_mismatch(
        api: httpx.Client,
    ) -> None:
        key, payload = new_case()
        first = submit(api, key, payload)
        assert first.status_code == 201, first.text
        original = first.json()
        assert original["status"] == "succeeded", original

        for _ in range(5):
            replay = submit(api, key, payload)
            assert replay.status_code == 201, replay.text
            assert replay.json() == original

        changed = {**payload, "amount_minor": 3000}
        conflict = submit(api, key, changed)
        assert conflict.status_code == 422, conflict.text
        assert (
            conflict.json()["error"]["code"]
            == "idempotency_payload_mismatch"
        )

        # A mismatch must not overwrite or poison the original result.
        replay = submit(api, key, payload)
        assert replay.status_code == 201, replay.text
        assert replay.json() == original

        assert_one_capture(
            api, payload["reference"], original["id"],
            payload["amount_minor"],
        )

    def test_concurrent_same_key(api: httpx.Client) -> None:
        key, payload = new_case()
        workers = 16
        start = Barrier(workers, timeout=5.0)

        def attempt(_: int) -> httpx.Response:
            start.wait()
            return submit(api, key, payload)

        with ThreadPoolExecutor(max_workers=workers) as pool:
            responses = list(pool.map(attempt, range(workers)))

        successes: list[httpx.Response] = []
        for response in responses:
            if response.status_code == 201:
                successes.append(response)
            else:
                assert response.status_code == 409, response.text
                assert (
                    response.json()["error"]["code"]
                    == "idempotency_in_progress"
                )

        assert successes, "No attempt completed successfully."

        original = successes[0].json()
        assert original["status"] == "succeeded", original
        assert all(r.json() == original for r in successes)

        final_replay = submit(api, key, payload)
        assert final_replay.status_code == 201, final_replay.text
        assert final_replay.json() == original

        assert_one_capture(
            api, payload["reference"], original["id"],
            payload["amount_minor"],
        )
    

Run the example with:

    pytest -q test_idempotency.py
    

Several details are intentional.

First, the concurrent test permits multiple successful HTTP responses. Those can be valid replays of one execution. Requiring exactly one 201 would incorrectly reject the stated contract.

Second, the test rejects unexpected errors rather than accepting a broad collection of status codes.

Third, the audit assertion is independent of response equality. For an API that returns current state instead of a stored response, replace full-body equality with the documented identity and state assertions.

Finally, the client barrier encourages overlap but does not prove that requests overlapped inside the server. Add a deterministic server-side test gate after the first request claims the key. Hold that request, send duplicates, verify the in-progress behavior, and then release the owner.

The example covers the essential mechanics. A production API idempotency testing suite needs additional dimensions.

Same Key, Different Operation Parameters

Change the amount, currency, destination, payment method, business reference, or target resource independently. Verify that the server does not execute the altered operation or overwrite the original fingerprint.

Repeat this test while the first request is still running. The immediate response may follow the documented in-progress policy, but the altered operation must never be executed under the original key.

Equivalent Serialization

Where the API promises semantic JSON comparison, reorder object properties and vary insignificant whitespace. Conversely, test distinctions such as omitted versus null fields, array order, and numeric representations according to the schema.

Do not silently normalize away meaningful differences merely to make tests pass.

Different Keys

Submit genuinely independent operations using different keys and verify that neither is incorrectly suppressed.

Also test the same business intent under different keys. This is a separate requirement: a rule such as “only one full payment for this order” must be enforced through business identity, not assumed to follow from client-generated retry keys.

Include a negative control where identical payloads are legitimately allowed to represent separate operations. Otherwise, payload-based deduplication or a business uniqueness constraint may hide a broken idempotency-key implementation.

Invalid and Missing Keys

Exercise missing, empty, oversized, and malformed values, along with ambiguous duplicate header fields. Verify the documented rejection or optional-key behavior and confirm that rejected requests create no payment effects.

Keep validation-failure tests distinct from post-execution failure tests. A single blanket assertion about whether all errors consume a key will miss that boundary.

Same-key requests ask the system to recognize one operation repeatedly.

Different-key requests may represent competing operations that must not both succeed.

For example, two separately identified refunds could each be individually valid while their combined amount exceeds the refundable balance. Idempotency protection for each refund does not, by itself, enforce the shared balance constraint.

Test that business invariant directly.

For versioned resources, HTTP conditional requests provide another conflict-control mechanism. If-Match uses strong entity-tag comparison to prevent stale writes. A failed precondition normally produces 412 Precondition Failed, subject to the documented already-applied-operation exception.

A useful test schedule is:

    Client A reads resource version "v7".
    Client B reads resource version "v7".

    A submits update A, key A, If-Match: "v7".
    B submits update B, key B, If-Match: "v7".
    

Verify that only an allowed version transition commits.

Verify that the stale update does not overwrite it.

Then replay the winning operation using its original key and original precondition. Confirm the documented replay behavior without a second mutation.

Keep these outcomes distinct in assertions and telemetry: an idempotency mismatch, an operation still in progress, and a stale resource version are different conditions even when an API happens to use overlapping status codes.

Protecting the public endpoint is only the beginning.

For a payment workflow, test the complete path:

    Client
        ↓
    Payment API
        ↓
    Payment provider
        ↓
    Local ledger
        ↓
    Event publication
        ↓
    Fulfillment or receipt processing
    

Preserve Downstream Operation Identity

For your design, persist a logical payment-operation identifier before making the provider call. Associate the downstream idempotency key with that operation and action.

A capture and a refund need distinct identities. Two intentionally separate partial captures also need distinct identities. Retries of the same capture must reuse its identity.

The critical recovery schedule is: the provider accepts the operation, the application crashes before recording success, and a replacement worker resumes processing. Verify that recovery reuses or locates the existing provider operation instead of creating a new one. PayPal documents precisely this kind of capture-timeout recovery using the original request identifier.

Do not require exactly one provider HTTP call in this fault scenario. Require at most one committed provider effect.

Test Webhook Duplication Independently

Stripe documents duplicate event deliveries and does not guarantee event ordering. It recommends tracking processed event identifiers and notes that some duplicate notifications can arrive as separate event objects.

Test repeated delivery, concurrent delivery, out-of-order delivery, and distinct notifications that represent the same documented business transition. Verify that ledger posting and fulfillment remain correct.

Also inject a crash between the business mutation and recording that the event was processed. The test should expose designs that can either repeat the effect or mark work complete before it actually happens.

Keep Security Replay Checks Separate

A legitimate redelivery is not the same as replaying an old captured signature. Stripe signs delivery timestamps and generates a new timestamp and signature for each retry.

Test valid redelivery, invalid signatures, and expired signing timestamps separately. A previously seen event identifier must not become a shortcut around authentication.

Retention Boundaries

Stripe permits pruning keys once they are at least 24 hours old and treats reuse after pruning as a new request. That is a bounded guarantee, not permanent business deduplication.

For your own idempotency store, use a controllable clock to test just before expiration, at the boundary, after logical expiration, and after physical cleanup.

Verify what happens to operations still in progress when their retention or ownership timers expire. Do not let a cleanup test pass merely because the record disappeared. Establish whether a retry can now duplicate unfinished work.

Test the combined local and provider retention contract as well. A longer local retention period does not extend the provider’s documented guarantee.

Tenant and Authorization Isolation

Reuse the same literal key under different tenants and operation namespaces. Depending on the contract, those requests may be independent or rejected, but they must not expose another caller’s result.

Repeat a request after credential rotation and after access revocation. For your API, explicitly require appropriate authorization before returning a stored result. OWASP’s object-level authorization guidance applies whenever an endpoint returns data associated with an identified object.

Restarts, Failover, and Expired Ownership

Restart workers between attempts. Route duplicates to different application instances. Exercise store outages, failover, and recovery after an ownership lease expires.

A particularly useful schedule pauses an old worker, allows recovery to transfer ownership, and then resumes the old worker. Verify that stale ownership cannot authorize conflicting writes or additional payment effects.

Test regional behavior against the actual scope guarantee. Adyen, for example, explicitly states that duplicate-key checks do not extend across simultaneously targeted regional endpoints.

For payment systems, make “do not silently remove retry protection when the store fails” an explicit resilience requirement.

A classic race-prone implementation is conceptually:

    Check whether key exists.
    Perform the payment.
    Save the key and result.
    

Two requests can both pass the initial check. A test that exercises only sequential retries will not expose that schedule.

Where the relevant state is local, test an atomic claim backed by an appropriate uniqueness constraint or conditional write. PostgreSQL’s ON CONFLICT mechanisms provide database-level conflict handling, but the surrounding workflow still needs correct transaction boundaries.

AWS’s idempotent-API guidance emphasizes atomicity between recording the request token and the related mutations. External provider calls require additional recovery coordination because they are outside that local transaction.

For event publication, a transactional outbox can couple the local business update with the event record. It does not remove the need to test duplicate delivery and idempotent consumption.

Organize execution around complementary layers:

• Contract tests for replay responses, mismatches, key validation, and scope.
• Deterministic integration tests for overlapping requests, commit-boundary failures, worker crashes, and downstream redelivery.
• Stress and recovery tests for varied fan-out, hot keys, many independent keys, failover, and retention transitions.

Record enough evidence to reconstruct each run: logical operation ID, per-attempt request ID, scoped key hash, claim owner, provider operation ID, state transitions, and recovery actions. Avoid logging payment credentials or sensitive payloads.

Finally, deliberately break the implementation. Disable the atomic claim, generate a new downstream key during retries, or remove a consumer’s deduplication check. The corresponding tests should fail for the expected reason.

That is a stronger confidence signal than a green suite that has never demonstrated sensitivity to the defect it claims to prevent.

API idempotency testing is not a two-request status-code comparison. It is a test of business invariants under repeated delivery, partial failure, and competing execution. A trustworthy suite verifies the documented replay contract, independently counts committed effects, exercises ambiguous payment outcomes, distinguishes duplicates from resource conflicts, and follows recovery through downstream consumers.

Build the contract before writing assertions. Use an independent oracle that reads authoritative systems rather than the replay cache. Test the actual failure boundaries, including commit, lost response, and retry. Then deliberately break the implementation and confirm that the suite catches the defect. Codoid’s QA automation services help teams build idempotency test suites that surface duplicate-charge and lost-payment defects before they reach production.

Need Help Building Your API Idempotency Testing Strategy?

Talk to an API Testing Expert

Frequently Asked Questions

API Negative Testing: Authentication, Authorization, Boundaries & Error Contracts

by Rajesh K | Oct 5, 2026 | API Testing, Blog, Latest Post | 0 comments

API negative testing deliberately sends invalid, unauthorized, malformed, out-of-range, or excessive requests to verify that an API rejects them safely and predictably. A successful negative test confirms not only the expected error status, but also that no forbidden state change occurred, no sensitive data leaked, and the documented error contract remained intact. A test that merely checks response.status == 400 is therefore incomplete. Good negative testing asks a stricter question: when a client violates one specific API rule, does the API fail in the exact way its security policy and contract require?

That distinction matters for authentication, authorization, input validation, rate limiting, and error handling because an API can return an error while still leaking data, modifying state, executing expensive backend work, or breaking client integrations. Codoid’s API and backend testing services apply these exact checks to production APIs.

Key Takeaways

• Treat authentication and authorization as different test dimensions. Authentication establishes identity. Authorization decides whether that identity may perform a specific action.
• Test boundaries immediately below, at, and immediately above every documented limit instead of relying only on obviously invalid values.
• Validate rate limits around the actual threshold, reset boundary, concurrency model, and identity dimension used by the limiter.
• Assert the complete failure contract: HTTP status, headers, media type, machine-readable error fields, and absence of prohibited information.
• For every rejected write operation, verify no state mutation or unintended downstream side effect occurred.
• Generate negative tests from OpenAPI or JSON Schema where practical, but supplement them with security and business-rule cases the schema cannot express.

Table of Content

• What Is API Negative Testing?
• What Does API Negative Testing Include?
• Why Is API Negative Testing Important?
• Step 1: Turn the API Contract into Rejection Rules
• Step 2: Create Controlled Identities
• Step 3: Test Authentication Failures First
• How Should Authorization Be Negative-Tested?
• How Should Input Boundaries Be Tested?
• How Should Rate Limits Be Negative-Tested?
• How Should Error Contracts Be Tested?
• Practical Example: Multi-Tenant Orders API
• API Negative Testing Best Practices
• Common API Negative Testing Mistakes
• Troubleshooting Common Failures
• Tools for API Negative Testing
• Limitations and Risks
• Conclusion

API negative testing covers requests that the service is expected to reject rather than successfully process. Instead of validating that a valid request produces a valid result, it validates that an invalid request produces the correct failure without causing harm.

A typical API negative testing workflow models the operation as a sequence of controls:

    Client request
        ↓
    Authentication
        ↓
    Authorization
        ↓
    Request and schema validation
        ↓
    Resource and rate controls
        ↓
    Business operation
        ↓
    Response and error mapping
    

Real systems may enforce these controls in a different order. A gateway might rate-limit before authentication, for example, while application-level quotas could be evaluated after identity has been established. The test should therefore focus on externally observable guarantees rather than assume an internal implementation.

For each negative case, verify five things:

• The request is rejected.
• The status and protocol headers are correct.
• The response body matches the documented error contract.
• No unauthorized data is returned.
• No prohibited state change or downstream side effect occurs.

The fifth assertion is particularly important for write operations. An API returning 403 after a database write has already occurred is still broken. This is the core idea behind API negative testing.

Common categories in a structured API negative testing plan include authentication, authorization, input validation, resource limits, rate limiting, and the error contract itself.

Negative testing is related to security testing and fuzz testing, but the terms are not interchangeable. Security testing investigates exploitable weaknesses and broader attack paths. Fuzzing generates large or unusual input spaces to discover unexpected behavior. API negative testing starts from an explicit rule, such as “quantity must be from 1 to 100,” and verifies behavior when that rule is violated.

APIs concentrate security-sensitive behavior at trust boundaries. The OWASP API Security Top 10 identifies broken object-level authorization, broken authentication, broken object-property authorization, unrestricted resource consumption, and broken function-level authorization among its leading API risks.

Negative tests expose failures that positive happy-path tests often cannot see. For example:

• A valid customer retrieving their own order proves that GET /orders/{id} works. It does not prove that the customer cannot retrieve another tenant’s order.
• A valid JWT proves that authentication can succeed. It does not prove that an expired or wrong-audience token is rejected.
• A request with quantity=50 proves that a normal value works. It does not prove that 0, 101, 1.5, “50”, or an extremely large integer is handled correctly.
• Ten successful requests prove that an endpoint responds. They do not prove what happens at the rate-limit threshold.
• Receiving a 400 proves that something failed. It does not prove that the response matches the API’s documented error schema.

The practical goal of API negative testing is therefore controlled failure. Invalid requests should fail without bypassing security, corrupting state, creating unexpected costs, or forcing API consumers to reverse-engineer inconsistent errors.

Start with the OpenAPI description, JSON Schema definitions, authentication configuration, authorization policy, quota rules, and business requirements.

The OpenAPI Specification describes operation responses as mappings between HTTP response codes and expected responses and explicitly says known errors should be documented. JSON Schema provides explicit validation keywords such as minimum, maximum, minLength, maxLength, minItems, and maxItems, which make natural sources for boundary tests.

Convert every constraint into a testable rule. For example:

    quantity:
      type: integer
      minimum: 1
      maximum: 100

    sku:
      type: string
      minLength: 1
      maxLength: 32

    lineItems:
      type: array
      minItems: 1
      maxItems: 50
    

Expected result: every documented constraint maps to at least one passing case and one or more failing cases.

Prepare test credentials representing the meaningful security states in the system. For a multi-tenant API, that might include:

• anonymous
• valid_user_tenant_A
• valid_user_tenant_B
• tenant_A_admin
• read_only_user
• expired_token_user
• revoked_token_user
• token_without_required_scope

Authentication and authorization must be tested separately. OWASP defines authentication as verifying an entity’s identity, while authorization determines whether that entity is approved to perform a requested operation. OWASP recommends deny-by-default authorization and permission checks on every request. For a deeper look at these controls, see Codoid’s guide on API test automation using REST Assured.

Expected result: a test can deliberately vary identity, role, scope, tenant, ownership, and credential validity without relying on production accounts.

Common mistake: using a single administrator token for the entire automated API negative testing suite. That can make authorization defects invisible.

For every protected endpoint, test at least the authentication states applicable to the API:

• No Authorization header
• Empty credentials
• Unsupported authentication scheme
• Malformed bearer token
• Invalid signature
• Expired token
• Token not yet valid
• Wrong issuer or audience
• Revoked credential where revocation is supported
• Multiple competing credential mechanisms when prohibited

For HTTP protected resources, RFC 9110 defines 401 Unauthorized as the response when the request lacks valid authentication credentials, and requires a server generating 401 to send at least one WWW-Authenticate challenge.

RFC 6750 adds bearer-token semantics: an invalid or expired bearer token normally maps to 401 with invalid_token, while insufficient scope normally maps to 403 with insufficient_scope.

Example assertion:

    pm.test("expired token is rejected", () => {
      pm.response.to.have.status(401);
      pm.expect(
        pm.response.headers.has("WWW-Authenticate")
      ).to.be.true;
    });
    

Do not stop at the status assertion. Also verify that the protected resource did not appear in the body.

For authentication endpoints such as login or password recovery, negative testing should additionally look for account-enumeration differences. OWASP recommends generic authentication responses so different messages, HTTP responses, or timing behavior do not reveal unnecessarily whether a particular account exists.

Authorization tests answer a specific question: the API knows who I am, but am I allowed to do this exact thing to this exact resource?

Horizontal Authorization

A caller uses another user’s or tenant’s resource identifier.

    GET /v1/orders/order-belongs-to-tenant-b
    Authorization: Bearer <tenant-A-token>
    

The operation must not expose Tenant B’s resource. OWASP identifies manipulation of user-controlled object identifiers as a common route to broken object-level authorization and recommends validating permission for each requested object.

Vertical Authorization

A lower-privileged account invokes an administrative operation:

    DELETE /v1/admin/users/123
    Authorization: Bearer <standard-user-token>
    

The API should deny the operation without performing it.

Function-Level Authorization

Test every relevant HTTP method, not merely the URL. A user permitted to GET /v1/projects/123 must not automatically be permitted to DELETE /v1/projects/123. OWASP’s broken function-level authorization category specifically covers situations where callers gain access to functions they should not be able to execute.

Property-Level Authorization

Try adding sensitive properties the normal client never sends:

    {
      "displayName": "Alice",
      "role": "super_admin",
      "accountBlocked": false
    }
    

OWASP warns that improperly protected object properties can result in unauthorized disclosure or manipulation and recommends explicitly controlling which properties may be read or changed.

Tenant Isolation

Cross-tenant testing should cover direct identifiers, searches, exports, nested resources, bulk operations, and indirect references. A useful authorization matrix is Actor x Resource x Action x Context.

Run this matrix automatically as authorization policies evolve.

Should an Unauthorized API Return 401, 403, or 404?

The correct answer depends on what failed.

RFC 9110 defines 403 as a request understood by the server but refused. It also explicitly permits an origin server to return 404 instead when it wants to hide the existence of a forbidden resource.

This means a wrong-tenant object test is not automatically defective because it returns 404. The important assertion is that the behavior matches the documented security policy and does not reveal whether another tenant owns the identifier through body contents, response metadata, or inconsistent behavior.

Boundary testing checks values immediately surrounding every allowed limit. If a field permits integers from 1 through 100, test:

• 0, reject
• 1, accept
• 2, accept
• 99, accept
• 100, accept
• 101, reject

Numeric Boundaries

Test minimum minus one, exact minimum, exact maximum, maximum plus one, negative values, zero, fractions where integers are required, extremely large positive and negative values, and numeric strings such as “100” if strict typing is expected. JSON Schema differentiates inclusive minimum/maximum from exclusiveMinimum/exclusiveMaximum, making the contract’s exact boundary semantics important.

String Boundaries

For maxLength: 32, test 31, 32, and 33 characters. Also test empty strings, whitespace-only strings, Unicode characters, normalization-sensitive values, invalid patterns, unexpected control characters, and extremely long values.

Collection Boundaries

For arrays, test 0 items, minimum minus one, minimum, maximum, maximum plus one, very large arrays, and duplicate elements when uniqueness is required.

Type Boundaries

Do not assume coercion is harmless. If the schema says quantity: 10, also test:

    {"quantity": "10"}
    {"quantity": true}
    {"quantity": null}
    {"quantity": []}
    {"quantity": {}}
    

Whether “10” is accepted or rejected should be a deliberate contract decision rather than an accidental framework conversion.

Missing and Unexpected Fields

Test when required properties are missing, and also inject undocumented properties such as an isAdmin flag. Unknown-field behavior should be explicit. Rejecting additional properties is often useful for security-sensitive write models. APIs that intentionally tolerate unknown fields should test that those values cannot alter internal state.

Request-Size Boundaries

OWASP recommends imposing request-size limits and rejecting payloads that exceed them. RFC 9110 defines 413 Content Too Large for requests the server refuses because the request content exceeds what it is willing or able to process. Similarly, an unsupported request media type is appropriately represented by 415 Unsupported Media Type.

Should Validation Errors Return 400 or 422?

Both can be valid depending on the API’s contract and the nature of the failure.

• 400 Bad Request is commonly used for a request that cannot be processed as a valid request.
• 422 Unprocessable Content has a more specific semantic meaning: the server understands the media type and the request syntax is correct, but it cannot process the contained instructions.

This mapping is not a universal mandate for every API. What matters for automated API negative testing is that the chosen behavior is deliberately documented and consistently implemented.

Rate-limit testing requires more than sending requests until one fails. First identify the policy, for example 60 requests per minute per API client. Then test the threshold:

• Request 59, allowed
• Request 60, allowed
• Request 61, rejected

This assumes that is the documented inclusive policy. RFC 6585 defines 429 Too Many Requests for clients that have sent too many requests in a given amount of time. A 429 response may include Retry-After to indicate how long the client should wait.

Identity Dimension

Determine whether quotas apply by user, OAuth client, API key, tenant, source IP, endpoint, account plan, or a combination of dimensions. Then prove isolation between them. If Client A exhausts its quota, Client B should not be throttled unless they intentionally share the same quota.

Window Boundaries

Test immediately before and after the reset. For fixed windows, pay particular attention to the transition between windows. For sliding-window or token-bucket algorithms, test the documented replenishment behavior instead of assuming a fixed reset.

Burst Behavior

An API might permit 1,000 requests per hour but only 20 requests per second. Test both sustained and burst limits when both exist.

Concurrency

A limiter that behaves correctly for sequential traffic may fail when 50 requests arrive concurrently.

Side Effects

This is one of the highest-value checks in API negative testing:

    Request 61 -> 429
    Database writes caused by request 61 -> 0
    Emails caused by request 61 -> 0
    Payment-provider calls caused by request 61 -> 0
    

Rate limiting that executes the expensive business operation and only later converts the response to 429 does not provide the intended resource protection. OWASP’s API resource-consumption guidance recommends limiting interaction frequency and setting limits on input sizes and other resource-consuming operations. Codoid’s API monitoring guide covers related verification patterns for production systems.

An error response is an API contract just as much as a successful response. Clients often depend on HTTP status, Content-Type, authentication or retry headers, a stable machine-readable error code, validation-field locations, human-readable detail, and a correlation or trace identifier.

RFC 9457 defines Problem Details for HTTP APIs and the application/problem+json media type. Its standard members include type, status, title, detail, and instance. APIs can define additional members for domain-specific information.

Example:

    HTTP/1.1 422 Unprocessable Content
    Content-Type: application/problem+json

    {
      "type": "https://api.example.com/problems/validation-error",
      "title": "Request validation failed",
      "status": 422,
      "code": "VALIDATION_ERROR",
      "detail": "One or more request fields are invalid.",
      "traceId": "01K5EXAMPLE",
      "errors": [
        {
          "field": "quantity",
          "code": "OUT_OF_RANGE"
        }
      ]
    }
    

In this example, code, traceId, and errors are application-specific extensions rather than mandatory RFC 9457 fields.

Negative tests should assert that:

• HTTP status equals body.status
• Content-Type equals the expected problem media type
• code is a documented machine-readable value
• Required members exist
• Member types match the schema
• Field errors identify the correct field
• Trace identifier has the expected format
• No unknown sensitive fields appear

RFC 9457 specifically warns that problem details should not expose implementation internals or sensitive information. OWASP similarly recommends avoiding stack traces and unnecessary technical information in client-facing errors.

Test for accidental exposure of:

• Stack traces
• SQL fragments
• Filesystem paths
• Internal hostnames
• Library versions
• Access tokens
• API keys
• Passwords
• Connection strings
• Personal information unrelated to the error

Consider POST /v1/orders with these rules:

• Authentication: OAuth bearer token
• Required scope: orders:write
• Tenant: derived from authenticated identity
• quantity: integer from 1 to 100
• sku: 1 to 32 characters
• lineItems: 1 to 50 entries
• Content-Type: application/json
• Rate limit: 60 requests per minute per client
• Error format: application/problem+json

A useful negative suite for this endpoint in API negative testing could contain:

The final assertion is what turns this from simple response testing into robust API negative testing. For example:

    response = client.post(
        "/v1/orders",
        headers={"Authorization": token_without_scope},
        json={"sku": "SKU-123", "quantity": 10},
    )

    assert response.status_code == 403

    body = response.json()
    assert body["status"] == 403
    assert body["code"] == "INSUFFICIENT_SCOPE"

    # Critical side-effect assertion
    assert orders.count_for_test_id(test_id) == 0
    

Authentication vs Authorization vs Validation vs Rate Limiting

These controls reinforce one another, but passing one never proves another. A request can be authenticated yet unauthorized. It can be authorized yet contain invalid data. It can be valid and authorized but over quota. And every one of those failures can still return a malformed or insecure error response.

Assert Exact Outcomes Instead of Any 4xx

A negative case should encode the rule being tested. Prefer specific assertions such as status code 403 with code INSUFFICIENT_SCOPE over a broad range check. Broad assertions allow regressions such as an expected 403 becoming 404, 422, or 429 without anyone noticing.

Verify State After Every Rejected Write

For POST, PUT, PATCH, and DELETE, validate persistence and downstream effects. A rejection is successful only when prohibited work did not occur. This is the core discipline of mature API negative testing.

Generate Boundary Cases Systematically

For every minimum and maximum, generate values around the transition: min minus one, min, min plus one, max minus one, max, max plus one. The same principle applies to dates, page sizes, file sizes, list cardinality, quotas, timeouts, and monetary limits.

Keep Authorization Rules in a Machine-Readable Matrix

An actor-resource-action matrix reduces gaps when endpoints or roles change. OWASP recommends automated authorization evaluation because authorization defects frequently appear as applications evolve.

Validate Responses Against the API Specification

OpenAPI can define successful and error responses, response headers, and body schemas. Treat contract mismatch as a test failure rather than accepting undocumented behavior. For a broader set of validation checks, refer to the REST API testing checklist.

Separate Protocol Assertions from Business Assertions

For a forbidden order cancellation, structure assertions into four groups:

• Protocol: status equals 403, media type equals application/problem+json
• Contract: error.code equals CANCELLATION_FORBIDDEN
• Security: response contains no forbidden order details
• State: order remains unchanged, no refund was initiated

This structure makes failures easier to diagnose.

Keep Negative Tests Deterministic

Use isolated identities, known resource ownership, controlled clocks where appropriate, and resettable rate-limit state. A test that depends on whatever quota happens to remain in a shared environment will eventually become flaky.

Why does an invalid request return 500 instead of 4xx?

The invalid condition may be reaching business code or a parser that throws an unhandled exception. Reproduce the smallest failing payload, inspect correlated server logs, and identify the layer that should have rejected it. A malformed client request should not normally force consumers to interpret an internal server failure.

Why do rate-limit tests pass locally but fail in staging?

The environments may use different limiters, distributed state stores, gateway policies, clock behavior, or shared quotas. Verify the limit value, identity key, window algorithm, burst configuration, shared versus isolated quota, number of gateway instances, and whether previous tests consume the same quota. Use dedicated test identities where possible.

Why do application validation errors match the schema but gateway errors do not?

Your API may have multiple error producers. Authentication, request-size enforcement, rate limiting, proxies, gateways, application middleware, and business code can each generate responses. Inventory every error-producing layer and apply the same external contract where feasible, or explicitly document different contracts when standardization is impossible.

Why does a rejected request still mutate state?

This is the most serious failure mode in API negative testing. Check whether the rejection happens before or after the business transaction commits. Check whether the operation runs in a queued worker that receives the request before validation. Check whether a downstream call fires before the response is constructed. Add explicit side-effect assertions to catch this class of defect.

No single tool covers every dimension of API negative testing.

OpenAPI and Schema-Driven Testing

Schemathesis can generate property-based test cases from OpenAPI and GraphQL schemas, including boundary values, type violations, and response-schema checks. It supports current OpenAPI versions as well as earlier ones. This is useful for broad schema-derived coverage, but business authorization and domain-specific invariants still require explicit assertions.

Request Collections and Workflow Tests

Postman supports post-response JavaScript assertions for status codes, response data, headers, and workflow behavior, and its documentation specifically includes invalid or incomplete input as an API-testing use case. It works well when teams already maintain request collections and want readable regression scenarios.

Code-Based API Suites

REST Assured remains an option for Java-based HTTP API testing, while Python teams can implement equivalent suites with their existing HTTP test clients and test frameworks. Code-based suites are especially useful when negative tests must inspect databases, queues, emitted events, or other state after the HTTP request. Codoid’s guide on API test automation using REST Assured covers this approach in detail.

Load-Capable Tooling

For rate-limit and concurrency tests, use tooling capable of controlled parallel request generation rather than a simple sequential loop. Codoid’s API monitoring guide covers related verification patterns.

Whatever tool is selected, record enough evidence to reproduce: test identity, request, response, timestamp, correlation ID, rate-limit state where observable, and post-request state.

Negative automation is powerful, but it does not replace a dedicated security assessment.

Schema-generated cases cannot automatically understand every business rule. A tool can infer that quantity has maximum: 100. It cannot necessarily infer that a support agent may read an invoice but may not approve a refund above a specific amount.

Rate-limit and authentication tests can also have real side effects. Poorly isolated tests may lock test accounts, trigger fraud defenses, consume SMS or email quotas, call paid providers, or throttle other teams sharing an environment.

Other limitations of API negative testing include:

• Eventual consistency can make post-condition assertions timing-sensitive.
• Long-lived JWT claims can complicate permission-revocation tests.
• Gateway and application configurations may differ between environments.
• A documented schema can itself be wrong.
• Automated negative tests cannot prove the absence of all exploitable authorization paths.
• A correct 4xx response alone does not prove the server avoided prohibited work.

Use negative automation as continuous regression protection, then complement it with architecture review, threat modeling, observability, and security testing. For a broader foundation, see the REST API testing checklist.

Effective API negative testing is not about proving that an API can return errors. It is about proving that the API fails correctly. For each protected operation, test invalid authentication, forbidden actions, cross-user and cross-tenant access, exact input boundaries, resource limits, rate-limit transitions, and the complete machine-readable error contract. Then add the assertion most teams overlook: verify that rejected operations did not change state or trigger prohibited downstream work.

A practical next step is to take one high-risk API endpoint and build a negative-test matrix containing authentication state, authorization state, input boundary, rate-limit state, expected error contract, and expected side effects. Automate that matrix in CI, expand it endpoint by endpoint, and treat every documented failure response as part of the API’s public contract. Codoid’s QA automation services help teams build this negative-test discipline into their delivery pipelines.

Need Help Building Your API Negative Testing Strategy?

Talk to an API Testing Expert

Frequently Asked Questions