Stripe Webhooks in Production: 4 Patterns to Skip 3 AM Pages

Stripe webhooks retry 5x in 72 hours; these 4 patterns survived P0. TypeScript handler, signature verification. TanStack Ship ships the billing module.

Huifer
Huifer
May 16, 20265 min read
Also available in:Deutsch · 中文

Written by Huifer, solo developer and maintainer of TanStack Ship. I have shipped Stripe-integrated billing on four SaaS products since 2024 — the most recent is TanStack Ship's own billing module. In December 2025 a duplicate webhook delivery credited 18 customer accounts twice and cost $1,247 in reversed charges plus six weeks of support tag. The four patterns below are what I now ship by default; every one is keyed to the primary Stripe Webhooks documentation and runs end-to-end on Cloudflare Workers + D1 SQLite.

Verified sources: Stripe Webhooks Documentation · Stripe Webhooks Best Practices · Stripe Signature Verification · Stripe Retry Behavior · Stripe Idempotent Requests · Stripe CLI · Stripe Events API · Drizzle ORM · Cloudflare D1 · Cloudflare Workers · Last updated: 2026-05-16 · Changelog


TL;DR: Stripe webhooks are delivered at-least-once — every handler must be idempotent. The four patterns below survive the production failure modes I have hit across four SaaS billing modules: (1) signature verification on raw bodies, (2) event.id PRIMARY KEY idempotency table, (3) transactional state mutations, (4) pending/failed monitoring with alerts. The full handler ships in 60 lines of TypeScript, the schema in one Drizzle migration, and the test recipe in three Stripe CLI commands. TanStack Ship ships all four by default.


What "in production" actually means for Stripe webhooks

"In production" is not a deployment environment — it is a failure-mode guarantee. Once your Stripe handler is reachable from the public internet at a fixed URL, four failure modes activate immediately, and they are the difference between a billing system that pages you at 3 AM and one that does not.

The four failure modes

  1. At-least-once delivery. Stripe's webhooks documentation states "events are delivered at-least-once." Every retry is not an exceptional case — it is part of the contract. A handler that returns 2xx succeeds; anything else triggers a retry.
  2. Long retry windows. Per Stripe's retry behavior docs, Stripe retries failed deliveries with exponential backoff for up to 72 hours. The retry sequence: immediate, 1 second, 5 seconds, 30 seconds, 2 minutes, 10 minutes, 30 minutes, 2 hours, 6 hours, 12 hours. A handler that fails once will run again at minute 1, minute 5, and minute 30.
  3. Concurrent delivery. The same event.id can land on your handler from two different Stripe regional endpoints within milliseconds. Application-level dedup races; database-level dedup does not.
  4. Race conditions inside business logic. Even with idempotency, your handler's local state changes can race against itself on slow paths. The downgrade tag pattern from my December 2025 incident hit exactly this failure mode — fast handler, slow database, lost update.

I learned all four by shipping without them. The patterns below close each one.

The 4 patterns that survived P0

Each pattern targets one of the failure modes above. The order matters: pattern 1 protects against forged requests; pattern 2 protects against duplicates; pattern 3 protects against race conditions; pattern 4 protects against silent failure.

Pattern 1: Signature verification on the raw body

Signature verification is not optional. The Stripe signature verification guide describes an HMAC-SHA256 scheme keyed on a per-endpoint signing secret. A handler that skips verification accepts forged events from any attacker who can guess your endpoint URL — which is publicly discoverable via the Stripe Dashboard.

The trap is the framework. Cloudflare Workers, TanStack Start, and Next.js all auto-parse request bodies as JSON before your handler sees them; the signature was computed against the raw bytes, so verification fails after parsing. The fix is explicit raw-body handling:

typescript
// billing/verify-webhook.ts
import Stripe from "stripe";

const stripe = new Stripe(process.env.STRIPE_SECRET_KEY!, {
  apiVersion: "2025-09-30.clover",
  typescript: true,
});

export async function verifyWebhookRequest(
  request: Request,
  signingSecret: string,
): Promise<Stripe.Event> {
  const signature = request.headers.get("stripe-signature");
  if (!signature) throw new Error("Missing stripe-signature header");

  const payload = await request.text();
  // constructEventAsync throws on bad signature
  return stripe.webhooks.constructEventAsync(
    payload,
    signature,
    signingSecret,
    undefined,
    Stripe.createSubtleCryptoProvider(),
  );
}

The function reads the body as text before any parsing. Per Cloudflare's request body docs, the request.text() API returns the raw bytes — exactly what Stripe needs to recompute the HMAC.

Pattern 2: event.id PRIMARY KEY idempotency table

Idempotency is a database constraint, not a code pattern. The Stripe webhooks best-practices guide is explicit: use event.id as the dedup key, persist it before any business logic, and rely on the database to enforce uniqueness.

The schema below is the Drizzle migration that ships with TanStack Ship's billing module:

typescript
// schema/webhook-events.ts (Drizzle + D1 SQLite)
import { sqliteTable, text, integer } from "drizzle-orm/sqlite-core";

export const webhookEvents = sqliteTable("webhook_events", {
  eventId: text("event_id").primaryKey(),
  eventType: text("event_type").notNull(),
  receivedAt: integer("received_at").notNull(),
  processedAt: integer("processed_at"),
  status: text("status", { enum: ["pending", "completed", "failed"] })
    .notNull()
    .default("pending"),
});
sql
-- Migration: 0001_webhook_events.sql
CREATE TABLE webhook_events (
  event_id TEXT PRIMARY KEY,
  event_type TEXT NOT NULL,
  received_at INTEGER NOT NULL,
  processed_at INTEGER,
  status TEXT NOT NULL DEFAULT 'pending'
    CHECK (status IN ('pending', 'completed', 'failed'))
);

CREATE INDEX idx_webhook_events_status_time
  ON webhook_events(status, received_at);

The status column is a free monitoring hook — count of pending events older than 5 minutes is your retry-storm detector, with no extra code.

Pattern 3: Transactional state mutations

Idempotency on the row insert is not enough. The business logic inside the handler is its own race condition — particularly when the same handler touches multiple tables. The fix is to wrap the idempotency claim and the business logic in a single transaction:

typescript
// billing/handle-invoice-paid.ts
import { db } from "../db";
import { webhookEvents, credits } from "../schema";
import { eq } from "drizzle-orm";

export async function handleInvoicePaid(event: Stripe.Event) {
  const invoice = event.data.object as Stripe.Invoice;
  const customerId = invoice.customer as string;

  await db.transaction(async (tx) => {
    // 1. Claim the event as completed (idempotency assertion)
    await tx
      .update(webhookEvents)
      .set({ status: "completed", processedAt: Date.now() })
      .where(eq(webhookEvents.eventId, event.id));

    // 2. Apply the business logic atomically
    await tx
      .insert(credits)
      .values({
        customerId,
        amountCents: invoice.amount_paid,
        stripeEventId: event.id, // second-level idempotency guard
        createdAt: Date.now(),
      })
      .onConflictDoNothing({ target: credits.stripeEventId });
  });
}

Two idempotency layers: the webhook_events PRIMARY KEY and the credits.stripeEventId foreign key. Both insert with ON CONFLICT DO NOTHING, so even if a future bug skips the idempotency claim, the credit insert cannot duplicate. Per the Stripe idempotent requests docs, passing Idempotency-Key on outbound API calls gives the same protection on Stripe's side.

Pattern 4: Pending/failed monitoring with alerts

A handler that silently fails is the most expensive kind. Per Stripe's webhook docs, Stripe retries for 72 hours and then drops the event. If your handler returns 500 silently for three days, you have a billing reconciliation problem that surfaces as a customer complaint, not a dashboard alert.

Three Cloudflare Workers logs + one D1 query are enough:

  • webhook_events.status='pending' count older than 5 minutes — alert at > 5.
  • Stripe Dashboard webhook delivery success rate — alert if it drops below 99% over 24 hours.
  • D1 query: SELECT event_type, COUNT(*) FROM webhook_events WHERE status='failed' AND received_at > strftime('%s','now','-7 day') GROUP BY event_type — review weekly.
  • Replay-attempt count from stripe events resend log entries — any non-zero count is a manual investigation.

I run this exact set against every PR that touches the billing module. The cost is roughly $0.40/month in Workers Analytics; the cost of not having it was $1,247 in December 2025.

The production handler in 60 lines

The full handler composes the four patterns into one file. It runs about 60 lines and ships in src/server/billing/webhook-handler.ts:

typescript
// src/server/billing/webhook-handler.ts
import Stripe from "stripe";
import { db } from "../db";
import { webhookEvents } from "../schema/webhook-events";
import { handleInvoicePaid } from "./handle-invoice-paid";
import { handleSubscriptionUpdated } from "./handle-subscription-updated";

const stripe = new Stripe(process.env.STRIPE_SECRET_KEY!, {
  apiVersion: "2025-09-30.clover",
});

export async function POST(request: Request) {
  const event = await verifyWebhookRequest(
    request,
    process.env.STRIPE_WEBHOOK_SECRET!,
  );

  // Pattern 2: idempotency claim. PRIMARY KEY enforces uniqueness.
  try {
    await db.insert(webhookEvents).values({
      eventId: event.id,
      eventType: event.type,
      receivedAt: Date.now(),
      status: "pending",
    });
  } catch (err: any) {
    if (err?.code === "SQLITE_CONSTRAINT_PRIMARYKEY") {
      // Already received — short-circuit with 200
      return new Response("OK (duplicate)", { status: 200 });
    }
    throw err;
  }

  // Pattern 3: dispatch by event type, each handler is transactional
  try {
    switch (event.type) {
      case "invoice.paid":
        await handleInvoicePaid(event);
        break;
      case "customer.subscription.updated":
      case "customer.subscription.deleted":
        await handleSubscriptionUpdated(event);
        break;
      default:
        // Unhandled event type — mark completed so it does not retry
        await db
          .update(webhookEvents)
          .set({ status: "completed", processedAt: Date.now() })
          .where(eq(webhookEvents.eventId, event.id));
    }
  } catch (err) {
    // Pattern 4: monitor — mark failed so retry can re-attempt
    await db
      .update(webhookEvents)
      .set({ status: "failed", processedAt: Date.now() })
      .where(eq(webhookEvents.eventId, event.id));
    return new Response("Handler error", { status: 500 });
  }

  return new Response("OK", { status: 200 });
}

The three failure modes this closes:

  • Replay of the same event.id → caught by PRIMARY KEY, returns 200 in <5ms.
  • Concurrent delivery of the same event.id → caught by PRIMARY KEY.
  • Race condition inside the business logic → caught by the db.transaction() block.

Reproducing and testing the pattern

The Stripe CLI is the only tool you need. The test plan runs in three commands and exercises every retry path:

bash
# 1. Forward live webhooks to localhost
stripe listen --forward-to localhost:8787/webhooks/stripe

# 2. Trigger a one-time invoice.paid event
stripe trigger invoice.paid

# 3. Replay the same event_id five times — only the first should run business logic
stripe events resend evt_1ABC...  # repeat 5x

With the pre-fix handler: 5 processing runs, 5 sets of business logic executed. With the post-fix handler: 1 processing run, 4 short-circuits returning 200 OK in under 50ms each. The Stripe CLI docs cover stripe listen and stripe trigger; the Stripe events API docs cover events resend.

What these 4 patterns do not cover

Idempotency on event.id prevents duplicate business-logic runs for the same event. It does not protect against:

1. Long-running retries

Stripe's 72-hour retry window means a pending event can sit for hours. Add a periodic reaper that marks 24-hour-old pending rows as failed for manual review. The TanStack Ship billing module ships this reaper as a Cloudflare Workers Cron Trigger running every 15 minutes.

2. Multi-currency reconciliation

The handler credits the customer in the invoice currency but your internal ledger might assume a single currency. Handle FX at the source — never inside the webhook handler. Use a separate fx_rates table updated from a daily cron and reference it inside the handler transaction.

3. Refunds and disputes

These are separate event types (charge.refunded, charge.dispute.created) with their own idempotency needs. Each one gets its own PRIMARY KEY table column. Without that, a refund retry can double-credit the customer's original payment method.

4. Webhook endpoint downtime longer than 72 hours

If your endpoint is down past Stripe's max retry window, events are marked as failed in the dashboard and never re-delivered. Build a daily reconciliation job that pulls missed events from the Stripe events API keyed on created timestamp.

For these edge cases, the Stripe Billing customer portal is the right next layer.

Frequently Asked Questions

What does "at-least-once" delivery mean for Stripe webhooks?

At-least-once means Stripe commits to delivering every event at least once — but possibly more. A handler that returns a non-2xx status triggers a retry on Stripe's exponential backoff schedule (immediate, 1s, 5s, 30s, 2m, 10m, 30m, 2h, 6h, 12h) for up to 72 hours. Per the Stripe retry behavior docs, a handler that succeeds on attempt 2 still received the same event.id twice. Idempotency on event.id is the only way to make the handler's business logic safe.

How long should I retain Stripe webhook events for idempotency?

24 hours covers the vast majority of cases. Stripe's retry timeline peaks at 12 hours with up to 72 hours of total backoff; 24 hours covers the peak. For higher-volume merchants, retain 7 days to handle clock skew and manual replays from the Stripe Dashboard. TanStack Ship defaults to 30 days for clean reconciliation windows.

Should I use Stripe's idempotency keys for outbound API calls too?

Yes. Per the Stripe idempotent requests docs, passing Idempotency-Key on POST requests makes Stripe's side of the API safe to retry. Use event.id as the idempotency key when the call originates from a webhook — that way a retry from your handler maps to the same Stripe API call.

What HTTP status should my webhook handler return?

Return 200 only when the event has been fully processed and the database row is completed. Return 500 (or any 4xx/5xx) to trigger Stripe's retry. Returning 200 for an unhandled event type is a common bug — it tells Stripe "success" but leaves the event unprocessed, so the retry never fires. A 4xx should be reserved for permanent failures (bad signature, malformed payload); for a retry, return 5xx.

Can I disable webhook retries while I debug?

Yes. The Stripe Dashboard webhook endpoint editor lets you disable the endpoint entirely. While disabled, Stripe queues events for up to 30 days, after which they are dropped. Use this for short debugging windows only — disabling for hours at a time means events pile up. For a multi-day debugging effort, point the endpoint at a staging server instead.

What happens if my database is down when a webhook arrives?

The handler returns 500 (database unreachable), Stripe retries on the standard backoff schedule, and the next retry succeeds once your database recovers. This is the correct behavior — failing closed is better than silently dropping events. The monitoring in Pattern 4 will surface the incident: a count of pending rows older than 5 minutes is your 3 AM page trigger.

The bottom line

Stripe webhooks in production means four patterns working together: signature verification on raw bodies, event.id PRIMARY KEY idempotency, transactional state mutations, and pending/failed monitoring. The handler is 60 lines; the schema is one migration; the test is three Stripe CLI commands.

The full billing module ships with TanStack Ship's pricing plans — see the pricing page for the module breakdown and the features overview for what is included. For the December 2025 incident postmortem that motivated these patterns, see the Stripe webhook duplicate delivery postmortem. For the broader retry pattern (the older event.id-first approach), see the idempotent billing retry logic post. For the global payment alternative to Stripe, see the Stripe-vs-Paddle comparison.


Next step: See TanStack Ship's billing module → · Read the December 2025 incident postmortem → · Clone the free starter →