Why I Stopped Assuming Webhooks Only Fire Once

1. The Mistake I Made Early On

When I first started building backend APIs, I had a very simple mental model for webhooks:

1. A user does something on an external service (like paying through Stripe or filling out a form).
2. That service sends an HTTP POST request to my API endpoint.
3. My code receives it, saves it to the database, and returns a 200 OK.

In my local tests, this worked 100% of the time. One action always resulted in one webhook call.

Then we went into production, and I saw something strange: occasionally, a single payment resulted in two identical records in the database, created just 300 milliseconds apart.

2. Why Webhooks Send Duplicates

The internet is not a direct wire; packets get delayed or dropped all the time.

When an external platform sends a webhook to your server, it waits for a response (usually a 200 OK within a few seconds). If your server is slightly slow or there is a minor network hiccup, the sender thinks: "My message was lost. I must try again."

This is called at-least-once delivery. The sender prefers to send the same message twice rather than risk you missing it entirely. According to Stripe's webhook documentation, duplicate events are an expected part of building on the web.

If your endpoint is not prepared for duplicates, bad things happen:

• Users get charged twice.
• Two welcome emails get sent.
• Inventory counts get messed up.

3. The Solution: Making Endpoints Idempotent

The word sounds fancy, but the idea is very simple: an action is idempotent if doing it multiple times produces the exact same result as doing it once.

Every webhook event comes with a unique ID from the sender (for example: evt_12345). Before processing the event, you check if you have already handled that ID.

Here is a straightforward example in C#:

[HttpPost("api/webhooks")]
public async Task<IActionResult> HandleWebhook([FromBody] WebhookPayload payload)
{
    string eventId = payload.Id;

    // 1. Check if we already processed this event ID in our cache or database
    bool isAlreadyProcessed = await _cache.KeyExistsAsync($"webhook:{eventId}");
    if (isAlreadyProcessed)
    {
        // Return 200 OK so the sender stops retrying, but DO NOT run the logic again
        return Ok(new { message = "Event already handled." });
    }

    // 2. Mark this event ID as processed with a 24-hour expiration
    await _cache.StringSetAsync($"webhook:{eventId}", "processed", TimeSpan.FromHours(24));

    // 3. Safely run your business logic
    await _orderService.ProcessPaymentAsync(payload);

    return Ok();
}

4. What I Learned

This was one of the biggest realizations in my first year as a backend developer:

• Never assume networks are perfect. If a request can retry, it will retry.
• Always look for unique event IDs. Store them in a fast cache like Redis or a unique column in your SQL database.
• A 200 OK is a receipt. If you already handled the event, gracefully acknowledge it with 200 OK so the sender knows everything is fine and stops resending.