Make background jobs and webhooks safe to retry

Help me make a background job, queue consumer, or webhook handler safe to retry. What it does: [e.g. "charges a card and sends a receipt", "syncs orders to the warehouse"] Code: [PASTE THE HANDLER] Trigger and delivery: [QUEUE OR WEBHOOK PROVIDER, AND WHETHER DELIVERY IS AT-LEAST-ONCE] Side effects: [DATABASE WRITES, EXTERNAL API CALLS, EMAILS, PAYMENTS] Current retry behavior: [WHAT HAPPENS ON FAILURE TODAY] What's gone wrong: [e.g. "duplicate emails", "jobs stuck", "silently dropped events" - or "nothing yet, preventing it"] Analyze: 1. What happens if the handler runs twice, runs concurrently with itself, or crashes halfway through each step. 2. Which side effects are naturally idempotent and which aren't. Then design: - Idempotency: an idempotency key (where it comes from, where it's stored, how long), and a pattern to check and record it atomically with the work. Use idempotency keys on external API calls where the provider supports them [CONFIRM provider support]. - Ordering: whether out-of-order delivery matters and how to handle it (version numbers, timestamps, re-fetching current state). - Retries: which errors are retryable vs. permanent, exponential backoff with jitter, a maximum number of attempts. - Dead-letter handling: where failed messages go, alerting, and how to safely replay them. - Webhooks specifically: signature verification, responding quickly and processing asynchronously, replay protection. Return the rewritten handler as code, a failure-scenario table (scenario, current behavior, new behavior), and tests to add.