Background Job Queue Design Guide

Act as a senior backend engineer. Design a background job queue architecture for [application/product] that needs to process [job types].

Application context: [SaaS app, marketplace, internal tool, ecommerce platform, AI workflow, etc.]
Job examples: [emails, webhooks, imports, exports, billing sync, media processing, notifications]
Expected volume: [jobs per minute/hour/day, burst patterns]
Latency requirements: [real-time, near-real-time, overnight batch, SLA]
Current stack: [language, framework, database, queue/broker, hosting]
Failure risks: [duplicate work, partial completion, external API limits, long-running jobs, poison messages]
Operational constraints: [team size, budget, compliance, observability tooling]

Deliver:
1. **Queue architecture recommendation** — broker, worker model, storage, and deployment topology
2. **Job taxonomy** — priority classes, payload shape, ownership, and timeout rules
3. **Retry and backoff policy** — transient vs permanent failures, max attempts, jitter, and dead-letter queue
4. **Idempotency design** — dedupe keys, state transitions, locks, and safe replays
5. **Scheduling model** — delayed jobs, recurring jobs, cron replacement, and timezone handling
6. **Scaling plan** — worker concurrency, autoscaling triggers, rate limits, and backpressure
7. **Observability** — metrics, logs, tracing, alerts, dashboards, and runbooks
8. **Testing strategy** — unit, integration, chaos, and replay tests
9. **Migration plan** — how to introduce the queue without breaking existing synchronous flows

Include concrete implementation notes for [preferred framework or queue tool].