Documentation

Docs and examples

Reference material rendered inside the app, not raw markdown dumps. Search filters guides and snippets as you type.

Architecture diagrams

SVG diagrams follow light and dark theme tokens. Toggle the theme to preview both.

Production layout: API, workers, Beat, Flower, brokers
ClientsAPIFastAPI × NPostgresjobs / outboxRabbitMQcommandsKafkaevents (opt)Redislocks / RL onlyWorkersCelery × NBeat1 leaderFlowerops UI privatePrometheusscrape metricsSame image · different CMD
Celery pipeline: API RL → enqueue → RabbitMQ → job RL + lock → side effect
ClientFastAPIAPI rate limitEnqueuejobs row + 202RabbitMQjobs.ioWorkerjob RL + lockSide effectvendor / DBRedis RLPostgresstatus → jobs table
Locks & rate limits: Redis beside the worker path
APIper-user RLEnqueue202 + job_idRabbitMQCelery workerRL then lockRedistoken bucketRedisentity lock NXorder: take token → acquire lock → side effect → release
Transactional outbox: commit then publish safely
APIDB transactionjob + outbox rowOutbox relaySKIP LOCKEDRabbitMQ / Kafkapublishsame commit → no dual-write gapmark published_at
Observability: Flower, Prometheus, app metrics
Workers-E eventsFlower/metricsPrometheusscrape 15sGrafanaAlertmanagerRabbitMQRMQ exporterdepth · DLQAPIGET /jobsproduct status = DB · Flower = ops only

Guides

20/20

Pseudocode snippets

5/5

Quick patterns. Full context lives in the guides above.

Enqueue + 202

async def enqueue_export(body, db, bus):
 job = await db.insert_job(type="export", status="pending", ...)
 await bus.publish("jobs.heavy", {"job_id": str(job.id), "type": "export"})
 return {"job_id": job.id} # HTTP 202

Idempotent worker

async def handle(message, db):
 job = await db.get_job(message["job_id"])
 if job.status == "succeeded":
 return
 await db.mark_running(job.id)
 await do_work(job)
 await db.mark_succeeded(job.id)

Transactional outbox

async with db.transaction():
 job = await db.insert_job(...)
 await db.insert_outbox(
 destination="jobs.default",
 payload={"job_id": str(job.id), "type": job.type},
 )

Lock + job rate limit

@celery_app.task(bind=True, max_retries=25)
def sync_account(self, job_id: str, account_id: str):
 if not take_token(redis, f"rl:vendor:{account_id}", rate=5, burst=10):
 raise self.retry(countdown=2)
 token = new_uuid()
 if not acquire_lock(redis, f"lock:account:{account_id}", token, ttl=300):
 raise self.retry(countdown=10)
 try:
 do_sync(account_id)
 finally:
 release_lock(redis, f"lock:account:{account_id}", token)

Job metrics sketch

metrics.incr("jobs_enqueued_total", tags={"type": job.type})
metrics.observe("job_runtime_seconds", elapsed, tags={"type": job.type})
# backlog: rabbitmq_queue_depth gauge per queue
# alerts: consumers==0 critical; dlq_depth>0 high

Machine-readable export

Prefer these pages in the browser. Agents can still ingest the full dump:

Open llms-full.txt