Checklist/Docs/Async code vs background jobs

Guide 01

Async code vs background jobs

Audience: FastAPI engineers choosing where work should run. Reading time: ~8 min Keywords: FastAPI async, BackgroundTasks, durable jobs, Celery, Taskiq

TL;DR

NeedPut the work here
Data for this responseasync def (or sync route if intentionally blocking)
Tiny best-effort after responseFastAPI BackgroundTasks
Must retry, scale, or survive deploysDurable job queue (RabbitMQ) or event bus (Kafka)

async only means “don’t block the event loop while waiting on I/O.” It does not mean durable, retriable, or independently scalable.

Contents

  1. Three layers people mix up
  2. Decision tree
  3. What belongs in the request
  4. BackgroundTasks rules
  5. Durable jobs
  6. Pseudocode patterns
  7. Performance pitfalls
  8. Checklist mapping

---

1. Three layers people mix up

LayerRuns whereSurvives crash/deploy?Scales how?Use when
async def in a requestSame request, event loopNoWith API replicasFast I/O needed for the response
FastAPI BackgroundTasksAfter response, same processNoStuck on that API podTiny, non-critical cleanup
Job queue + workerSeparate processYes (durable broker)Worker replicas / queuesMust retry, scale, or outlive deploys
Domain event + consumerSeparate processYesConsumer groups / lagMany services react to a fact

2. Decision tree

text
Must the HTTP response body include this work's result?
 YES → do it in the request path (async I/O), or
 enqueue + return 202 and let the client poll job status
 NO → Would users/business care if this work is lost on deploy?
 NO → BackgroundTasks is acceptable (keep it tiny)
 YES → Durable queue (command on RabbitMQ) or event (Kafka)

Examples

WorkPlacement
Load user + permissions for GETRequest path
Send “welcome” email after signupJob queue
Write analytics pixel after 200BackgroundTasks OK
Generate 50MB exportJob queue (jobs.heavy)
Charge card + ledger entryRequest for charge API or command job; never BackgroundTasks
Fan-out “order placed” to billing + emailDomain event (Kafka) after commit

3. What belongs in the request

Keep request handlers short and predictable:

  • Validate input (Pydantic)
  • Authorize
  • One transaction (or a clear saga start)
  • Return a response

If work is slow but the user must wait, still avoid holding the connection for minutes: prefer 202 + job_id and a status endpoint.

Async vs sync routes

python
# Prefer async for concurrent I/O (DB, HTTP clients)
@app.get("/users/{user_id}")
async def get_user(user_id: int, db: Db = Depends(get_db)):
 return await db.fetch_user(user_id)

# Sync is OK if the whole stack is sync and you understand the thread pool
@app.get("/report-sync")
def report_sync():
 return heavy_cpu_report() # better: offload to worker

Rule: never call blocking I/O inside async def without asyncio.to_thread / a proper executor. Prefer async drivers (asyncpg, httpx.AsyncClient).

4. BackgroundTasks rules

Allowed:

  • Delete a temp file
  • Best-effort metrics that you already accepted losing
  • Non-critical cache warm

Forbidden for production-critical work:

  • Emails users expect
  • Payment webhooks
  • PDF invoices
  • Anything with SLAs or legal weight
python
# PSEUDOCODE : OK: best-effort
@app.post("/events")
async def track(event: EventIn, bg: BackgroundTasks):
 bg.add_task(log_event_best_effort, event)
 return {"ok": True}

5. Durable jobs

When work must survive:

  1. Insert job row (pending) in your DB
  2. Publish command to RabbitMQ (or outbox → broker)
  3. Return 202 + job_id
  4. Worker runs, updates status, retries with bounds
  5. Client polls GET /jobs/{id} or receives a webhook

See 03 Job lifecycle and 04 Outbox.

6. Pseudocode patterns

Request-only async

python
# PSEUDOCODE
@app.get("/users/{user_id}")
async def get_user(user_id: int, db: Db = Depends(get_db)):
 user = await db.fetch_user(user_id)
 if not user:
 raise HTTPException(404)
 return user

Enqueue durable work (preferred)

python
# PSEUDOCODE
@app.post("/exports", status_code=202)
async def start_export(body: ExportIn, db: Db, bus: JobBus, user=Depends(auth)):
 async with db.transaction():
 job = await db.insert_job(
 type="export_csv",
 status="pending",
 entity_id=str(user.id),
 idempotency_key=body.idempotency_key,
 payload_ref=body.filters_ref,
 )
 await bus.publish(
 queue="jobs.heavy",
 message={"job_id": str(job.id), "type": "export_csv"},
 )
 return {"job_id": job.id, "status": "pending"}

Wrong: critical work only on BackgroundTasks

python
# PSEUDOCODE : DO NOT DO THIS for mail users expect
@app.post("/signup")
async def signup(body: SignupIn, bg: BackgroundTasks):
 user = await create_user(body)
 bg.add_task(send_welcome_email, user.id) # lost on deploy/crash
 return user

7. Performance pitfalls

PitfallSymptomFix
Sync HTTP inside async defLatency spikes, worker stallsAsync client or thread offload
Huge work in requestTimeouts, user abandons202 + job
Unbounded BackgroundTasksMemory growth under loadCap work; move to queue
CPU in async worker without process poolEvent loop lagSeparate heavy queue / prefork

8. Checklist mapping

  • Development: no blocking in async routes; BackgroundTasks limited
  • Jobs: durable broker for critical work; 202 + job_id
  • Anti-patterns: critical email only on BackgroundTasks

Next: 02 Brokers and frameworks