ASYNC ARCHITECTURE • CLUSTER A

Async AI Agent Architecture with Redis Queues

Synchronous agents lose work the moment a call times out. We break down the async AI agent architecture: a Redis-backed queue (BullMQ), durable execution…

By Alex, Principal AI Infrastructure Architect | Updated September 2026 | 11 min read

A production AI agent is not a function call — it is a stateful, long-running job that spans model calls, tool invocations, and retries across seconds to minutes. The moment you run more than one at a time, you need three things: a durable queue (Redis + BullMQ) so a crash does not drop in-flight work, a durable state store so execution progress survives restarts, and idempotent workers so a redelivered job does not double-spend money or send two emails. Synchronous "await the agent" code is fine for a demo and a liability in production. Build async from day one if you expect concurrency.

TL;DR
  • Redis + BullMQ is the de-facto Node queue for agents: job persistence, retries with backoff, delayed and repeatable jobs, and visibility into stuck work.
  • Durable state is separate from the queue. Keep execution progress and conversation memory in Redis for hot access and Postgres/S3 for the durable copy.
  • Exactly-once execution does not exist in distributed systems. Design for at-least-once delivery plus idempotent handlers (jobId dedup).
  • Size workers with Little's Law: workers = ceil(in_flight / concurrency_per_worker). Use the calculator below.
  • A worker that holds a tool's side effect (send email, charge card) must be idempotent or a redelivery costs you a customer.
Table of Contents

Why synchronous agents break at scale

The first agent most teams write is a single await runAgent(input). It works in a test with one request. It breaks the moment reality shows up:

None of these are edge cases; they are Tuesday. The fix is not a better model — it is an architecture that treats each agent run as a job that can be queued, retried, and resumed.

Our multi-agent outbound pipeline writeup runs on exactly this pattern, and vendor fallback only helps if the job that called the model is still alive to try the next vendor.

The async agent anatomy

An async agent run flows through five stages, and each stage fails differently if you skip it:

Stage Responsibility Breaks if missing
Trigger Accept the request, enqueue a job Requests lost on restart
Queue Hold jobs, order retries In-flight work dropped on crash
Worker Execute the agent loop No concurrency, one thread
State store Persist progress + memory No resume, context lost
Side-effect guard Idempotent external writes Double charges, duplicate emails

The queue and the state store are different systems with different jobs. The queue decides what runs when; the state store remembers where a run was. Confusing them — stuffing everything in Redis and hoping — is the most common design error.

Queue: Redis + BullMQ

Redis is the right queue backbone for agents because it is fast, persists to disk (AOF/RDB), and BullMQ sits on top with the features you actually need:

A minimal worker looks like this. The handler must be idempotent (see below) because BullMQ delivers at-least-once.

import { Queue, Worker } from 'bullmq';
import IORedis from 'ioredis';

const connection = new IORedis(process.env.REDIS_URL, { maxRetriesPerRequest: null });
const agentQueue = new Queue('agent-runs', { connection });

new Worker('agent-runs', async (job) => {
  // job.id is stable across retries — use it as the idempotency key
  return runAgent(job.data, { runId: job.id });
}, {
  connection,
  attempts: 3,
  backoff: { type: 'exponential', delay: 2000 },
  concurrency: 5,
});

concurrency: 5 means one worker process handles 5 jobs in flight. That number drives your sizing math later.

Durable state: what must survive a crash

Not everything belongs in the queue. Separate hot state (needed mid-run) from durable state (needed after restart or for audit):

The rule: if losing it means a user's run cannot resume or be explained later, it goes to the durable store. Redis is for speed, not for truth. Many teams learn this the hard way when a Redis restart erases "finished" runs that were only in memory.

For self-hosted orchestrators, n8n's queue mode uses the same Redis + Postgres split — the compose guide shows the wiring, and the Postgres vs SQLite benchmark explains why Postgres wins under concurrency.

Idempotency: don't double-execute

Distributed systems give you at-least-once delivery, never exactly-once. A job can be delivered twice (ack lost, worker died mid-commit). Your handler must make a second run harmless:

Idempotency key

Derive a stable key from the job (e.g. job.id or a content hash) and record "already done" in Redis/Postgres before the side effect. A repeat sees the marker and returns the cached result.

Side effects last

Do the deterministic work first, the external write (email, charge, post) last, and only after the idempotency check. Ordering is what stops the double-send.

Outbox over direct call

Write "send email" to a durable outbox and let a separate process drain it. A crash between "decided to send" and "sent" no longer loses the intent or sends twice.

Idempotent downstream

If you call Stripe or an ESP, use their idempotency headers. Your retry and theirs must agree, or you move the problem downstream.

"Exactly-once" is a lie sold by marketing copy. At-least-once plus idempotent is the honest, working design.

Failure modes in async agent systems

Stuck jobs

A worker dies holding a job with no timeout; BullMQ never requeues it. Set lockDuration and a stall checker, or jobs hang forever.

Poison messages

One malformed input retries 3× and goes to the failed queue — fine. But if your retry has no cap, it hammers the model and burns money. Cap attempts and route failures to a dead-letter.

Queue backlog

Throughput drops, jobs pile up, latency climbs. Little's Law tells you how many workers you actually need — under-provisioning is the silent SLA killer.

Orphaned side effects

A job runs, writes to a DB, then crashes before acking. On redelivery it writes again. Without idempotency the second write is a duplicate row or a double charge.

Sizing the queue (Little's Law)

Little's Law says the number of in-flight jobs L = λ × W where λ is arrival rate (jobs/sec) and W is average handling time (sec). If each worker holds C jobs in flight, you need workers = ceil(L / C). The calculator does it live:

In-flight jobs (Little's Law): 40
Recommended workers: 8
Redis RAM for in-flight: 1.56 MB

Reference architecture (Docker Compose)

A minimal async-agent stack: Redis for the queue and hot state, Postgres for durable execution records, and one or more worker containers. Shared env uses an x- extension so it is never a bare top-level key.

x-agent-env: &agent-env
  REDIS_URL: redis://redis:6379
  DATABASE_URL: postgres://agent:${AGENT_DB_PASSWORD}@postgres:5432/agent
  IDENTITY_KEY_TTL: 86400
  MAX_RETRIES: 3

services:
  redis:
    image: redis:7-alpine
    command: ["redis-server", "--appendonly", "yes"]
    volumes:
      - redis-data:/data

  postgres:
    image: postgres:16
    environment:
      POSTGRES_USER: agent
      POSTGRES_PASSWORD: ${AGENT_DB_PASSWORD}
      POSTGRES_DB: agent
    volumes:
      - pg-data:/var/lib/postgresql/data

  agent-worker:
    image: your-registry/agent-worker:latest
    <<: *agent-env
    deploy:
      replicas: 4
    depends_on:
      - redis
      - postgres

volumes:
  redis-data:
  pg-data:

Four worker replicas at concurrency: 5 each handle 20 in-flight jobs — tune replicas from the calculator, not from a guess. Postgres holds durable runs; Redis holds the queue and hot memory. Pin image tags and keep secrets in env files, not the compose.

When to go async vs stay synchronous

The cost of async is operational: you now run Redis, watch a queue, and think about idempotency. The cost of not going async is lost work and angry customers, and it shows up exactly when traffic peaks.

FAQ

Deploy this stack in production

Every config, default, and failure mode in this guide comes from live deployment, not documentation. Our Make.com playbook covers the orchestration patterns end to end.

Get the Make.com Automation Playbook →

Download this guide’s assets

Get the configuration and data files referenced in this guide. Subscribe and we’ll send the bundle to your inbox.

Get the bundle →
Is Redis durable enough to trust as the queue?

Redis persists via RDB snapshots and AOF append logs, so a restart does not erase the queue. But "durable enough" depends on your tolerance: for the system of record (finished runs, audit), use Postgres/S3. Redis is the queue and hot cache; Postgres is the truth. Losing Redis should lose in-flight jobs, not history.

Why BullMQ instead of a raw Redis list?

A raw LPUSH/RPOP list has no retries, no delayed jobs, no visibility, and no stall detection. BullMQ adds all of that on top of Redis. You can build it by hand, but you will re-implement backoff, dead-lettering, and observability — BullMQ already did, and the bugs are someone else's solved problem.

Can I really never get exactly-once?

Not across the queue, worker, and external system boundary. You can get effectively-once by making every step idempotent and using an outbox, so a redelivery is harmless. That is the real goal; "exactly-once" as a global guarantee does not exist in practical distributed systems.

How many workers do I actually need?

Use the calculator above: workers = ceil(arrival_rate × handling_time / concurrency_per_worker). Start there, then watch the queue depth in production and add replicas when backlog grows. Under-provisioning shows up as latency, not errors, so instrument the queue.

Engineering transparency: The architecture described (Redis + BullMQ queue, separate durable state store, at-least-once delivery with idempotent handlers, Little's Law sizing) reflects standard distributed-systems practice for job queues and is not a benchmark we ran. BullMQ options shown (attempts, backoff, concurrency, lockDuration) are documented BullMQ behaviors; confirm the exact field names against the BullMQ version you deploy. The worker calculator applies Little's Law (L = λ × W) with simple arithmetic — the Redis RAM figure is a rough estimate from an assumed per-job size and should be validated against your real job payloads, not treated as a capacity guarantee. n8n queue-mode references point at its Redis + Postgres split as a concrete self-hosted example.

Related Cluster Intelligence