You Don't Need a Message Queue

Every backend tutorial that mentions background jobs ends the same way: “and then you add a message queue.” Redis. RabbitMQ. SQS. Choose one, wire it up, and now you have another service to deploy, monitor, and debug.

I built a newsletter system without any of that. The emails go out. Failed deliveries retry automatically. It’s been running in production quietly, without me thinking about it. The entire thing is a database table and a background worker.

What a Message Queue Actually Does#

Strip away the marketing and a message queue does three things: stores a task somewhere durable, delivers it to a worker, and handles retries when the worker fails. That’s the whole job.

The complexity in RabbitMQ and Kafka comes from doing this across many services, at high throughput, with multiple consumers competing for work. That’s a real problem for distributed systems processing millions of events per minute.

Most backends are not that. Most backends need to send a welcome email, process an uploaded file, or retry a failed webhook. Your database can do all three. You already have one.

The Pattern#

Instead of pushing a task to a queue, write a row to a database table. A background worker periodically reads that table, processes what’s pending, and updates the row.

User action
    ↓
Write row to jobs table  →  HTTP response (immediate)
    ↓
Background worker wakes up on a timer
    ↓
Reads pending rows → processes → updates status

No broker. No separate worker process. No infrastructure to configure.

How It Looks in Practice#

When a newsletter goes out, each recipient gets a row in a jobs table with a status of pending. A worker started at server boot processes those rows and retries any that fail.

Here’s the entire worker setup in Go:

func StartWorker(interval time.Duration) {
    ticker := time.NewTicker(interval)
    go func() {
        for range ticker.C {
            if err := processJobs(); err != nil {
                log.Printf("worker error: %v", err)
            }
        }
    }()
}

A goroutine, a ticker, one function call. It starts when the server starts and runs forever.

processJobs reads the jobs table, finds rows with status = failed and attempts < 3, checks if enough time has passed since the last attempt, and retries them. The backoff is a plain switch:

func retryDelay(attempts int) time.Duration {
    switch attempts {
    case 0:
        return 1 * time.Minute
    case 1:
        return 5 * time.Minute
    case 2:
        return 15 * time.Minute
    default:
        return 0
    }
}

Exponential backoff. Max 3 retries. The logic fits in one screen. No dependency to install.

The same pattern works for one-off tasks. When a user triggers something slow, the handler fires a goroutine and returns immediately:

go func() {
    result, err := doSlowThing()
    // update status in DB when done
}()

w.WriteHeader(http.StatusAccepted)

The HTTP response is instant. The slow work happens in the background. The client polls a status endpoint. No queue needed.

What You Get for Free#

Using the database as the job store gives you things that message queues charge extra for.

Visibility is the obvious one. Want to see what’s queued? Run a SELECT. No separate management UI, no extra CLI tool. The jobs are where the rest of your data is, queryable the same way.

Transactional writes are the less obvious but more important one. You can insert the job row in the same transaction as the record that triggered it. If the transaction rolls back, the job disappears too. With an external queue there’s always a window where the DB commit succeeded but the enqueue failed, or the other way around. Both are bugs that only show up under load.

Durability comes for free too. Your database is already backed up. Your jobs are backed up with it. Any pending work, retry count, error message — it all comes back if the server dies.

Debugging gets simpler. When something fails, you check the row. The error is right there in the table. No hunting through queue consumer logs across services.

When This Breaks Down#

High throughput is the first limit. If you’re processing thousands of jobs per second, polling a table becomes a bottleneck. Queues use push delivery and are built for this. SQLite in particular has write concurrency limits that will hit you before Postgres does.

Multiple consumers is the second. This pattern assumes one worker pulling from the table. If you need to scale horizontally, multiple instances will race on the same rows. Postgres has SELECT ... FOR UPDATE SKIP LOCKED for this, but now you’re managing that complexity yourself.

Cross-service is the third. If the producer and consumer are different services, a shared database is tight coupling. A message queue is the right abstraction there — that’s what it was built for.

Near-instant pickup is the last one. Polling every N seconds means jobs wait up to N seconds to start. If you need sub-second job pickup, look at Postgres LISTEN/NOTIFY or just use a proper queue.

Is This Right for You?#

This pattern works if you have one backend process, your job volume is in the hundreds to low thousands per day, and you want fewer moving parts to deploy and debug.

It doesn’t work if you’re building something that needs to scale horizontally, process jobs in real time, or communicate across service boundaries.

In my case, the worker wakes up on a regular interval, finds nothing to do most of the time, and goes back to sleep. The one time a batch of emails failed mid-send, it caught them on the next cycle without me doing anything.

That’s the bar I was aiming for. Something that works quietly, without infrastructure I have to manage.