# BullMQ worker deployment

The `workers/` directory ships a single-process Node worker that consumes the queues defined in `workers/index.ts`:

| Queue | Source | Purpose |
|-------|--------|---------|
| `photo-processing` | `workers/jobs/process-photo.ts` | Sharp-based resize / thumbnail / WebP variants |
| `email` | `workers/jobs/send-email.ts` | Outbound transactional email with exponential backoff |
| `default-galleries` | `workers/jobs/create-default-galleries.ts` | Per-user gallery scaffolding on signup |
| `push` | `workers/jobs/push.ts` | Web-push fan-out (R15 D.1) |

The Netlify Functions runtime cannot host long-lived workers. Pick one of the options below.

## Recommended hosts

### Railway

1. Connect the repo, point Railway at `node ./node_modules/.bin/tsx workers/index.ts`.
2. Add env: `REDIS_URL`, `DATABASE_URL`, `DIRECT_URL`, `VAPID_PUBLIC_KEY`, `VAPID_PRIVATE_KEY`, `RESEND_API_KEY`, anything the inner job needs.
3. Set restart-on-crash. Railway's default Nixpacks builder is fine.

### Fly.io

1. `fly launch` from the repo root with `process = ["worker"]`.
2. `fly.toml` adds:
   ```toml
   [processes]
     worker = "node ./node_modules/.bin/tsx workers/index.ts"

   [[mounts]]
     source = "data"
     destination = "/data"
   ```
3. Set secrets via `fly secrets set REDIS_URL=...` etc.

### Render

1. Create a Background Worker service.
2. Build: `npm install && npx prisma generate`
3. Start: `node ./node_modules/.bin/tsx workers/index.ts`
4. Add env vars in the dashboard.

## PM2 fallback (single VPS)

```bash
npm install -g pm2
pm2 start "npx tsx workers/index.ts" --name aw-workers --max-memory-restart 500M --restart-delay 5000
pm2 save
pm2 startup  # follow the printed instructions to enable on boot
```

## Monitoring with Bull Board

[`@bull-board/api`](https://github.com/felixmosh/bull-board) gives a web dashboard listing jobs across queues with replay/retry controls.

```ts
// scripts/bull-board.ts
import express from "express";
import { createBullBoard } from "@bull-board/api";
import { BullMQAdapter } from "@bull-board/api/bullMQAdapter";
import { ExpressAdapter } from "@bull-board/express";
import { Queue } from "bullmq";
import IORedis from "ioredis";

const connection = new IORedis(process.env.REDIS_URL!);
const adapter = new ExpressAdapter();
adapter.setBasePath("/admin/queues");

createBullBoard({
  queues: [
    new BullMQAdapter(new Queue("photo-processing", { connection })),
    new BullMQAdapter(new Queue("email", { connection })),
    new BullMQAdapter(new Queue("push", { connection })),
    new BullMQAdapter(new Queue("default-galleries", { connection })),
  ],
  serverAdapter: adapter,
});

const app = express();
app.use("/admin/queues", adapter.getRouter());
app.listen(3001, () => console.log("Bull Board on :3001"));
```

Run alongside the worker process. Gate access by IP allowlist or a basic-auth proxy — the board exposes job payloads.

## Health check

The worker process is a single point of failure. At minimum, configure the host's auto-restart + alert on consecutive crashes. If a queue depth spikes past expected steady state (Bull Board shows it under "Waiting"), the worker is wedged or a downstream service is down.

## Failure modes seen in production

- **Redis disconnect**: BullMQ retries internally with exponential backoff. Long disconnects cause job timeouts; check `REDIS_URL` reachability from the worker host.
- **Sharp memory pressure**: photo-processing concurrency is set to 3 — raise only after benchmarking the host's RAM. 50 concurrent uploads can swamp a 1GB instance.
- **Web-push 410**: stale subscription. The job handler prunes them automatically; spikes here usually correlate with a Service Worker version bump.