Two Redis instances, opposite eviction policies
Why the cache and the queue run the same technology with contradictory configuration, and the class of bug that forces it.
- Client
- Jabal Al Noor Pharmacy
- Industry
- Pharmacy retail
- Outcome
- No silently dropped jobs
Same technology, opposite correct answer
This is a small decision with an outsized consequence, and it is the kind of thing that only shows up after an incident.
Redis is split into two instances with opposite eviction policies:
| Instance | Policy | Behaviour at the memory ceiling |
|---|---|---|
| Cache | allkeys-lru | Evicts the least-recently-used key |
| Queue | noeviction | Returns OOM errors on write |
Why they cannot share a configuration
A dropped BullMQ job corrupts the queue: work vanishes with no error, no retry and no record, and the system reports success. An evicted cache key costs a round trip.
Stale cache data is expendable, so LRU is exactly right under pressure. Lost queue data is not, so failing the write loudly is exactly right. Same technology, opposite correct answer.
This matters more here than in most systems, because the queues carry the catalogue sync, the AI classification deferrals, and the delayed retry schedule described in the other write-ups. Those delayed jobs live in Redis specifically so the schedule survives a worker restart. An eviction policy that discarded them would quietly undo that guarantee.
The wider platform
The API runs in Docker under PM2 cluster mode on a VPS. Staging and production are co-hosted on one host, isolated by Docker Compose project name, with a shared nginx edge container as the only thing binding ports 80 and 443.
CI/CD hardening
- Every GitHub Action pinned to a commit SHA, with
harden-runneron every job - Least-privilege
contents: read - End-to-end tests against an ephemeral database branch, created and destroyed per run, so there is no cross-run bleed and migrations are validated against the real managed engine
- Production deploys additionally gated on a vulnerability scan that fails the build on CRITICAL or HIGH
A test that links TypeScript to nginx
One test in that suite is worth describing, because it connects two things nothing else does: the shared-types product-images spec reads the nginx config files and asserts the edge still permits the upload budget the API advertises.
It is the only mechanical link between a TypeScript constant and an nginx directive. Without it, the failure it catches — an oversized batch answered by nginx with an empty 413 — is invisible until a staff member hits it in production.