#crawler architecture

Migrating Job State Management from Redis to Postgres: Why We Centralized Crawler Jobs in a Single Source of Truth

We moved job claiming in the Vultr Crawler from Redis to Postgres for better consistency, auditability, and operational simplicity.