#crawler architecture

Migrating Job State Management from Redis to Postgres: Why I Centralized Crawler Jobs in a Single Source of Truth

I moved job claiming in the Vultr Crawler from Redis to Postgres for better consistency, auditability, and operational simplicity.