During the Salama campaign the persister (a shared, config-driven Kafka→PostgreSQL sink for ~101 topics) fell behind by up to 6.4M messages; inserts were stuck at 2–3k/s and duplicate-key errors flooded. The team applied emergency patches that restored service. But need to have a long term solution.
Created a design doc.
During the Salama campaign the persister (a shared, config-driven Kafka→PostgreSQL sink for ~101 topics) fell behind by up to 6.4M messages; inserts were stuck at 2–3k/s and duplicate-key errors flooded. The team applied emergency patches that restored service. But need to have a long term solution.
Created a design doc.