Comment on Scaling request logging with ClickHouse, Kafka, and VectorparentComments−anticodon10moOr vice versa: make ClickHouse ingest batches directly from Kafka. Messages are already buffered in Kafka, I don't get why Vector is necessary here.−ch202610motbh the only thing they needed was a correct schema that didn’t constantly spawn new parts and async_insert enabled.−est10mohttps://clickhouse.com/docs/knowledgebase/kafka-to-clickhous...For anyone if curious.
Comments
Or vice versa: make ClickHouse ingest batches directly from Kafka. Messages are already buffered in Kafka, I don't get why Vector is necessary here.
tbh the only thing they needed was a correct schema that didn’t constantly spawn new parts and async_insert enabled.
https://clickhouse.com/docs/knowledgebase/kafka-to-clickhous...
For anyone if curious.