Change Data Capture with Apache Kafka, PostgreSQL, Kafka Connect and Debezium

Standard

I always wondered how Enterprise Systems are able to perform analytics on continuously increasing data. I have read articles which talk about Map Reduce jobs to perform bulk of data processing and later on utilize tools like Kafka, Flink etc. This enables to perform continuous data streaming and keep analytics in sync with latest changes. I admit I am still noob to all this stuff, But my curiosity doesn’t let me rest. I definitely wanted to get started somewhere. Recently I started working on a problem which needed a solution to build reliable batch system.

Continue reading