Recommendations on Improving the debezium performance even on medium workload

[Durgamahesh Manne <maheshpostgres9@xxxxxxxxx>]

Hi Team 

How do we improve the debezium performance?

Recommendations at kafka configuration side

Agenda is to minimize the lag during moderate or high work load on db 

default > poll.interval.ms = 500ms 
Recommended value for balanced performance > 1000ms
Recommended value for high throughput > 1000ms to 5000ms

 

 

default > max.batch.size = 2048 allocated > 2560
Recommended value for balanced performance > 500 to 1000
Recommended value for high throughput > 2000 or 5000

 

default > max.queue.size = 8192 allocated > 10000
Recommended value for balanced performance > 7500 (1000 to 10000)
Recommended value for high throughput > 10000

based on kafka load we can optimize debezium connector performance 

 

default > heartbeat.interval.ms = 0 allocated > 5000ms
Recommended value for this > 5000ms to 10000ms

Are these helpful  ?

Need auto connectivity even on connector(slot) failure 

please let me know your recommendations please 

Regards,

Durga Mahesh