Apache Kafka Interview Questions

1. What is Apache Kafka and what problems does it solve?

2. What is the difference between Kafka and a traditional message queue?

3. What is a Kafka topic and how does partitioning work?

4. What is a Kafka partition and why is it the unit of parallelism?

5. What is a Kafka broker and how does a Kafka cluster work?

6. What is a producer in Kafka and how does it choose a partition?

7. What is a consumer group in Kafka and how does it enable scaling?

8. What is a Kafka offset and how is consumer position tracked?

9. What is the role of ZooKeeper in Kafka and what is KRaft mode?

10. How does Kafka guarantee message ordering?

11. What are the different delivery semantics in Kafka (at-most-once, at-least-once, exactly-once)?

12. How does Kafka achieve durability with replication and the ISR?

13. What is the difference between a leader and follower replica in Kafka?

14. What is acks configuration in a Kafka producer and what do 0, 1, and all mean?

15. How does log compaction work in Kafka?

16. What is the difference between Kafka retention by time and by size?

17. How does a Kafka consumer commit offsets and what is the risk of auto-commit?

18. What is consumer rebalancing in Kafka and why can it be disruptive?

19. What is Kafka Connect and when would you use it?

20. What is Kafka Streams and how does it differ from a plain consumer?

21. What is the difference between Kafka Streams and ksqlDB?

22. How does Kafka handle backpressure and slow consumers?

23. What is idempotent producer in Kafka and how does it prevent duplicates?

24. What is a transactional producer in Kafka?

25. How do you scale Kafka throughput?

26. What is the difference between Kafka and RabbitMQ?

27. How does Kafka handle message keys and what happens without a key?

28. What is the purpose of the __consumer_offsets topic in Kafka?

29. How does replication factor and min.insync.replicas affect availability in Kafka?

30. What are common causes of consumer lag in Kafka and how do you address it?

31. What causes Kafka consumer rebalance storms and how do you prevent them?

32. Why is Kafka partition count a one-way door, and how do you choose it?

33. How do you implement exactly-once processing in a Kafka read-process-write pipeline?

34. What causes a hot partition in Kafka and how do you fix it?

35. Why does the ISR shrink in Kafka, and when would you allow unclean leader election?

36. How do Schema Registry compatibility modes work, and how do you evolve a Kafka event schema safely?

37. How do you handle a poison message in Kafka and design a dead-letter topic?

38. How does Kafka rack awareness work, and how do you survive an availability-zone failure?

39. How do you reassign Kafka partitions and add brokers without downtime?

40. How do you tune a Kafka producer for throughput versus latency?

41. Which Kafka broker metrics do you monitor, and what do you actually alert on?

42. How do you secure a Kafka cluster with TLS, SASL and ACLs?

43. How do you perform a rolling upgrade of a Kafka cluster without downtime?

44. How do you diagnose and recover a Kafka broker that has run out of disk space?

45. How do you test Kafka producers, consumers and Streams applications reliably?

46. Why does a Kafka producer throw TimeoutException with 'Expiring N record(s)', and how do you debug it?

47. How do you operate Kafka Connect when a connector task keeps failing?

48. How do you replicate Kafka across clusters for disaster recovery, and why are offsets the hard part?

49. Why does Kafka broker performance depend on the OS page cache, and what does that change about tuning?

50. How do Kafka quotas work, and how do you isolate tenants on a shared cluster?

51. How do Kafka Streams state stores recover after a failure, and what do standby replicas change?

52. How do Kafka record timestamps work, and how do you replay from a point in time?

53. How does cooperative incremental rebalancing differ from eager rebalancing in Kafka?

54. How does static group membership work in Kafka, and when should you use it?

55. When should you use synchronous versus asynchronous offset commits in Kafka?

56. Where do Kafka's exactly-once semantics stop, and what must the application still handle?

57. What is the last stable offset, and how does read_committed isolation work in Kafka?

58. How do tombstones and delete.retention.ms work in a compacted Kafka topic?

59. How do you repartition a skewed Kafka key space without losing per-key ordering?

60. How do you tell whether Kafka consumer lag is a throughput shortfall or a stall?

61. How do you roll out a breaking Kafka schema change across producers and consumers?