Kafka Internals: FAQs

DZone 's Guide to

Kafka Internals: FAQs

A quick cheat sheet for the basic questions new comers to Apache Kafka often face,.

· Big Data Zone ·
Free Resource

Is there chance that a consumer can read from more than one partition in a topic?

Yes, of course. If the number of consumers is less than the number of partitions in the topic.

What is the optimal number of consumers in a group for a topic?

As many as the number of partitions.

Is there a chance for a consumer to be idle?

Yes, if the number of live consumers in a group is larger than the number of partitions. These idle consumers will be leveraged when a consumer crashes.

Does the Kafka broker retain messages until they are consumed by all subscribers?

No, it is the consumer's responsibility to consume messages when they are available in a topic.

It is analogous to subscribing and watching content from satellite channel. The subscriber should tune to the channel while the content is being played to watch.

If you want to take a closer look at Apache Kafka, check out my previous two articles: 

apache kafka, big data, kafka partitions, kafka topics

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}