Processing Data: Integrating Kafka with Python & Using Consumer Groups

Skillsoft issued completion badges are earned based on viewing the percentage required or receiving a passing score when assessment is required. Producers and consumers are applications that write events to and read events from Kafka. In this course, you will focus on integrating Python applications with a Kafka environment, implementing consumer groups, and tweaking Kafka configurations. Begin by connecting to Kafka from Python. You will produce to and consume messages from a Kafka topic using Python. Next, discover how to tweak Kafka broker configurations. You will place limits on the size of messages and disable deletion of topics. Then, publish messages to partitioned topics and explore the use of partitioning algorithms to determine the placement of messages on partitions. Explore consumer groups, which allow a set of consumers to process messages published to partitioned Kafka topics in parallel - without any duplication of effort. Finally, learn different ways to optimize Kafka's performance, using configurations for brokers and topics, as well as producer and consumer apps.