Category Archives: Kafka

Integrating Kafka and Storm

Intro This article focuses on integrating Storm and Kafka Data is not encrypted in this case Create client_jaas file (under /usr/hdp/current/storm-client/conf/) KafkaClient{ required useTicketCache=true renewTicket=true serviceName=”kafka”; }; Client{ required useTicketCache=true renewTicket=false serviceName=”zk”; }; StormClient{ required … Continue reading

Posted in Kafka, Storm | Tagged , , | Leave a comment

Kafka Producers and Consumers (Console / Java) using SASL_SSL

Intro Producers / Consumers help to send / receive message to / from Kafka SASL is used to provide authentication and SSL for encryption JAAS config files are used to read kerberos ticket and authenticate as a part of SASL … Continue reading

Posted in Kafka | Tagged , , , , | Leave a comment

Hands-on Kafka

About Kafka Messaging system It doesnt transform data Messages are organized into Topics Producers push messages Consumers pull messages Kafka runs in a cluster Nodes are called brokers Why Kafka –¬†Advantages¬† Large number of Consumers Ad-hoc consumers Batch Consumers Automatic … Continue reading

Posted in Kafka | Tagged , , , , , , | Leave a comment