Tag Archives: Kafka

Integrating Kafka and Storm

Intro This article focuses on integrating Storm 0.10.0.2 and Kafka 0.8.2.2 Data is not encrypted in this case Create client_jaas file (under /usr/hdp/current/storm-client/conf/) KafkaClient{ com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true renewTicket=true serviceName=”kafka”; }; Client{ com.sun.security.auth.module.Krb5LoginModule required useTicketCache=true renewTicket=false serviceName=”zk”; }; StormClient{ com.sun.security.auth.module.Krb5LoginModule required … Continue reading

Posted in Kafka, Storm | Tagged , , | Leave a comment

Kafka Producers and Consumers (Console / Java) using SASL_SSL

Intro Producers / Consumers help to send / receive message to / from Kafka SASL is used to provide authentication and SSL for encryption JAAS config files are used to read kerberos ticket and authenticate as a part of SASL … Continue reading

Posted in Kafka | Tagged , , , , | Leave a comment

Hands-on Kafka

About Kafka Messaging system It doesnt transform data Messages are organized into Topics Producers push messages Consumers pull messages Kafka runs in a cluster Nodes are called brokers Why Kafka –¬†Advantages¬† Large number of Consumers Ad-hoc consumers Batch Consumers Automatic … Continue reading

Posted in Kafka | Tagged , , , , , , | Leave a comment