Apache Storm

Architecture / Components

  • Nimbus and Supervisor daemons are designed to be
    • fail-fast (process self-destructs whenever any unexpected situation is encountered)
    • stateless (all state is kept in Zookeeper or on disk)
  • Nimbus and Supervisor daemons must be run under supervision using a tool like daemontools or monit.
  • So if the Nimbus or Supervisor daemons die, they restart like nothing happened.

Topology

Graph of computation where the

nodes represent some individual computations and

edges data being passed between nodes

Tuple

Ordered list of values

Storm provides mechanisms for assigning names to the values.

Stream

Unbounded sequence of Tuples.

Spout

Source of stream in  a topology

Bolt

Accepts tuple from input stream

performs computation / transformation

Emits a new tuple to its output stream

**Each spout and bolt will have one or many individual instances that perform all of this processing in parallel

Stream Grouping 

tells Storm how to send individual tuples between instances of the spout and bolts

Shuffle Grouping: tuples are emitted to instances of bolts at random

 

Advertisements

About shalishvj : My Experience with BigData

6+ years of experience using Bigdata technologies in Architect, Developer and Administrator roles for various clients. • Experience using Hortonworks, Cloudera, AWS distributions. • Cloudera Certified Developer for Hadoop. • Cloudera Certified Administrator for Hadoop. • Spark Certification from Big Data Spark Foundations. • SCJP, OCWCD. • Experience in setting up Hadoop clusters in PROD, DR, UAT , DEV environments.
This entry was posted in Storm, Uncategorized and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s