Oozie : Coordinator Jobs : Quickly schedule a job

Hi Guys,

I am back..
Lets schedule the pig action developed in my previous post quickly…We would use oozie coordinator engine for the same.

In your Unix machine, make a folder and create a property file – coordinator.properties

oozie.coord.application.path=${nameNode}${oozieCoordXmlHdfsPath}
oozie.libpath=${nameNode}/user/oozie/share/lib
nameNode=NAMENODEHOST:8020
jobTracker=RMMANAGERHOST:8021
queueName=<QUEUENAME>

Now create the coordinator.xml file

<coordinator-app name=”oozie-coord” frequency=”10″ start=”2015-05-29T11:25Z” end=”2015-05-30T12:59Z” timezone=”Canada/Eastern” xmlns=”uri:oozie:coordinator:0.1″>
<action>
<workflow>
<app-path><HDFS PATH where workflow.xml is kept></app-path>
</workflow>
</action>
</coordinator-app>

and move it to a location in HDFS.

hdfs dfs -put coordinator.xml <hdfs-path>/coordinator.xml

Setting frequency 10, the job starts at start time specified and runs the job every 10 minutes till the end time is reached.

Lets run the scheduler from your local unix path where you created your coordinator.properties file.

oozie job -oozie http://OOZIEHOST:11000/oozie -config <unix localpath>/coordinator.properties -run

 

Enabling debugging would be helpful

export OOZIE_DEBUG=1

In your web browser, go to http://OOZIEHOST:11000/oozie 

under coordinator jobs tab to view the details about the job.

 

Stay tuned .. 🙂

Advertisements

About shalishvj : My Experience with BigData

6+ years of experience using Bigdata technologies in Architect, Developer and Administrator roles for various clients. • Experience using Hortonworks, Cloudera, AWS distributions. • Cloudera Certified Developer for Hadoop. • Cloudera Certified Administrator for Hadoop. • Spark Certification from Big Data Spark Foundations. • SCJP, OCWCD. • Experience in setting up Hadoop clusters in PROD, DR, UAT , DEV environments.
This entry was posted in oozie and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s