Tag Archives: sqoop

Tips: Sqoop

Override Cluster properties Eg:- disable compression for sqoop output when compression is turned on in the cluster sqoop import -Dmapred.job.queue.name=default \ -Dmapreduce.map.output.compress=false \ -Dmapreduce.output.fileoutputformat.compress=false \ –driver com.ibm.db2.jcc.DB2Driver –connect jdbc:db2://<host>/<db>\ –username <user>–password <pwd> \ –table <db2 table> –target-dir <hdfs path> \ … Continue reading

Posted in sqoop, Tips | Tagged , , , , , | Leave a comment

Sqoop : Incremental Imports using Last-Modified mode

As discussed in my previous post, Sqoop is a tool designed to transfer data between Hadoop and relational databases. Incremental imports mode can be used to retrieve only rows newer than some previously-imported set of rows. Why & When Last-Modified … Continue reading

Posted in sqoop | Tagged , , , , | 3 Comments

Sqoop : Incremental Imports using Append mode

As you all know, Sqoop is a tool designed to transfer data between Hadoop and relational databases. Incremental imports mode can be used to retrieve only rows newer than some previously-imported set of rows. Why¬†Append mode ?? works¬†for numerical data … Continue reading

Posted in sqoop | Tagged , , , , , | 7 Comments