Java-Hadoop : Create a file in HDFS programmatically and write data into it

Here is a java program with its pom file which lets you to create a file in HDFS, write data into it. The pom file lets you create 2 jar files, one which has all the dependencies included in it.


import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;

public class createFileHdfs {

static String newFolder;

public static void main(String args[]) throws IOException

newFolder = args[0];
Path newFolderPath= new Path(newFolder);

FileSystem hdfs =FileSystem.get(new Configuration());
Path homeDir=hdfs.getHomeDirectory();
System.out.println(“Home folder -” +homeDir);

hdfs.delete(newFolderPath, true);


Path newFilePath=new Path(newFolder+”/newFile.txt”);

StringBuilder sb=new StringBuilder();
for(int i=1;i<=5;i++)
byte[] byt=sb.toString().getBytes();
FSDataOutputStream fsOutStream = hdfs.create(newFilePath);





<project xmlns=”; xmlns:xsi=”;






<!– Additional configuration. –>
<transformer implementation=”org.apache.maven.plugins.shade.resource.ManifestResourceTransformer”>



About shalishvj : My Experience with BigData

6+ years of experience using Bigdata technologies in Architect, Developer and Administrator roles for various clients. • Experience using Hortonworks, Cloudera, AWS distributions. • Cloudera Certified Developer for Hadoop. • Cloudera Certified Administrator for Hadoop. • Spark Certification from Big Data Spark Foundations. • SCJP, OCWCD. • Experience in setting up Hadoop clusters in PROD, DR, UAT , DEV environments.
This entry was posted in Java-Maven-Hadoop and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s