Support Questions

Find answers, ask questions, and share your expertise

How to copy files from one location(/tmp/files folder) to another location(to hdfs://user/username directory) using java application

I'm trying to move files from one location to another location while executing java application on execution server

Example:From one location(/tmp/files folder) to another location(hdfs://user/username directory) using java application.


Currently I'm doing it using hdfs dfs -cp -f file:"/tmp/files" hdfs:/user/username in script.

I want to do it in java now

Expert Contributor

@Amithesh Merugu

Use this method: copyFromLocalFile(Path src, Path dst). The first parameter is a path on your local disk (in your example /tmp/files) and the second is the HDFS path (hdfs://user/username). The documentation doesn't make it clear, but the source can be a dictionary and then the whole content is copied to the HDFS.

FileSystem fs = FileSystem.get(hdfsUri, conf);
fs.copyFromLocalFile(new Path("/tmp/files"), new Path("/user/username"));


Thanks for quick hdfs uri looks like "hdfs://demo"

I'm getting below exception demo

java.lang.IllegalArgumentException: demo
	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(
	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(
	at org.apache.hadoop.hdfs.DFSClient.<init>(
	at org.apache.hadoop.hdfs.DFSClient.<init>(
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(
	at org.apache.hadoop.fs.FileSystem.createFileSystem(
	at org.apache.hadoop.fs.FileSystem.access$200(
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(
	at org.apache.hadoop.fs.FileSystem$Cache.get(
	at org.apache.hadoop.fs.FileSystem.get(
	at org.apache.hadoop.fs.FileSystem.get(
	at java.util.concurrent.Executors$
	at java.util.concurrent.ThreadPoolExecutor.runWorker(
	at java.util.concurrent.ThreadPoolExecutor$
Caused by: demo

Expert Contributor

@Amithesh Merugu Try to use the IP address of the NameNode. And also add the port (default is 8020).