Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ViewFS use in java application

Solved Go to solution
Highlighted

ViewFS use in java application

New Contributor

Hi,

I have already tested my HDFS HA cluster with my java application (i.e simple file upload/download).

Now that I have configured federated cluster, How can I use it in same code with few necessary modifications. Here is my code example for HDFS HA cluster. Please suggest for Federation HA cluster. Problem is I think my Configuration object is getting null while getting the FileSystem object for federated cluster.

conf.set("fs.defaultFS", "hdfs://HadoopTestHA");

conf.set("dfs.replication","4");
conf.set("dfs.ha.namenodes.HadoopTestHA", "nn1,nn2");
conf.set("dfs.namenode.rpc-address.HadoopTestHA.nn1","hadoop4ind.india:8020");
conf.set("dfs.namenode.rpc-address.HadoopTestHA.nn2","hadoop5ind.india:8020");
conf.set("dfs.client.failover.proxy.provider.HadoopTestHA","org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider");
copyFromLocalHDFS(conf,"Fed_Java_Test");  // File uploading code
1 ACCEPTED SOLUTION

Accepted Solutions

Re: ViewFS use in java application

In your code, you may have to add the following line:

conf.set("dfs.nameservices", "HadoopTestHA")
2 REPLIES 2

Re: ViewFS use in java application

@Viraj Vekaria could you please try setting these properties in conf and see if it helps?

dfs.nameservices=HadoopTestHA

dfs.ha.namenodes.HadoopTestHA=nn1,nn2

dfs.namenode.rpc-address.HadoopTestHA.namenode1=hadoop4ind.india:8020

dfs.namenode.rpc-address.HadoopTestHA.namenode2=hadoop5ind.india:8020

dfs.client.failover.proxy.provider.HadoopTestHA=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider

Re: ViewFS use in java application

In your code, you may have to add the following line:

conf.set("dfs.nameservices", "HadoopTestHA")
Don't have an account?
Coming from Hortonworks? Activate your account here