ViewFS use in java application

[#1]

Hi,

I have already tested my HDFS HA cluster with my java application (i.e simple file upload/download).

Now that I have configured federated cluster, How can I use it in same code with few necessary modifications. Here is my code example for HDFS HA cluster. Please suggest for Federation HA cluster. Problem is I think my Configuration object is getting null while getting the FileSystem object for federated cluster.

conf.set("fs.defaultFS", "hdfs://HadoopTestHA");

conf.set("dfs.replication","4");
conf.set("dfs.ha.namenodes.HadoopTestHA", "nn1,nn2");
conf.set("dfs.namenode.rpc-address.HadoopTestHA.nn1","hadoop4ind.india:8020");
conf.set("dfs.namenode.rpc-address.HadoopTestHA.nn2","hadoop5ind.india:8020");
conf.set("dfs.client.failover.proxy.provider.HadoopTestHA","org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider");
copyFromLocalHDFS(conf,"Fed_Java_Test");  // File uploading code

[#2]

@Viraj Vekaria could you please try setting these properties in conf and see if it helps?

dfs.nameservices=HadoopTestHA

dfs.ha.namenodes.HadoopTestHA=nn1,nn2

dfs.namenode.rpc-address.HadoopTestHA.namenode1=hadoop4ind.india:8020

dfs.namenode.rpc-address.HadoopTestHA.namenode2=hadoop5ind.india:8020

dfs.client.failover.proxy.provider.HadoopTestHA=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider

[#3 · Accepted solution]

In your code, you may have to add the following line:

conf.set("dfs.nameservices", "HadoopTestHA")