Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

ViewFS use in java application

Solved Go to solution

ViewFS use in java application

Hi,

I have already tested my HDFS HA cluster with my java application (i.e simple file upload/download).

Now that I have configured federated cluster, How can I use it in same code with few necessary modifications. Here is my code example for HDFS HA cluster. Please suggest for Federation HA cluster. Problem is I think my Configuration object is getting null while getting the FileSystem object for federated cluster.

conf.set("fs.defaultFS", "hdfs://HadoopTestHA");

conf.set("dfs.replication","4");
conf.set("dfs.ha.namenodes.HadoopTestHA", "nn1,nn2");
conf.set("dfs.namenode.rpc-address.HadoopTestHA.nn1","hadoop4ind.india:8020");
conf.set("dfs.namenode.rpc-address.HadoopTestHA.nn2","hadoop5ind.india:8020");
conf.set("dfs.client.failover.proxy.provider.HadoopTestHA","org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider");
copyFromLocalHDFS(conf,"Fed_Java_Test");  // File uploading code
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: ViewFS use in java application

In your code, you may have to add the following line:

conf.set("dfs.nameservices", "HadoopTestHA")

View solution in original post

2 REPLIES 2
Highlighted

Re: ViewFS use in java application

@Viraj Vekaria could you please try setting these properties in conf and see if it helps?

dfs.nameservices=HadoopTestHA

dfs.ha.namenodes.HadoopTestHA=nn1,nn2

dfs.namenode.rpc-address.HadoopTestHA.namenode1=hadoop4ind.india:8020

dfs.namenode.rpc-address.HadoopTestHA.namenode2=hadoop5ind.india:8020

dfs.client.failover.proxy.provider.HadoopTestHA=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider

Highlighted

Re: ViewFS use in java application

In your code, you may have to add the following line:

conf.set("dfs.nameservices", "HadoopTestHA")

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here