Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on
03-15-2018
02:22 AM
- last edited on
04-21-2026
06:44 AM
by
GrazittiAPI
My system have
2 NameNode as Cluster (nn1 active, nn2 standby)
2 DataNode
1 Yarn Resource Manager service is on nn1
2 Yarn Node Manager is on 2 DataNode
Before config cluster NameNode everything work well
After config Namde Node as cluster , Yarn start ok but when submit script from Hue I have some error
2018-03-15 15:46:49,203 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Failed to download resource { { hdfs://mycluster/user/oozie/share/lib/lib_20171215093741/pig/jsp-2.1-6.1.14.jar, 1513305464001, FILE, null },pending,[(container_1521103549281_0001_02_000001)],8742037577451777,DOWNLOADING}
java.lang.IllegalArgumentException: java.net.UnknownHostException: mycluster
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:406)
at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:728)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:671)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:155)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2815)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:98)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2852)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2834)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:387)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:249)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: mycluster
View log on both Yarn Node manager service I see above error .
How Can I fix this
Thanks
Created 03-15-2018 10:32 AM
Have a look at your HDFS configuration and the name you are using for Name Node service. Confirm that you are using correct name for Name Node service. You can look at your hdfs-site.xml file on your deployed Hue server to confirm.
Created 03-15-2018 10:32 AM
Have a look at your HDFS configuration and the name you are using for Name Node service. Confirm that you are using correct name for Name Node service. You can look at your hdfs-site.xml file on your deployed Hue server to confirm.
Created 03-15-2018 08:10 PM
After check again hdfs-site.xml on 2 DataNode where Yarn Node Manager stand on , I see that the hdfs-site file missing this line when compare with the hdfs-site on Name Node
<property> <name>dfs.client.failover.proxy.provider.radcluster</name> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> </property>
restart the Yarn Node Manager Service , It work now , Thanks you