question Re: RuntimeException running the hdfs command (core-site.xml not found) in Archives of Support Questions (Read Only)

RuntimeException running the hdfs command (core-site.xml not found)

TS — Fri, 05 Jun 2015 00:30:19 GMT

I have just installed CDH 5.4 into a new cluster.

CM console looks good. But, when I run the hdfs command I get the following error:

<><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>

$ hdfs dfs -ls /
Exception in thread "main" java.lang.RuntimeException: core-site.xml not found
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2476)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2402)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2319)
        at org.apache.hadoop.conf.Configuration.set(Configuration.java:1057)
        at org.apache.hadoop.conf.Configuration.set(Configuration.java:1029)
        at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1367)
        at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:319)
        at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:485)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
        at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)

Re: RuntimeException running the hdfs command (core-site.xml not found)

GautamG — Fri, 05 Jun 2015 00:34:52 GMT

On the node where the command is issues, can you please check if
/etc/hadoop/conf/core-site.xml is present?

Is this node marked as an HDFS Gateway? If so can you try redeploying
client configuration across the cluster (dropdown action next to the
cluster name in Cloudera Manager) and trying again?

Re: RuntimeException running the hdfs command (core-site.xml not found)

TS — Fri, 05 Jun 2015 00:38:52 GMT

Actually it is under the /etc/hadoop/conf.cloudera.hdfs0 folder: [root@]# ls -l /etc/hadoop/conf.cloudera.hdfs0/ total 32 -rw-r--r-- 1 root root 20 Jun 4 23:18 __cloudera_generation__ -rw-r--r-- 1 root root 3558 Jun 4 23:18 core-site.xml -rw-r--r-- 1 root root 2546 Jun 4 23:18 hadoop-env.sh -rw-r--r-- 1 root root 1678 Jun 4 23:18 hdfs-site.xml -rw-r--r-- 1 root root 314 Jun 4 23:18 log4j.properties -rw-r--r-- 1 root root 315 Jun 4 23:18 ssl-client.xml -rw-r--r-- 1 root root 647 Jun 4 23:18 topology.map -rwxr-xr-x 1 root root 1510 Jun 4 23:18 topology.py*

Re: RuntimeException running the hdfs command (core-site.xml not found)

TS — Fri, 05 Jun 2015 00:40:13 GMT

Here is a better 'picture': <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><> [root@]# ls -l /etc/hadoop/conf.cloudera.hdfs0/ total 32 -rw-r--r-- 1 root root 20 Jun 4 23:18 __cloudera_generation__ -rw-r--r-- 1 root root 3558 Jun 4 23:18 core-site.xml -rw-r--r-- 1 root root 2546 Jun 4 23:18 hadoop-env.sh -rw-r--r-- 1 root root 1678 Jun 4 23:18 hdfs-site.xml -rw-r--r-- 1 root root 314 Jun 4 23:18 log4j.properties -rw-r--r-- 1 root root 315 Jun 4 23:18 ssl-client.xml -rw-r--r-- 1 root root 647 Jun 4 23:18 topology.map -rwxr-xr-x 1 root root 1510 Jun 4 23:18 topology.py*

Re: RuntimeException running the hdfs command (core-site.xml not found)

TS — Fri, 05 Jun 2015 00:52:13 GMT

Look at some interesting soft links below: [root@]# ls -l /etc/hadoop/conf lrwxrwxrwx 1 root root 29 Jun 4 23:18 /etc/hadoop/conf -> /etc/alternatives/hadoop-conf ---- [root@]# ls -l /etc/alternatives/hadoop-conf lrwxrwxrwx 1 root root 35 Jun 4 23:18 /etc/alternatives/hadoop-conf -> /etc/hadoop/conf.cloudera.mapreduce ---- [root@]# ls -l /etc/hadoop/conf.cloudera.mapreduce ls: cannot access /etc/hadoop/conf.cloudera.mapreduce: No such file or directory How do I fix them???

Re: RuntimeException running the hdfs command (core-site.xml not found)

TS — Fri, 05 Jun 2015 01:09:19 GMT

Issue has been resolved! I had to create a new sl for /etc/alternatives/hadoop-conf