Cloudera Community
TS
user rank
Contributor

CM's Metadata Data Store (Postgres)

by Contributor in Support Questions
‎07-15-2015 01:18 PM
‎07-15-2015 01:18 PM
All, Your opinion on this topic will be greatly appreciated. I understand the default RDBMS for CM's metadata data store is PostgresDB. Because my firm's strategic RDBMS list doesn't include Postgres I am forced to deploy another relational database. <Q1> How often CM talks to its backend metadata store? <Q2> Is it possible to quantify the amount of data traffic from the data store to the apps? <Q3> Should (or must) the database exist into the same server as CM?? (In my view YES, but I am looking for strong justifications!!!)   The same questions hold for CDH services (Hive, Oozie, Hue) where I would like to have MySQL instead of a 'strategic' RDBMS!!!   ... View more

Re: Which CDH release will include Spark 1.4.x?

by Contributor in Support Questions
‎07-03-2015 05:40 AM
‎07-03-2015 05:40 AM
That's even better! I could give a try, at least for my Lab environment. Could you please provide some info, links, docs, blogs how this could be done?   ... View more

Re: Which CDH release will include Spark 1.4.x?

by Contributor in Support Questions
‎07-03-2015 05:34 AM
‎07-03-2015 05:34 AM
Greta, thank you for your quick response! Hoping to have CDH 5.5 released sooner than 4-5 months 🙂   ... View more

Which CDH release will include Spark 1.4.x?

by Contributor in Support Questions
‎07-03-2015 05:25 AM
1 Kudo
‎07-03-2015 05:25 AM
1 Kudo
Does anyone which CDH release will include/support Spark 1.4? If so, any timetable?   ... View more

Re: RuntimeException running the hdfs command (cor...

by Contributor in Support Questions
‎06-04-2015 06:09 PM
‎06-04-2015 06:09 PM
Issue has been resolved! I had to create a new sl for /etc/alternatives/hadoop-conf ... View more

Re: RuntimeException running the hdfs command (cor...

by Contributor in Support Questions
‎06-04-2015 05:52 PM
‎06-04-2015 05:52 PM
Look at some interesting soft links below: [root@]# ls -l /etc/hadoop/conf lrwxrwxrwx 1 root root 29 Jun 4 23:18 /etc/hadoop/conf -> /etc/alternatives/hadoop-conf ---- [root@]# ls -l /etc/alternatives/hadoop-conf lrwxrwxrwx 1 root root 35 Jun 4 23:18 /etc/alternatives/hadoop-conf -> /etc/hadoop/conf.cloudera.mapreduce ---- [root@]# ls -l /etc/hadoop/conf.cloudera.mapreduce ls: cannot access /etc/hadoop/conf.cloudera.mapreduce: No such file or directory How do I fix them??? ... View more

Re: RuntimeException running the hdfs command (cor...

by Contributor in Support Questions
‎06-04-2015 05:40 PM
‎06-04-2015 05:40 PM
Here is a better 'picture': <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><> [root@]# ls -l /etc/hadoop/conf.cloudera.hdfs0/ total 32 -rw-r--r-- 1 root root 20 Jun 4 23:18 __cloudera_generation__ -rw-r--r-- 1 root root 3558 Jun 4 23:18 core-site.xml -rw-r--r-- 1 root root 2546 Jun 4 23:18 hadoop-env.sh -rw-r--r-- 1 root root 1678 Jun 4 23:18 hdfs-site.xml -rw-r--r-- 1 root root 314 Jun 4 23:18 log4j.properties -rw-r--r-- 1 root root 315 Jun 4 23:18 ssl-client.xml -rw-r--r-- 1 root root 647 Jun 4 23:18 topology.map -rwxr-xr-x 1 root root 1510 Jun 4 23:18 topology.py* ... View more

Re: RuntimeException running the hdfs command (cor...

by Contributor in Support Questions
‎06-04-2015 05:38 PM
‎06-04-2015 05:38 PM
Actually it is under the /etc/hadoop/conf.cloudera.hdfs0 folder: [root@]# ls -l /etc/hadoop/conf.cloudera.hdfs0/ total 32 -rw-r--r-- 1 root root 20 Jun 4 23:18 __cloudera_generation__ -rw-r--r-- 1 root root 3558 Jun 4 23:18 core-site.xml -rw-r--r-- 1 root root 2546 Jun 4 23:18 hadoop-env.sh -rw-r--r-- 1 root root 1678 Jun 4 23:18 hdfs-site.xml -rw-r--r-- 1 root root 314 Jun 4 23:18 log4j.properties -rw-r--r-- 1 root root 315 Jun 4 23:18 ssl-client.xml -rw-r--r-- 1 root root 647 Jun 4 23:18 topology.map -rwxr-xr-x 1 root root 1510 Jun 4 23:18 topology.py* ... View more

RuntimeException running the hdfs command (core-si...

by Contributor in Support Questions
‎06-04-2015 05:30 PM
‎06-04-2015 05:30 PM
I have just installed CDH 5.4 into a new cluster.   CM console looks good. But, when I run the hdfs command I get the following error:   <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><> $ hdfs dfs -ls / Exception in thread "main" java.lang.RuntimeException: core-site.xml not found         at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2476)         at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2402)         at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2319)         at org.apache.hadoop.conf.Configuration.set(Configuration.java:1057)         at org.apache.hadoop.conf.Configuration.set(Configuration.java:1029)         at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1367)         at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:319)         at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:485)         at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)         at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)         at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)   ... View more
Labels:

Re: hdfs:/user/spark/share/lib/spark-assembly.jar ...

by Contributor in Support Questions
‎05-26-2015 12:28 PM
‎05-26-2015 12:28 PM
I have upgraded both CM & CDH to 5.4 release.   ... View more
Contact Me
Online
Offline
Last Visited
‎05-18-2016 02:06 PM
Community Statistics
Member Since ‎02-10-2015 06:48 AM
Last Visited ‎05-18-2016 02:06 PM
Posts 84
Kudos received 2