Cloudera Community
vikash145
user rank
Contributor
My Accepted Solutions
Show All

Re: Spark 2.1 didn't work in CDH 5.12

by Contributor in Support Questions
‎07-28-2017 09:58 AM
‎07-28-2017 09:58 AM
@mbigelow Thank you, that solved the issue. The /opt/cloudera/csd dir was empty, I downloaded CSD and configured it and then re-added service. The above exception is resolved. ... View more

Re: Spark 2.1 didn't work in CDH 5.12

by Contributor in Support Questions
‎07-27-2017 11:52 PM
‎07-27-2017 11:52 PM
@mbigelow  I have tried to add but there is no option in clouder-manager UI to add anything called 'Spark2' . It has 'Spark' and 'Spark(Standalone)' only. PFA screenshot.              ... View more

Spark 2.1 didn't work in CDH 5.12

by Contributor in Support Questions
‎07-27-2017 09:46 PM
‎07-27-2017 09:46 PM
I have installed and activated spark 2.1 from CDH 5.12 parcel. But its not working or configured properly. Executing command 'spark2-shell' gives below exception:   $ spark2-shell Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream at org.apache.spark.deploy.SparkSubmitArguments$$anonfun$mergeDefaultSparkProperties$1.apply(SparkSubmitArguments.scala:118) at org.apache.spark.deploy.SparkSubmitArguments$$anonfun$mergeDefaultSparkProperties$1.apply(SparkSubmitArguments.scala:118) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.deploy.SparkSubmitArguments.mergeDefaultSparkProperties(SparkSubmitArguments.scala:118) at org.apache.spark.deploy.SparkSubmitArguments.<init>(SparkSubmitArguments.scala:104) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FSDataInputStream at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358)   It seems looking for some Hadoop class file, but its not available. I have hadoop version:   $ hadoop version Hadoop 2.6.0-cdh5.12.0 Subversion http://github.com/cloudera/hadoop -r dba647c5a8bc5e09b572d76a8d29481c78d1a0dd Compiled by jenkins on 2017-06-29T11:31Z Compiled with protoc 2.5.0 From source with checksum 7c45ae7a4592ce5af86bc4598c5b4 This command was run using /opt/cloudera/parcels/CDH-5.12.0-1.cdh5.12.0.p0.29/jars/hadoop-common-2.6.0-cdh5.12.0.jar   Do I need to configure anything explicitly ?   Also, there are no files in ' /etc/spark2/conf' , which looks to me that spark2 is not configured properly. ... View more
Labels:

The Report Manager could not complete the request....

by Contributor in Support Questions
‎07-18-2017 08:55 AM
1 Kudo
‎07-18-2017 08:55 AM
1 Kudo
I have used CDH 5.12 for installation. While accessing HDFS -> File Browser , I am getting below exception.   11:46:24.924 AM INFO HeadlampServiceImpl Searching for: '[QueryTerm(fileSearchType:PATH, queryText:/, isPrefix:false)]' 11:46:24.924 AM ERROR HeadlampServiceImpl Index not initialized. query: '+PATH:/' java.lang.Exception: Index not initialized. at com.cloudera.headlamp.HeadlampServiceImpl.logAndThrowException(HeadlampServiceImpl.java:706) at com.cloudera.headlamp.HeadlampServiceImpl.performQuery(HeadlampServiceImpl.java:616) at com.cloudera.headlamp.HeadlampServiceImpl.fileSearch2(HeadlampServiceImpl.java:379) at com.cloudera.headlamp.HeadlampRequestRedirector.fileSearch2(HeadlampRequestRedirector.java:80) at com.cloudera.headlamp.api.HeadlampService$Processor$fileSearch2.getResult(HeadlampService.java:1285) at com.cloudera.headlamp.api.HeadlampService$Processor$fileSearch2.getResult(HeadlampService.java:1273) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:32) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:176) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) ... View more
Labels:

Re: Spark History Server failed to start

by Contributor in Support Questions
‎07-17-2017 09:49 PM
1 Kudo
‎07-17-2017 09:49 PM
1 Kudo
/user/spark directory was missing. I created it manually and it worked fine. ... View more

Re: HIVE Metastore server fails to start

by Contributor in Support Questions
‎07-17-2017 11:41 AM
‎07-17-2017 11:41 AM
@mbigelow   Thank you for pointing the issue. I created DB manually from UI and now HIVE is fine. ... View more

Re: HIVE Metastore server fails to start

by Contributor in Support Questions
‎07-17-2017 11:35 AM
‎07-17-2017 11:35 AM
My bad, I didn't searched as root. I have postgresql running, but i didn't found any database there.   sudo psql -U cloudera-scm -p 7432 -h localhost -d postgres   postgres-# l postgres-# ... View more

Re: HIVE Metastore server fails to start

by Contributor in Support Questions
‎07-17-2017 11:25 AM
‎07-17-2017 11:25 AM
This is weird. Just checked postgresql is not running and when I tried to start it manually. it failed. But I remember, during the installation phase, the DB connection test was succesful. Can I recover DB now ffrom CLI or clodera manager ? ... View more

Re: Spark History Server failed to start

by Contributor in Support Questions
‎07-17-2017 11:06 AM
‎07-17-2017 11:06 AM
Once this HIVE service is launched. I will post the log. ... View more

Re: HIVE Metastore server fails to start

by Contributor in Support Questions
‎07-17-2017 11:05 AM
‎07-17-2017 11:05 AM
I did the search in   /var/run/cloudera-scm-agent/process/39-hive-HIVEMETASTORE:   hive-site.xml:7: <value>jdbc:postgresql://compute-15:7432/hive</value> ... View more
Contact Me
Online
Offline
Last Visited
‎08-21-2017 10:09 PM
Community Statistics
Member Since ‎07-11-2017 09:18 AM
Last Visited ‎08-21-2017 10:09 PM
Posts 31
Kudos received 2