Prakash144
Explorer
My Accepted Solutions
Show All

Spark2 History Server failed to start in CDH5.11.0

by Explorer in Support Questions
‎05-12-2017 12:22 PM
‎05-12-2017 12:22 PM
I have recently created two node hadoop cluster with CDH5.11.0 with Cloudera manager. It installed spark 1.16 and I am able to launch spark-shell successfully. Later, I installed spark2 from CSD and getting below error when I try to start the service.     + exec /data/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/bin/spark-class org.apache.spark.deploy.history.HistoryServer --properties-file /var/run/cloudera-scm-agent/process/75-spark2_on_yarn-SPARK2_YARN_HISTORY_SERVER/spark2-conf/spark-history-server.conf Exception in thread "main" java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:278) at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala) Caused by: java.io.FileNotFoundException: Log directory specified does not exist: file:/user/spark/spark2ApplicationHistory at org.apache.spark.deploy.history.FsHistoryProvider.org$apache$spark$deploy$history$FsHistoryProvider$$startPolling(FsHistoryProvider.scala:207) at org.apache.spark.deploy.history.FsHistoryProvider.initialize(FsHistoryProvider.scala:153) at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:149) at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:77) ... 6 more Caused by: java.io.FileNotFoundException: File file:/user/spark/spark2ApplicationHistory does not exist at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:598) at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:811) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:588) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425) at org.apache.spark.deploy.history.FsHistoryProvider.org$apache$spark$deploy$history$FsHistoryProvider$$startPolling(FsHistoryProvider.scala:197) ... 9 more   File file:/user/spark/spark2ApplicationHistory does not exist I have verified this file exists in HDFS and has full access to any user. Please help me if there are any solution to come out of this problem. ... View more
Labels:

Re: Creating Oozie database step failed during clu...

by Explorer in Support Questions
‎03-02-2015 08:55 AM
‎03-02-2015 08:55 AM
This is resolved after cleaning up oozie directory. In my case rm -rf /rms/var/lib/oozie/*     ... View more

Creating Oozie database step failed during cluster...

by Explorer in Support Questions
‎03-01-2015 03:13 PM
‎03-01-2015 03:13 PM
Create Oozie database step failed when setting up cluster. I did clean up my previous install as that was failed during deploy client config step. I tried after cleaning up /var/lib/oozie/* but no luck. Please let me know some solution to come out of this. I tried to connect psql uitlity but not recognizing my psql command.   Error: DB schema exists Stack trace for the error was (for debug purposes): -------------------------------------- java.lang.Exception: DB schema exists at org.apache.oozie.tools.OozieDBCLI.validateDBSchema(OozieDBCLI.java:929) at org.apache.oozie.tools.OozieDBCLI.createDB(OozieDBCLI.java:184) at org.apache.oozie.tools.OozieDBCLI.run(OozieDBCLI.java:127) at org.apache.oozie.tools.OozieDBCLI.main(OozieDBCLI.java:78) -------------------------------------- ... View more

Re: Error while Installing CDH 5.3.

by Explorer in Support Questions
‎03-01-2015 09:22 AM
‎03-01-2015 09:22 AM
I didn't folow those steps before enabling single user mode, Can i uninstall everything and resinstall without singe user mode? will that cause any issues? ... View more

Re: Error while Installing CDH 5.3.

by Explorer in Support Questions
‎03-01-2015 08:09 AM
‎03-01-2015 08:09 AM
Hi Jayanth,   Can you please let me know what changes you made in /etc/sudoers file, I am getting same error and not able to resolve for 2days.   I disabled #Defaults    requiretty   but getting following error. sudo: no tty present and no askpass program specified ... View more

Error when installing cloudera Manager. Need minim...

by Explorer in Support Questions
‎02-10-2015 03:25 PM
‎02-10-2015 03:25 PM
Hi,   This is the first time I am trying to install cloudera manager in Redhat linux environment. My /usr filesystem has limited space, so I am getting following error when installing cloudera manager using command line (automated installation path A).   Transaction Check Error:   installing package cloudera-manager-daemons-5.3.1-1.cm531.p0.191.el5.x86_64 needs 338MB on the /usr filesystem   installing package cloudera-manager-server-5.3.1-1.cm531.p0.191.el5.x86_64 needs 338MB on the /usr filesystem   Error Summary ------------- Disk Requirements:   At least 338MB needed on the /usr filesystem.   I dont have any unwanted directories to remove from /usr. Trying to understand if there are any alternative to install in different location.   Thanks, Prakash A ... View more
Contact Me
Online
Offline
Last Visited
‎05-23-2017 02:37 PM
Public Statistics
Date Registered ‎02-10-2015 03:29 PM
Last Visited ‎05-23-2017 02:37 PM
Total Messages Posted 10
; ;