Member since
02-10-2015
10
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3796 | 03-02-2015 08:55 AM |
05-12-2017
12:22 PM
I have recently created two node hadoop cluster with CDH5.11.0 with Cloudera manager. It installed spark 1.16 and I am able to launch spark-shell successfully. Later, I installed spark2 from CSD and getting below error when I try to start the service. + exec /data/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/lib/spark2/bin/spark-class org.apache.spark.deploy.history.HistoryServer --properties-file /var/run/cloudera-scm-agent/process/75-spark2_on_yarn-SPARK2_YARN_HISTORY_SERVER/spark2-conf/spark-history-server.conf
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:278)
at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala)
Caused by: java.io.FileNotFoundException: Log directory specified does not exist: file:/user/spark/spark2ApplicationHistory
at org.apache.spark.deploy.history.FsHistoryProvider.org$apache$spark$deploy$history$FsHistoryProvider$$startPolling(FsHistoryProvider.scala:207)
at org.apache.spark.deploy.history.FsHistoryProvider.initialize(FsHistoryProvider.scala:153)
at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:149)
at org.apache.spark.deploy.history.FsHistoryProvider.<init>(FsHistoryProvider.scala:77)
... 6 more
Caused by: java.io.FileNotFoundException: File file:/user/spark/spark2ApplicationHistory does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:598)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:811)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:588)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425)
at org.apache.spark.deploy.history.FsHistoryProvider.org$apache$spark$deploy$history$FsHistoryProvider$$startPolling(FsHistoryProvider.scala:197)
... 9 more File file:/user/spark/spark2ApplicationHistory does not exist I have verified this file exists in HDFS and has full access to any user. Please help me if there are any solution to come out of this problem.
... View more
Labels:
- Labels:
-
Spark
03-02-2015
08:55 AM
This is resolved after cleaning up oozie directory. In my case rm -rf /rms/var/lib/oozie/*
... View more
03-01-2015
03:13 PM
Create Oozie database step failed when setting up cluster. I did clean up my previous install as that was failed during deploy client config step. I tried after cleaning up /var/lib/oozie/* but no luck. Please let me know some solution to come out of this. I tried to connect psql uitlity but not recognizing my psql command. Error: DB schema exists
Stack trace for the error was (for debug purposes):
--------------------------------------
java.lang.Exception: DB schema exists
at org.apache.oozie.tools.OozieDBCLI.validateDBSchema(OozieDBCLI.java:929)
at org.apache.oozie.tools.OozieDBCLI.createDB(OozieDBCLI.java:184)
at org.apache.oozie.tools.OozieDBCLI.run(OozieDBCLI.java:127)
at org.apache.oozie.tools.OozieDBCLI.main(OozieDBCLI.java:78)
--------------------------------------
... View more
- Tags:
- client
03-01-2015
09:22 AM
I didn't folow those steps before enabling single user mode, Can i uninstall everything and resinstall without singe user mode? will that cause any issues?
... View more
03-01-2015
08:09 AM
Hi Jayanth, Can you please let me know what changes you made in /etc/sudoers file, I am getting same error and not able to resolve for 2days. I disabled #Defaults requiretty but getting following error. sudo: no tty present and no askpass program specified
... View more
02-10-2015
03:25 PM
Hi, This is the first time I am trying to install cloudera manager in Redhat linux environment. My /usr filesystem has limited space, so I am getting following error when installing cloudera manager using command line (automated installation path A). Transaction Check Error: installing package cloudera-manager-daemons-5.3.1-1.cm531.p0.191.el5.x86_64 needs 338MB on the /usr filesystem installing package cloudera-manager-server-5.3.1-1.cm531.p0.191.el5.x86_64 needs 338MB on the /usr filesystem Error Summary ------------- Disk Requirements: At least 338MB needed on the /usr filesystem. I dont have any unwanted directories to remove from /usr. Trying to understand if there are any alternative to install in different location. Thanks, Prakash A
... View more