Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

When i try to run this tutorial " http://hortonworks.com/hadoop-tutorial/a-lap-around-apache-spark/ " i found this error "Error: Could not find or load main class Dhdp.version=2.3.4.1-10 " but i couldn't resolve it :/

Highlighted

When i try to run this tutorial " http://hortonworks.com/hadoop-tutorial/a-lap-around-apache-spark/ " i found this error "Error: Could not find or load main class Dhdp.version=2.3.4.1-10 " but i couldn't resolve it :/

New Contributor

2959-java-opts.png

This is the file Java Opts & i work with HDP 2.4.0.0-169

3 REPLIES 3

Re: When i try to run this tutorial " http://hortonworks.com/hadoop-tutorial/a-lap-around-apache-spark/ " i found this error "Error: Could not find or load main class Dhdp.version=2.3.4.1-10 " but i couldn't resolve it :/

Mentor

Did you try replacing the version with yours?

Re: When i try to run this tutorial " http://hortonworks.com/hadoop-tutorial/a-lap-around-apache-spark/ " i found this error "Error: Could not find or load main class Dhdp.version=2.3.4.1-10 " but i couldn't resolve it :/

What version of HDP did you try to run the tutorial on? Pl paste the complete error.

Re: When i try to run this tutorial " http://hortonworks.com/hadoop-tutorial/a-lap-around-apache-spark/ " i found this error "Error: Could not find or load main class Dhdp.version=2.3.4.1-10 " but i couldn't resolve it :/

From the error message in the stack trace, it looks like you may have mistyped the spark-submit command line. The main class definition is provided by the --class <main-class> parameter, as shown in this syntax definition:

./bin/spark-submit \
  --class <main-class> \
  --master <master-url> \
  --deploy-mode <deploy-mode> \
  --conf <key>=<value> \
  ... # other options
  <application-jar> \
  [application-arguments]

If you have put the string "Dhdp.version=2.3.4.1-10" on the command-line, then it could lead to the error. The other possibility is that you have entered this string into the "spark-env.sh" file within $SPARK_HOME/conf directory. Double check this file and look for any parameter ending in "OPTS", such as "SPARK_DAEMON_JAVA_OPTS". This could be adding something wrong to the spark-submit arg list, and lead to the error.

Don't have an account?
Coming from Hortonworks? Activate your account here