Created on 03-24-2016 10:30 AM - edited 08-19-2019 01:12 AM
This is the file Java Opts & i work with HDP 220.127.116.11-169
Created 03-24-2016 11:02 AM
Did you try replacing the version with yours?
Created 03-24-2016 06:05 PM
What version of HDP did you try to run the tutorial on? Pl paste the complete error.
Created 03-24-2016 08:26 PM
From the error message in the stack trace, it looks like you may have mistyped the spark-submit command line. The main class definition is provided by the --class <main-class> parameter, as shown in this syntax definition:
./bin/spark-submit \ --class <main-class> \ --master <master-url> \ --deploy-mode <deploy-mode> \ --conf <key>=<value> \ ... # other options <application-jar> \ [application-arguments]
If you have put the string "Dhdp.version=18.104.22.168-10" on the command-line, then it could lead to the error. The other possibility is that you have entered this string into the "spark-env.sh" file within $SPARK_HOME/conf directory. Double check this file and look for any parameter ending in "OPTS", such as "SPARK_DAEMON_JAVA_OPTS". This could be adding something wrong to the spark-submit arg list, and lead to the error.