Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

unhandled Null pointer exception while transforming

Solved Go to solution

unhandled Null pointer exception while transforming

Rising Star

While running over tutorial https://community.hortonworks.com/articles/53903/spark-machine-learning-pipeline-by-example.html, I face issue in below line

val header = flight2007.first

val trainingData = flight2007 .filter(x => x != header) unhandled exception while transforming <console> error: uncaught exception during compilation: java.lang.NullPointerException

---------------------

Error message in detail after successful display of the header val.

while compiling: <console> during phase: specialize library version: version 2.10.5 compiler version: version 2.10.5 reconstructed args: -classpath /usr/hdp/2.4.0.0-169/zeppelin/lib/interpreter/spark/zeppelin-spark-0.6.0.2.4.0.0-169.jar:/etc/spark/2.4.0.0-169/0:/usr/hdp/2.4.0.0-169/spark/lib/spark-assembly-1.6.0.2.4.0.0-169-hadoop2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/spark/lib/datanucleus-api-jdo-3.2.6.jar:/usr/hdp/2.4.0.0-169/spark/lib/datanucleus-core-3.2.10.jar:/usr/hdp/2.4.0.0-169/spark/lib/datanucleus-rdbms-3.2.9.jar:/etc/hadoop/2.4.0.0-169/0:/usr/hdp/current/zeppelin-server/lib/interpreter/spark/zeppelin-spark-0.6.0.2.4.0.0-169.jar:/usr/hdp/current/spark-historyserver/conf:/usr/hdp/2.4.0.0-169/spark/lib/spark-assembly-1.6.0.2.4.0.0-169-hadoop2.7.1.2.4.0.0-169.jar:/usr/hdp/2.4.0.0-169/spark/lib/datanucleus-api-jdo-3.2.6.jar:/usr/hdp/2.4.0.0-169/spark/lib/datanucleus-core-3.2.10.jar:/usr/hdp/2.4.0.0-169/spark/lib/datanucleus-rdbms-3.2.9.jar:/usr/hdp/current/hadoop-client/conf:/usr/hdp/current/zeppelin-server/lib/interpreter/spark/zeppelin-spark-0.6.0.2.4.0.0-169.jar last tree to typer: TypeTree(anonymous class $anonfun) symbol: anonymous class $anonfun (flags: final <synthetic>) symbol definition: final class $anonfun extends AbstractFunction1[Array[String],Flight] with Serializable tpe: scala.runtime.AbstractFunction1[Array[String],Flight] with Serializable symbol owners: anonymous class $anonfun -> value trainingData -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $read -> package $line568 context owners: anonymous class $anonfun -> value trainingData -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $iwC -> class $read -> package $line568 == Enclosing template or block == ClassDef( // final class $anonfun extends AbstractFunction1[String,Boolean] with Serializable final <synthetic> @{ SerialVersionUID(0) } "$anonfun" [] Template( // val <local $anonfun>: <notype>, tree.tpe=scala.runtime.AbstractFunction1[String,Boolean] with Serializable "scala.runtime.AbstractFunction1", "scala.Serializable" // parents ValDef( private "_" <tpt> <empty> ) // 2 statements DefDef( // def <init>(): scala.runtime.AbstractFunction1[String,Boolean] with Serializable <method> <triedcooking> "<init>" [] List(Nil) <tpt> // tree.tpe=scala.runtime.AbstractFunction1[String,Boolean] with Serializable Block( // tree.tpe=Unit Apply( // def <init>(): scala.runtime.AbstractFunction1[T1,R] in class AbstractFunction1, tree.tpe=scala.runtime.AbstractFunction1[String,Boolean] $anonfun.super."<init>" // def <init>(): scala.runtime.AbstractFunction1[T1,R] in class AbstractFunction1, tree.tpe=()scala.runtime.AbstractFunction1[String,Boolean] Nil ) () ) ) DefDef( // final def apply(x: String): Boolean <method> final "apply" [] // 1 parameter list ValDef( // x: String <param> <triedcooking> "x" <tpt> // tree.tpe=String <empty> ) <tpt> // tree.tpe=Boolean Apply( // final def !=(x$1: Object): Boolean in class Object, tree.tpe=Boolean "x"."$bang$eq" // final def !=(x$1: Object): Boolean in class Object, tree.tpe=(x$1: Object)Boolean Apply( // val header(): String, tree.tpe=String $iwC.this.$VAL1317().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw().$iw()."header" // val header(): String, tree.tpe=()String Nil ) ) ) ) ) == Expanded type of tree == TypeRef( TypeSymbol( final class $anonfun extends AbstractFunction1[Array[String],Flight] with Serializable ) ) unhandled exception while transforming <console> error: uncaught exception during compilation: java.lang.NullPointerException

----------------------------------------------

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: unhandled Null pointer exception while transforming

Rising Star

Closed issue with data error.

View solution in original post

1 REPLY 1
Highlighted

Re: unhandled Null pointer exception while transforming

Rising Star

Closed issue with data error.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here