SparkSQL - org.apache.spark.sql.catalyst.types.StructField fails

vaskoyordan — Fri, 16 Sep 2022 09:23:51 GMT

I am trying to use certain functionality from SparkSQL ( namely “programmatically specifying a schema” as described in the Spark 1.1.0 documentation)

I am getting the following error:

15/03/10 17:00:16 INFO storage.BlockManagerMaster: Updated info of block broadcast_2_piece0

15/03/10 17:00:16 INFO spark.SparkContext: Created broadcast 2 from broadcast at MSP.scala:52

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.types.StructField.<init>(Ljava/lang/String;Lorg/apache/spark/sql/catalyst/types/DataType;Z)

at MSP$$anonfun$3.apply(MSP.scala:57)

The code is:

val schemaString="ATR1,ATTR2,....”

line 57:

val schema =

StructType(

schemaString.split(",").map(fieldName => StructField(fieldName, StringType, true)))

Why this fails ? I am using Spark 1.2

Re: SparkSQL - org.apache.spark.sql.catalyst.types.StructField fails

srowen — Wed, 11 Mar 2015 18:46:44 GMT

In general, NoSuchMethodError in Java means you compiled against one version of something, and executed against a different version. Check your build.

question Re: SparkSQL - org.apache.spark.sql.catalyst.types.StructField fails in Archives of Support Questions (Read Only)

SparkSQL - org.apache.spark.sql.catalyst.types.StructField fails

Re: SparkSQL - org.apache.spark.sql.catalyst.types.StructField fails