Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

column name with " . " spark sql

column name with " . " spark sql

Explorer

hi guys am using dataset with column name containing "." , here is the code am using and after that the issue i got , i think there is a problem with the "." character ,what should i do ? Thanks.

13266-capture.png

val spark = SparkSession.builder.master("local").appName("my-spark-app").getOrCreate() val df = spark.read.option("inferSchema", "true").option("header", "true").csv("D:\ProcessDataSet\anis_data\Set _1Mud Pumps_Merged.csv") val aggs = df.columns.map(c => stddev(c).as(c)) val stddevs = df.select(aggs: _*) stddevs.show(5,false)
1 REPLY 1
Highlighted

Re: column name with " . " spark sql

Expert Contributor

Yes. It seems like that. Could you print `df.printSchema()` for more information?

Don't have an account?
Coming from Hortonworks? Activate your account here