Created 11-13-2017 05:40 PM
I am using Spark 1.6.
I get the following error when I try to read a json file as per documentation of Spark 1.6
scala> val colors = sqlContext.read.json("C:/Downloads/colors.json"); colors: org.apache.spark.sql.DataFrame = [_corrupt_record: string]
1. The Spark 1.6 works fine and I have been able to read other text files.
2. Attached the json file as text(since upload of json is blocked), I have also validated the json using jsoneditoronline to ensure the json file is well formulated
Created 11-13-2017 05:45 PM
This is usually a problem with multiline json files.
use the following in your json file:
{"colors":[{"color":"black","category":"hue","type":"primary","code":{"rgba":[255,255,255,1],"hex":"#000"}},{"color":"white","category":"value","code":{"rgba":[0,0,0,1],"hex":"#FFF"}},{"color":"red","category":"hue","type":"primary","code":{"rgba":[255,0,0,1],"hex":"#FF0"}},{"color":"blue","category":"hue","type":"primary","code":{"rgba":[0,0,255,1],"hex":"#00F"}},{"color":"yellow","category":"hue","type":"primary","code":{"rgba":[255,255,0,1],"hex":"#FF0"}},{"color":"green","category":"hue","type":"secondary","code":{"rgba":[0,255,0,1],"hex":"#0F0"}}]}
Created 11-13-2017 05:45 PM
This is usually a problem with multiline json files.
use the following in your json file:
{"colors":[{"color":"black","category":"hue","type":"primary","code":{"rgba":[255,255,255,1],"hex":"#000"}},{"color":"white","category":"value","code":{"rgba":[0,0,0,1],"hex":"#FFF"}},{"color":"red","category":"hue","type":"primary","code":{"rgba":[255,0,0,1],"hex":"#FF0"}},{"color":"blue","category":"hue","type":"primary","code":{"rgba":[0,0,255,1],"hex":"#00F"}},{"color":"yellow","category":"hue","type":"primary","code":{"rgba":[255,255,0,1],"hex":"#FF0"}},{"color":"green","category":"hue","type":"secondary","code":{"rgba":[0,255,0,1],"hex":"#0F0"}}]}
Created 11-13-2017 05:53 PM
That helped. Thank you..!!