Support Questions

Find answers, ask questions, and share your expertise

'saveAsOrcFile' error in tutorial lab-4 on Spark

avatar
Explorer

in the sandbox tutorial Lab-4 on Spark, when running this statement:

risk_factor_spark.saveAsOrcFile("risk_factor_spark")

it gives me the following error:

error: value saveAsOrcFile is not a member of org.apache.spark.sql.DataFrame

I have done the 'import org.apache.spark.sql.hive.orc._' earlier in the lab. Also tried capital '..ORC..'. And I am using the built-in web client for the SSH client.

Any suggestions?

1 ACCEPTED SOLUTION

avatar

. @David Andreae If you are using Spark >= 1.4 try the following command

risk_factor_spark.write.format("orc").save("risk_factor_spark")

View solution in original post

3 REPLIES 3

avatar

. @David Andreae If you are using Spark >= 1.4 try the following command

risk_factor_spark.write.format("orc").save("risk_factor_spark")

avatar
Contributor

This helped me - thank you.

avatar

what spark version is this?