question Re: 'saveAsOrcFile' error in tutorial lab-4 on Spark in Archives of Support Questions (Read Only)

'saveAsOrcFile' error in tutorial lab-4 on Spark

dandreae — Sat, 19 Dec 2015 06:21:50 GMT

in the sandbox tutorial Lab-4 on Spark, when running this statement:

risk_factor_spark.saveAsOrcFile("risk_factor_spark")

it gives me the following error:

error: value saveAsOrcFile is not a member of org.apache.spark.sql.DataFrame

I have done the 'import org.apache.spark.sql.hive.orc._' earlier in the lab. Also tried capital '..ORC..'. And I am using the built-in web client for the SSH client.

Any suggestions?

Re: 'saveAsOrcFile' error in tutorial lab-4 on Spark

jstraub — Mon, 21 Dec 2015 13:37:32 GMT

. @David Andreae If you are using Spark >= 1.4 try the following command

risk_factor_spark.write.format("orc").save("risk_factor_spark")

Re: 'saveAsOrcFile' error in tutorial lab-4 on Spark

jstraub — Mon, 21 Dec 2015 13:41:33 GMT

what spark version is this?

Re: 'saveAsOrcFile' error in tutorial lab-4 on Spark

mvogt — Thu, 28 Jan 2016 00:10:49 GMT

This helped me - thank you.