Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

provide pyspark3 via livy

avatar
Explorer

As said in the following post https://hortonworks.com/blog/recent-improvements-apache-zeppelin-livy-integration/

User can use livy.pyspark3 for python3 with spark. Can anyone help me which all configuration need to be set for the same

1 ACCEPTED SOLUTION

avatar
Super Guru

@btandel @Saurabh,

Did you try setting "livy.spark.yarn.appMasterEnv.PYSPARK3_PYTHON" in the livy interperter setting.

Use %livy.python3 in your paragraph.

Thanks,

Aditya

View solution in original post

2 REPLIES 2

avatar
Guru

Can someone tell what configuration need to be change to use pyspark3 ?

avatar
Super Guru

@btandel @Saurabh,

Did you try setting "livy.spark.yarn.appMasterEnv.PYSPARK3_PYTHON" in the livy interperter setting.

Use %livy.python3 in your paragraph.

Thanks,

Aditya