Support Questions
Find answers, ask questions, and share your expertise

Is sharing spark RDD or context a supported in HDP 2.5?

Super Guru

Is sharing spark RDD or context a supported in HDP 2.5 via livy server? Everything I see is via zeppelin interrupter to livy. I want to know if strictly using spark is sharing spark (not zeppelin) RDD or context supported.

13 REPLIES 13

Livy is not "hidden". If you have started the Livy server, you can interact with its REST API from any application.

@Randy Gelhausen

Yes. With HDP 2.5. I missed that memo. Before that was "Installed as Spark Ambari Service, not yet exposed outside of Zeppelin". The following was needed: https://github.com/hortonworks-gallery/ambari-zeppelin-service

Super Guru

Just to close this out as i was able to get info required. Sharing RDD and context outside of zeppelin is not supported.

@Sunile Manjee

As I mentioned in my original response: Not without Livy. Yes with Livy (@vshukla). However, it is exposed only to Zeppelin, for now.