question Re: How does the Livy server for Zeppelin communicate with the Spark server? in Archives of Support Questions (Read Only)

How does the Livy server for Zeppelin communicate with the Spark server?

michaelgregory1 — Thu, 08 Sep 2016 17:17:33 GMT

Does it act like a Spark client and communicate over the normal Spark port numbers or is it all REST?

Which ports need to be opened in the firewall in the following flow?

Zeppelin <-> Livy Client <-> Livy Server <-> |-Firewall-| <-> Spark Cluster

Re: How does the Livy server for Zeppelin communicate with the Spark server?

mqureshi — Thu, 08 Sep 2016 23:37:03 GMT

Livy is a Rest server that acts as a Spark client. So you need to open nothing more than normal spark ports, listed here.

I would very highly recommend going over slide 14-31 on this link. These are just images so it should be easy and simple to go over this but it will give you a better understanding of Livy and will give you confidence in what you are trying to do.

Re: How does the Livy server for Zeppelin communicate with the Spark server?

lgeorge — Fri, 09 Sep 2016 01:55:58 GMT

There's a nice overview video from the June Hadoop Summit at

https://www.youtube.com/watch?v=gtql7z_kj3Y&list=PLKnYDs_-dq16K1NH83Bke2dGGUO3YKZ5b&index=114

Re: How does the Livy server for Zeppelin communicate with the Spark server?

lgeorge — Fri, 09 Sep 2016 01:58:22 GMT

& corresponding slides, especially slides 15-27:

http://www.slideshare.net/HadoopSummit/apache-zeppelin-livy-bringing-multi-tenancy-to-interactive-data-analysis-63887800