Sample code to automate interacting with Zeppelin Interpreter APIs

[#1]

Zeppelin interpreters allow Zeppelin to submit requests to multiple languages or data processing engines (like Scala, PySpark, Shell, Hive, HBase etc). More details available at https://zeppelin.incubator.apache.org/docs/0.5.6-incubating/manual/interpreters.html

However, before you can use interpreters, in most cases you need to configure them (e.g. point hive interpreter to server/port where HiveServer2 is running on your cluster). You can do this manually via the Zeppelin UI by clicking on the Interpreter tab.

However, in some cases you will want to automate/script out this step. Below is some sample code on how to do this. To run, on the host where Zeppelin is running, copy/paste the code into a file called zeppelin.py using your favourite editor, and then run it by typing:

python zeppelin.py

Sample code:

#!/usr/local/bin/python
def post_request(url, body):
  import json, urllib2
  encoded_body = json.dumps(body)
  req = urllib2.Request(str(url), encoded_body)
  req.get_method = lambda: 'PUT'
  try:
    response = urllib2.urlopen(req, encoded_body).read()
  except urllib2.HTTPError, error:
    print 'Exception: ' + error.read()
  jsonresp = json.loads(response.decode('utf-8'))
  print jsonresp['status']
        


import json, urllib2
zeppelin_int_url = 'http://localhost:9995/api/interpreter/setting/'
data = json.load(urllib2.urlopen(zeppelin_int_url))
for body in data['body']:
  if body['group'] == 'psql':
    psqlbody = body
  elif body['group'] == 'hive':
    hivebody = body    
    
hivebody['properties']['hive.hiveserver2.url'] = 'jdbc:hive2://localhost:10000'
post_request(zeppelin_int_url + hivebody['id'], hivebody)


psqlbody['properties']['postgresql.user'] = 'gpadmin'
psqlbody['properties']['postgresql.password'] = 'gpadmin'
psqlbody['properties']['postgresql.url'] = 'jdbc:postgresql://localhost:10432/postgres'
post_request(zeppelin_int_url + psqlbody['id'], psqlbody)

What does the code do?

1. First it is connecting to Zeppelin REST API on port 9995 and fetching the configs for hive and psql interpreters
2. Then it is updating hive interpreter configs (by setting the HS2 url to jdbc:hive2://localhost:10000) and posting the updated config back to Zeppelin
3. Finally it is updating psql interpreter configs (by setting the username/pass and udpating psql url to

   jdbc:postgresql://localhost:10432/postgres) and posting the updated config back to Zeppelin

Sample output is shown below. For each request made, it will output OK if the POST request succeeded.

# python zeppelin.py
OK
OK

Also if your urls/credentials are different from above, feel free to modify the python script and re-run the script (you can re-run the script as many times as you like)

Now if you open your Zeppelin interpreter tab, you will notice that the Hive and psql interpreters have been updated with above configs.