When submitting a script to Livy (vs using interactive Scala or Python), is there a way to make 1 call and get back the results in that 1 call?
Typically, script submission such as in submitting a PySpark job is for batch processing and the immediate JSON result set will give information about the session ID. For example, when I submit a PySpark script to Live I get back the following:
I then take the Session ID from the above output and make a second call to get back the log file which contains the output data:
1. Wrap the submission and pulling api by yourself
2. Use zeppelin rest api, zeppelin has already integrated livy interpreter. And zeppelin provides synchronously for running a paragraph. So that you can just call one zeppelin rest api to get the result.