Created on 11-01-2017 01:57 AM - last edited on 11-01-2019 05:45 AM by cjervis
I'm trying to submit a batch job using Livy REST API, my application takes two named arguments but I can't find a way to submit those. I tried this:
curl -X POST -H "Content-Type: application/json" --data '{"className": "com.geoint.spark.utils.filter.FilterAndClean","executorMemory":"4g","numExecutors":3,"executorCores":5,"driverMemory":"2g", "args":["--input /plablo/muestra_variables.csv", "--output /plablo/salida_parquet"],"file": "/user/jars/Geoint-Spark-Utils-assembly-0.1.jar"}' localhost:8999/batches
but I'm getting an error (in Spark history server):
[scallop] Error: Unknown option 'input /plablo/muestra_variables.csv'
So it seams the hyphens are getting lost somewhere.
Is it possible to use named arguments? or should I use only positional arguments?
Thanks
Created 11-28-2017 04:46 PM
Created 12-01-2017 02:42 PM
I will try what you suggest, although I didn't find any reference to that in the links you submitted
Created 10-29-2019 12:09 AM
Hi ,
How did you solve this issue.
I am looking for a solution to pass values for appName, inputfile etc as args