Support Questions

Find answers, ask questions, and share your expertise

Livy Post Batch with named arguments

avatar
New Contributor

I'm trying to submit a batch job using Livy REST API, my application takes two named arguments but I can't find a way to submit those. I tried this:

curl -X POST -H "Content-Type: application/json" --data '{"className": "com.geoint.spark.utils.filter.FilterAndClean","executorMemory":"4g","numExecutors":3,"executorCores":5,"driverMemory":"2g", "args":["--input /plablo/muestra_variables.csv", "--output /plablo/salida_parquet"],"file": "/user/jars/Geoint-Spark-Utils-assembly-0.1.jar"}' localhost:8999/batches

but I'm getting an error (in Spark history server):

[scallop] Error: Unknown option 'input /plablo/muestra_variables.csv'

So it seams the hyphens are getting lost somewhere.

Is it possible to use named arguments? or should I use only positional arguments?

Thanks

3 REPLIES 3

avatar
New Contributor

I will try what you suggest, although I didn't find any reference to that in the links you submitted

avatar
New Contributor

Hi ,

How did you solve this issue.

 I am looking for a solution to pass values for appName, inputfile etc as args