Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Livy Post Batch with named arguments

avatar
New Contributor

I'm trying to submit a batch job using Livy REST API, my application takes two named arguments but I can't find a way to submit those. I tried this:

curl -X POST -H "Content-Type: application/json" --data '{"className": "com.geoint.spark.utils.filter.FilterAndClean","executorMemory":"4g","numExecutors":3,"executorCores":5,"driverMemory":"2g", "args":["--input /plablo/muestra_variables.csv", "--output /plablo/salida_parquet"],"file": "/user/jars/Geoint-Spark-Utils-assembly-0.1.jar"}' localhost:8999/batches

but I'm getting an error (in Spark history server):

[scallop] Error: Unknown option 'input /plablo/muestra_variables.csv'

So it seams the hyphens are getting lost somewhere.

Is it possible to use named arguments? or should I use only positional arguments?

Thanks

3 REPLIES 3

avatar
New Contributor

I will try what you suggest, although I didn't find any reference to that in the links you submitted

avatar
New Contributor

Hi ,

How did you solve this issue.

 I am looking for a solution to pass values for appName, inputfile etc as args