Member since
09-14-2017
2
Posts
0
Kudos Received
0
Solutions
12-01-2017
02:42 PM
I will try what you suggest, although I didn't find any reference to that in the links you submitted
... View more
11-01-2017
01:57 AM
I'm trying to submit a batch job using Livy REST API, my application takes two named arguments but I can't find a way to submit those. I tried this:
curl -X POST -H "Content-Type: application/json" --data '{"className": "com.geoint.spark.utils.filter.FilterAndClean","executorMemory":"4g","numExecutors":3,"executorCores":5,"driverMemory":"2g", "args":["--input /plablo/muestra_variables.csv", "--output /plablo/salida_parquet"],"file": "/user/jars/Geoint-Spark-Utils-assembly-0.1.jar"}' localhost:8999/batches
but I'm getting an error (in Spark history server):
[scallop] Error: Unknown option 'input /plablo/muestra_variables.csv'
So it seams the hyphens are getting lost somewhere.
Is it possible to use named arguments? or should I use only positional arguments?
Thanks
... View more
Labels:
- Labels:
-
Apache Spark