04-03-2019 01:39 PM
I have built my standalone spark app and works fine on my standalone environment. When i attempt to run on the yarn cluster, it complains about not being able to find the kudu client jar. I can add the package to the spark-submit arguments, but i wondering why or how to use kudu-client-1.8.0-cdh6.1.1.jar provided with the CDH 6.1.1 distribution, i would've expected it to be there. What would be the official way to link in the provided jar avoid adding the additional package argument to spark-submit ?
any suggestions to a best practice of deploying a vanilla spark/kudu job?