Created 01-09-2019 11:17 PM
How can I add multiple jars to --driver-class-path. It works when I add 1 jar file, but does not work when I try to add multiple jars using a comma separated format.
Here is the shell command I am using. It does not pick up the second jar file.
pyspark2 --driver-class-path /home/jar/noarch-aster-jdbc-driver.jar,/home/jar/sqljdbc42.jar --jars /home/jar/noarch-aster-jdbc-driver.jar,/home/jar/sqljdbc42.jar
??
Thanks.
Created 01-10-2019 08:10 AM
@A C, separator for --driver-class-path will be ":" instead of ",". Do update the command and rerun the pyspark shell.
let know if this helps.
Created 01-10-2019 09:34 AM
Have you tried with colons?
pyspark2 --driver-class-path /home/jar/noarch-aster-jdbc-driver.jar:/home/jar/sqljdbc42.jar --jars /home/jar/noarch-aster-jdbc-driver.jar,/home/jar/sqljdbc42.jar
Created 01-10-2019 08:51 PM
I don't know if this has anything to do with different pyspark version. I have version 2.3.0. Finally, this format worked for me.
pyspark2 --driver-class-path "/home/jar/*" --jars /home/jar/noarch-aster-jdbc-driver.jar,/home/jar/sqljdbc42.jar