Support Questions

Find answers, ask questions, and share your expertise

How can I change spark package repository?

avatar
Rising Star

15393-spark.png

Using spark-shell with --packages options like databricks,

Of course, spark is downloading package library on the maven repository of internet.

But, in case of offline mode, it not useful.

How can I change or add spark package repository?

4 REPLIES 4

avatar
Rising Star

Hm, I just fixed this issue with multiple jars option.

avatar

Did you tried the multiple jars options?

avatar
Rising Star

Yes I did.

avatar
Explorer

In my case, I navigate to the folder /data/user/flamingo/.ivy2/jars

...
Ivy Default Cache set to: /data/user/flamingo/.ivy2/cache
The jars for the packages stored in: /data/user/flamingo/.ivy2/jars
...

And copy all the jars below to the directory you want to store jars, then execute the spark command like:

SPARK_MAJOR_VERSION=2 bin/spark-shell --jars="/path/to/jars"

Then the result seems worked!