Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How can I change spark package repository?

How can I change spark package repository?

New Contributor

15393-spark.png

Using spark-shell with --packages options like databricks,

Of course, spark is downloading package library on the maven repository of internet.

But, in case of offline mode, it not useful.

How can I change or add spark package repository?

4 REPLIES 4

Re: How can I change spark package repository?

New Contributor

Hm, I just fixed this issue with multiple jars option.

Re: How can I change spark package repository?

Did you tried the multiple jars options?

Re: How can I change spark package repository?

New Contributor

Yes I did.

Re: How can I change spark package repository?

New Contributor

In my case, I navigate to the folder /data/user/flamingo/.ivy2/jars

...
Ivy Default Cache set to: /data/user/flamingo/.ivy2/cache
The jars for the packages stored in: /data/user/flamingo/.ivy2/jars
...

And copy all the jars below to the directory you want to store jars, then execute the spark command like:

SPARK_MAJOR_VERSION=2 bin/spark-shell --jars="/path/to/jars"

Then the result seems worked!