Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created on
10-18-2019
12:33 AM
- last edited on
10-18-2019
03:36 AM
by
VidyaSargur
Hi, guys
My master's /usr/bin was deleted, and i have copy other slave's /usr/bin to the master.
but when i type spark-shell, it occur this error:
spark-shell error
but when i type /opt/cloudera/parcels/CDH/bin/spark-shell, it works fine.
Seems the commands in /user/bin directory should be soft link to/ opt/cloudera/parcels/CDH , is there any way to recover it?
thanks a lot.
Created on 10-18-2019 01:09 AM - edited 10-18-2019 01:18 AM
Hi ,
I would suggest to do the following:-
1. Login to slave's /usr/bin.
2. Type the following command to check to which utility the spark-shell is linked.
ls -al spark*
It will print the following depending on your configuration.
lrwxrwxrwx 1 root root 29 Oct 18 01:03 spark-shell -> /etc/alternatives/spark-shell
lrwxrwxrwx 1 root root 30 Sep 30 03:11 spark-submit -> /etc/alternatives/spark-submit
lrwxrwxrwx 1 root root 30 Sep 30 03:11 spark2-shell -> /etc/alternatives/spark2-shell
lrwxrwxrwx 1 root root 31 Sep 30 03:11 spark2-submit -> /etc/alternatives/spark2-submit
3. Login to master shell /usr/bin:
ln -s /etc/alternatives/spark-shell spark-shell
The above command will create a link to the executable. Change the path "/etc/alternatives/spark-shell" as per your slave's configuration
Please remember to login as root user.
Let me know if it works.
Thanks,
Ganesh
Created on 10-18-2019 01:09 AM - edited 10-18-2019 01:18 AM
Hi ,
I would suggest to do the following:-
1. Login to slave's /usr/bin.
2. Type the following command to check to which utility the spark-shell is linked.
ls -al spark*
It will print the following depending on your configuration.
lrwxrwxrwx 1 root root 29 Oct 18 01:03 spark-shell -> /etc/alternatives/spark-shell
lrwxrwxrwx 1 root root 30 Sep 30 03:11 spark-submit -> /etc/alternatives/spark-submit
lrwxrwxrwx 1 root root 30 Sep 30 03:11 spark2-shell -> /etc/alternatives/spark2-shell
lrwxrwxrwx 1 root root 31 Sep 30 03:11 spark2-submit -> /etc/alternatives/spark2-submit
3. Login to master shell /usr/bin:
ln -s /etc/alternatives/spark-shell spark-shell
The above command will create a link to the executable. Change the path "/etc/alternatives/spark-shell" as per your slave's configuration
Please remember to login as root user.
Let me know if it works.
Thanks,
Ganesh
Created 10-18-2019 01:50 AM
Thanks @CLDR
Your solution works!
I have try the mehod:
1. tar the slave's /usr/bin which must contain the soft link.
2. copy tar.gz to the master, and it works fine!
Thanks a lot!