Created on 10-18-2019 12:33 AM - last edited on 10-18-2019 03:36 AM by VidyaSargur
Hi, guys
My master's /usr/bin was deleted, and i have copy other slave's /usr/bin to the master.
but when i type spark-shell, it occur this error:
but when i type /opt/cloudera/parcels/CDH/bin/spark-shell, it works fine.
Seems the commands in /user/bin directory should be soft link to/ opt/cloudera/parcels/CDH , is there any way to recover it?
thanks a lot.
Created on 10-18-2019 01:09 AM - edited 10-18-2019 01:18 AM
Hi ,
I would suggest to do the following:-
1. Login to slave's /usr/bin.
2. Type the following command to check to which utility the spark-shell is linked.
ls -al spark*
It will print the following depending on your configuration.
lrwxrwxrwx 1 root root 29 Oct 18 01:03 spark-shell -> /etc/alternatives/spark-shell
lrwxrwxrwx 1 root root 30 Sep 30 03:11 spark-submit -> /etc/alternatives/spark-submit
lrwxrwxrwx 1 root root 30 Sep 30 03:11 spark2-shell -> /etc/alternatives/spark2-shell
lrwxrwxrwx 1 root root 31 Sep 30 03:11 spark2-submit -> /etc/alternatives/spark2-submit
3. Login to master shell /usr/bin:
ln -s /etc/alternatives/spark-shell spark-shell
The above command will create a link to the executable. Change the path "/etc/alternatives/spark-shell" as per your slave's configuration
Please remember to login as root user.
Let me know if it works.
Thanks,
Ganesh
Created on 10-18-2019 01:09 AM - edited 10-18-2019 01:18 AM
Hi ,
I would suggest to do the following:-
1. Login to slave's /usr/bin.
2. Type the following command to check to which utility the spark-shell is linked.
ls -al spark*
It will print the following depending on your configuration.
lrwxrwxrwx 1 root root 29 Oct 18 01:03 spark-shell -> /etc/alternatives/spark-shell
lrwxrwxrwx 1 root root 30 Sep 30 03:11 spark-submit -> /etc/alternatives/spark-submit
lrwxrwxrwx 1 root root 30 Sep 30 03:11 spark2-shell -> /etc/alternatives/spark2-shell
lrwxrwxrwx 1 root root 31 Sep 30 03:11 spark2-submit -> /etc/alternatives/spark2-submit
3. Login to master shell /usr/bin:
ln -s /etc/alternatives/spark-shell spark-shell
The above command will create a link to the executable. Change the path "/etc/alternatives/spark-shell" as per your slave's configuration
Please remember to login as root user.
Let me know if it works.
Thanks,
Ganesh
Created 10-18-2019 01:50 AM
Thanks @CLDR
Your solution works!
I have try the mehod:
1. tar the slave's /usr/bin which must contain the soft link.
2. copy tar.gz to the master, and it works fine!
Thanks a lot!