Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Cloudera manger , HBase Deploy Client Configuration fails

avatar
Expert Contributor

i am trying to deploy client configuration on hbase , and it always fails , i am having 1 master , and 2 workers  , and here are the logs:

 

 

Generate client configuration.
Successfully generated client configuration.

 


Run 3 steps in parallel
Completed 2/3 steps successfully

 

 

 

Cleanup client configs that failed to deploy.
Successfully cleaned up client configs that failed to deploy.

 

Can't open /var/run/cloudera-scm-agent/process/740-deploy-client-config/hbase-conf/hive-env.sh: No such file or directory.
+ '[' '!' -e /etc/hbase ']'
+ cd /etc/hbase
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/taskcontroller.cfg ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/container-executor.cfg ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/topology.py ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/topology.map ']'
+ '[' -d conf -a '!' -L conf ']'
+ '[' -n '' ']'
+ rm -rf conf.cloudera.hbase
+ cp -a /var/run/cloudera-scm-agent/process/740-deploy-client-config/hbase-conf conf.cloudera.hbase
+ chmod -R ugo+r conf.cloudera.hbase
+ [[ HBASE == HIVE ]]
+ /usr/sbin/update-alternatives --install /etc/hbase/conf hbase-conf /etc/hbase/conf.cloudera.hbase 90
+ /usr/sbin/update-alternatives --auto hbase-conf

2nd log :

 

Can't open /var/run/cloudera-scm-agent/process/739-deploy-client-config/hbase-conf/hive-env.sh: No such file or directory.
+ '[' '!' -e /etc/hbase ']'
+ cd /etc/hbase
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/taskcontroller.cfg ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/container-executor.cfg ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/topology.py ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/topology.map ']'
+ '[' -d conf -a '!' -L conf ']'
+ '[' -n '' ']'
+ rm -rf conf.cloudera.hbase
+ cp -a /var/run/cloudera-scm-agent/process/739-deploy-client-config/hbase-conf conf.cloudera.hbase
+ chmod -R ugo+r conf.cloudera.hbase
+ [[ HBASE == HIVE ]]
+ /usr/sbin/update-alternatives --install /etc/hbase/conf hbase-conf /etc/hbase/conf.cloudera.hbase 90
/var/lib/alternatives/hbase-conf empty!

 

3rd log :

 

Can't open /var/run/cloudera-scm-agent/process/738-deploy-client-config/hbase-conf/hive-env.sh: No such file or directory.
+ '[' '!' -e /etc/hbase ']'
+ cd /etc/hbase
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/taskcontroller.cfg ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/container-executor.cfg ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/topology.py ']'
+ for SPECIAL_FILE in '$DEST_PATH/{taskcontroller.cfg,container-executor.cfg,topology.py,topology.map}'
+ '[' -e /etc/hbase/conf.cloudera.hbase/topology.map ']'
+ '[' -d conf -a '!' -L conf ']'
+ '[' -n '' ']'
+ rm -rf conf.cloudera.hbase
+ cp -a /var/run/cloudera-scm-agent/process/738-deploy-client-config/hbase-conf conf.cloudera.hbase
+ chmod -R ugo+r conf.cloudera.hbase
+ [[ HBASE == HIVE ]]
+ /usr/sbin/update-alternatives --install /etc/hbase/conf hbase-conf /etc/hbase/conf.cloudera.hbase 90
+ /usr/sbin/update-alternatives --auto hbase-conf

 

so the problem is that hive-env.sh is not created , also having same problem on deploying client configuration for spark ,

 

and in the log it shows same error "hive-env.sh: No such file or directory."  

 ,,Please help me with this issue 

 

 

 

 

1 ACCEPTED SOLUTION

avatar
Master Guru

This issue can occur if there are problems with alternatives.

 

Try this on your nodes:

 

/usr/sbin/alternatives --display hadoop-conf

 

If that command does not return valid alternatives, you may be missing the following file:

 

/var/lib/alternatives/hadoop-conf

 

If you can find a host with the file, copy it from that host to the host or hosts where it is missing.

 

Regards,

 

Ben

View solution in original post

10 REPLIES 10

avatar
Expert Contributor

i also noticed something really weird , in path : /var/run/cloudera-scm-agent/process  ,  i found  739-deploy-client-config    directory only and ofcourse hive-env.sh wasn't there , and didn't found folders 738-deploy-client-config   nor 740-deploy-client-config 

avatar
Explorer

Hi

 

I am trying CDH automatic installation on AWs EC2 using cloudera manager bin. I have created one ubuntu Precise 12.04 LTS micro instance,

 

I followed the on screen instructions as instructions on per this tutorial.. " http://www.cloudera.com/content/cloudera/en/documentation/cloudera-manager/v4-7-1/Cloudera-Manager-I...

 

1) this is my vi /etc/hosts file

127.0.0.1 localhost
172.31.13.46 master

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

 

2) Downloaded and changed the permission for cloudera-manager-installer.bin also.

 

3) sudo ./cloudera-manager-installer.bin after this command cloudera manager installed,

 

4) But i couldnot access cloudera manager webconsole using " http://172.31.13.46:7180 " and i have opened my port 7180 while creating an instance, but still not able to acess through webconsole, 

 

5) my cloudera manager db and cloudera manager server both are running.

 

6) and the port 7180 is also not listning in my ubuntu server and i used the following comand, " sudo ufs allow 7180"  but no use,,

 

7) I checked $ sudo ufw status and the result is inactive

 

😎 when I check $ sudo service cloudera-scm-agent status on 172.31.13.46 it comes as unrecognized service

 

 

I am struggling in this part, Could you please let me know where I went wrong in installing cloudera in a clustered environment..???

 

if yes, it will be helpful for me, please,

 

Thanks in advance,

 

Regards,

Bharath

avatar
Explorer
Hi a newbie here, so did you ever solve this problem? I'm currently having the same issue. Thank you.

avatar
New Contributor

I also have the same problem as OP. But I haven't found any additional info from online or other areas of this forum (except repeated, unanswered questions)..

 

 

When I deploy the config scripts to the hosts on my cluster via Cloudera Manager (Home -> pull down by cluster name -> Deploy Client Configuration), I get a failure in

 

/var/run/cloudera-scm-agent/process/ccdeploy_hadoop-conf_etchadoopconf.cloudera.hdfs_XXXX/logs/stderr.log

/var/run/cloudera-scm-agent/process/ccdeploy_hadoop-conf_etchadoopconf.cloudera.hdfs_XXXX/logs/stdout.log

 

stating:

 

 

Can't open /var/run/cloudera-scm-agent/process/ccdeploy_hadoop-conf_etchadoopconf.cloudera.hdfs_XXXX/hadoop-conf/hive-env.sh: No such file or directory.

I am running Cloudera Enterprise 5.4.1

 

avatar
New Contributor

I am also getting same eror on cdh 5.7 

+ perl -pi -e 's#{{HIVE_HBASE_JAR}}#/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hive/lib/hive-hbase-handler-1.1.0-cdh5.7.0.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/hbase-hadoop2-compat.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/lib/htrace-core4-4.0.1-incubating.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/lib/htrace-core-3.2.0-incubating.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/lib/htrace-core.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/hbase-hadoop-compat.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/hbase-client.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/hbase-server.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/hbase-protocol.jar,/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hbase/hbase-common.jar#g' /run/cloudera-scm-agent/process/ccdeploy_hadoop-conf_etchadoopconf.cloudera.hdfs_-9016664614313200359/hadoop-conf/hive-env.sh
Can't open /run/cloudera-scm-agent/process/ccdeploy_hadoop-conf_etchadoopconf.cloudera.hdfs_-9016664614313200359/hadoop-conf/hive-env.sh: No such file or directory.

avatar
Champion Alumni

I have the same here, after un update from CDH 5.3...

GHERMAN Alina

avatar
Master Guru

This issue can occur if there are problems with alternatives.

 

Try this on your nodes:

 

/usr/sbin/alternatives --display hadoop-conf

 

If that command does not return valid alternatives, you may be missing the following file:

 

/var/lib/alternatives/hadoop-conf

 

If you can find a host with the file, copy it from that host to the host or hosts where it is missing.

 

Regards,

 

Ben

avatar
Contributor
Thanks Ben,

This helped resolved my issue.

In my case , all the below files were empty on my cloudera manager host. Copied the contents from secondary node that resolved the issue
/var/lib/alternatives/hadoop-conf
/var/lib/alternatives/habase-conf
/var/lib/alternatives/hive-conf
/var/lib/alternatives/spark-conf
/var/lib/alternatives/spark2-conf
/var/lib/alternatives/solr-conf

avatar
New Contributor
This helped me too. Thanks