Support Questions
Find answers, ask questions, and share your expertise

Having CDM but need to manually stop services (without the API).

Explorer

Hello there,

 

I am trying to automatize a production environment, without going through Cloudera Manager. To do so, script to start and stop services are required, and so I tried to base them on such commands for example :

 

for x in `cd /etc/init.d ; ls hadoop-hdfs-*` ; do sudo service $x start ; done

I found it in the documentation : 

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-Installation-Guide/c...

 

Is there any way to get something like :

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/c...

to work with my installation ?

 

I saw there is a another way using API like in:

http://cloudera.github.io/cm_api/apidocs/v1/tutorial.html

But having clear password is out of question for a matter of security we can't use this solution.

 

I am using:

Linux 64 bit - RHEL 5

Cloudera 5.0

Cloudera Manager 5.0

 

 

Thanks a lot,

Regards, K.

--
Lefevre Kevin
5 REPLIES 5

Re: Having CDM but need to manually stop services (without the API).

Expert Contributor
for x in `cd /etc/init.d ; ls hadoop-*` ; do sudo service $x stop ; done
Em Jay

Re: Having CDM but need to manually stop services (without the API).

That doens't work since CM doesn't use the init scripts to run the daemons.

 

There's not really a great way to have people without access to CM but are stil able to start/stop the cluster. CM is supposed to manage your cluster, so going around it doesn't make sense.

 

CM5.1 will have more security roles that may help your use-case, where you can defined limited-access user accounts that can do start and stop but not a lot else. Details will come with that release.

 

You can also use tricks to put your user credentials into a script that is read-restricted and only allows start and stop operations, then use some mechanism to allow people to run that script. You could probably write another script that runs with elevated privileges and runs the start/stop script. I'm not an expert in scripting this kind of stuff, but there's probably some way to do this kind of thing.

Re: Having CDM but need to manually stop services (without the API).

Explorer

Maybe there is a way to create thoses scripts, but it seems rather complicated.

 

I'll try to use the API, and then see how I can secure it. I should encrypt the password in a db, and use https to make request to the CM.

 

I'll give a search about that and post here again, if I need help or to post a solution.

 

Thanks,

K.

--
Lefevre Kevin

Re: Having CDM but need to manually stop services (without the API).

Explorer

I had an idea, when you make a ps -ef |grep java, you get some lines like:

 

/usr/java/jdk1.7.0_45-cloudera/bin/java -Dproc_regionserver -XX:OnOutOfMemoryError=kill -9 %p -Xmx1000m -Djava.net.preferIPv4Stack=true -Xms950009856 -Xmx950009856 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Dhbase.log.dir=/var/log/hbase -Dhbase.log.file=hbase-cmf-hbase-REGIONSERVER-evl2400469.eu.verio.net.log.out -Dhbase.home.dir=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase -Dhbase.id.str= -Dhbase.root.logger=INFO,RFA -Djava.library.path=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop/lib/native -Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.regionserver.HRegionServer start

 I tried to use it directly, but it is not working like that, do you have some ideas ?

--
Lefevre Kevin

Re: Having CDM but need to manually stop services (without the API).

That definitely wouldn't work. The only way to do this is by going through the CM API, as I mentioned before.