Reply
Explorer
Posts: 14
Registered: ‎03-20-2016
Accepted Solution

cloudera manager will not start - fresh install

[ Edited ]

Hello,

 

I have completed installation of Cloudera Enterprise 5.6.x on Red Hat Enterprise Linux Server release 6.7 in steps described as "installation path b" without errors.

 

http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_install_path_b.html#cmig_topic_...

 

Start of cloudera manager immediately fails - cloudera-scm-server.out only contains a few lines..

 

cd /var/log/cloudera-scm-server/
[hduser@slrs01va6f1clm1 cloudera-scm-server]$ cat *.out


JAVA_HOME=/usr/java/jdk1.8.0_72
/usr/share/cmf/bin/cmf-server: line 129: /usr/java/jdk1.8.0_72/bin/java: Permission denied
/usr/share/cmf/bin/cmf-server: line 129: exec: /usr/java/jdk1.8.0_72/bin/java: cannot execute: Permission denied

 

Java JDK was installed as described here:

 

http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_jdk_installation.html?scroll=t...

 

Java environment for "hduser"..

 

echo $JAVA_HOME
/usr/java/jdk1.8.0_72


[hduser@slrs01va6f1clm1 cloudera-scm-server]$ java -version
java version "1.8.0_72"


Java(TM) SE Runtime Environment (build 1.8.0_72-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode)

 

I would appreciate any suggestions on how to proceed.

 

Thank you

Highlighted
Posts: 1,042
Topics: 1
Kudos: 262
Solutions: 130
Registered: ‎04-22-2014

Re: cloudera manager will not start - fresh install

It appears you may not be starting Cloudera Manager when root or sudo. This is required and is likely the reason you are not allowed permission to java.

 

Try "service cloudera-scm-server start" as root or with sudo.

Explorer
Posts: 14
Registered: ‎03-20-2016

Re: cloudera manager will not start - fresh install

 

Sorry I did not paste startup command - I'm definitely using sudo..


[hduser@slrs01va6f1clm1 ~]$ sudo service cloudera-scm-server start
[sudo] password for hduser:
Starting cloudera-scm-server: [FAILED]


[hduser@slrs01va6f1clm1 ~]$ cd /var/log/cloudera-scm-server/

 

[hduser@slrs01va6f1clm1 cloudera-scm-server]$ cat cloudera-scm-server.out


JAVA_HOME=/usr/java/jdk1.8.0_72
/usr/share/cmf/bin/cmf-server: line 129: /usr/java/jdk1.8.0_72/bin/java: Permission denied
/usr/share/cmf/bin/cmf-server: line 129: exec: /usr/java/jdk1.8.0_72/bin/java: cannot execute: Permission denied

Cloudera Employee
Posts: 289
Registered: ‎07-08-2013

Re: cloudera manager will not start - fresh install

can provide the output for following command $ stat /usr/java/jdk1.8.0_72/bin/java 

Explorer
Posts: 14
Registered: ‎03-20-2016

Re: cloudera manager will not start - fresh install

 

[hduser@slrs01va6f1clm1 ~]$ stat /usr/java/jdk1.8.0_72/bin/java


File: `/usr/java/jdk1.8.0_72/bin/java'
Size: 7734 Blocks: 16 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 419958 Links: 1
Access: (0776/-rwxrwxrw-) Uid: ( 504/ hduser) Gid: ( 504/ hadoop)
Access: 2016-03-24 18:08:14.042587642 -0500
Modify: 2015-12-23 00:10:09.000000000 -0600
Change: 2016-03-23 16:30:45.733564454 -0500

Explorer
Posts: 14
Registered: ‎03-20-2016

Re: cloudera manager will not start - fresh install

strace output..

 

[hduser@slrs01va6f1clm1 cloudera-scm-server]$ strace sudo service cloudera-scm-server start
execve("/usr/bin/sudo", ["sudo", "service", "cloudera-scm-server", "start"], [/* 25 vars */]) = 0
brk(0) = 0x7fb6a44d9000
fcntl(0, F_GETFD) = 0
fcntl(1, F_GETFD) = 0
fcntl(2, F_GETFD) = 0
access(0x7fb6a29af58c, F_OK) = -1 ENOENT (No such file or directory)
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb6a2bb2000
access(0x7fb6a29b1280, R_OK) = -1 ENOENT (No such file or directory)
open(0x7fb6a29af901, O_RDONLY) = 3
fstat(3, {...}) = 0
mmap(NULL, 34014, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb6a2ba9000
close(3) = 0
open(0x7fb6a2bb2640, O_RDONLY) = 3
read(3, 0x7ffeb5d5ea78, 832) = 832
fstat(3, {...}) = 0
mmap(NULL, 2236976, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb6a2771000
mprotect(0x7fb6a2788000, 2097152, PROT_NONE) = 0
mmap(0x7fb6a2988000, 49152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7fb6a2988000
close(3) = 0
open(0x7fb6a2bb2b08, O_RDONLY) = 3
read(3, 0x7ffeb5d5ea48, 832) = 832
fstat(3, {...}) = 0
mmap(NULL, 2221912, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb6a2552000
mprotect(0x7fb6a256f000, 2093056, PROT_NONE) = 0
mmap(0x7fb6a276e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c000) = 0x7fb6a276e000
mmap(0x7fb6a2770000, 1880, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fb6a2770000
close(3) = 0
open(0x7fb6a2bb2fd0, O_RDONLY) = 3
read(3, 0x7ffeb5d5ea18, 832) = 832
fstat(3, {...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb6a2ba8000
mmap(NULL, 2105600, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb6a234f000
mprotect(0x7fb6a2351000, 2093056, PROT_NONE) = 0
mmap(0x7fb6a2550000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x7fb6a2550000
close(3) = 0
open(0x7fb6a2ba84b0, O_RDONLY) = 3
read(3, 0x7ffeb5d5e9e8, 832) = 832
fstat(3, {...}) = 0
mmap(NULL, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb6a214b000
mprotect(0x7fb6a214d000, 2097152, PROT_NONE) = 0
mmap(0x7fb6a234d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fb6a234d000
close(3) = 0
open(0x7fb6a2ba8978, O_RDONLY) = 3
read(3, 0x7ffeb5d5e9b8, 832) = 832
fstat(3, {...}) = 0
mmap(NULL, 3750152, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb6a1db7000
mprotect(0x7fb6a1f41000, 2097152, PROT_NONE) = 0
mmap(0x7fb6a2141000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x18a000) = 0x7fb6a2141000
mmap(0x7fb6a2146000, 18696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fb6a2146000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb6a2ba7000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb6a2ba5000
arch_prctl(ARCH_SET_FS, 0x7fb6a2ba57a0) = 0
mprotect(0x7fb6a2141000, 16384, PROT_READ) = 0
mprotect(0x7fb6a234d000, 4096, PROT_READ) = 0
mprotect(0x7fb6a2550000, 4096, PROT_READ) = 0
mprotect(0x7fb6a276e000, 4096, PROT_READ) = 0
mprotect(0x7fb6a2988000, 4096, PROT_READ) = 0
mprotect(0x7fb6a2dd2000, 4096, PROT_READ) = 0
mprotect(0x7fb6a2bb3000, 4096, PROT_READ) = 0
munmap(0x7fb6a2ba9000, 34014) = 0
statfs(0x7fb6a25683d3, {...}) = 0
brk(0) = 0x7fb6a44d9000
brk(0x7fb6a44fa000) = 0x7fb6a44fa000
open(0x7fb6a25683b7, O_RDONLY) = 3
fstat(3, {...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb6a2bb1000
read(3, 0x7fb6a2bb1000, 1024) = 304
read(3, "", 1024) = 0
close(3) = 0
munmap(0x7fb6a2bb1000, 4096) = 0
open(0x7fb6a1f108e0, O_RDONLY) = 3
fstat(3, {...}) = 0
mmap(NULL, 99158576, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb69bf26000
close(3) = 0
geteuid() = 504
open(0x7ffeb5d5eb20, O_RDONLY) = 3
fstat(3, {...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb6a2bb1000
read(3, 0x7fb6a2bb1000, 4096) = 2512
read(3, "", 4096) = 0
close(3) = 0
munmap(0x7fb6a2bb1000, 4096) = 0
open(0x7fb6a44da110, O_RDONLY) = -1 ENOENT (No such file or directory)
open(0x7fb6a44db1a0, O_RDONLY) = -1 ENOENT (No such file or directory)
open(0x7fb6a44da1a0, O_RDONLY) = -1 ENOENT (No such file or directory)
open(0x7fb6a44db120, O_RDONLY) = -1 ENOENT (No such file or directory)
open(0x7fb6a44db230, O_RDONLY) = -1 ENOENT (No such file or directory)
open(0x7fb6a44da220, O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, 0x7ffeb5d5f82a, 4sudo) = 4
write(2, 0x7fb6a2bcc977, 2: ) = 2
write(2, 0x7ffeb5d5c650, 54effective uid is not 0, is sudo installed setuid root?) = 54
write(2, 0x7fb6a2145903, 1
) = 1
close(0) = 0
access(0x7fb6a1f0edc1, F_OK) = -1 ENOENT (No such file or directory)
open(0x7fb6a1f0edb3, O_RDONLY|O_CLOEXEC) = 0
fcntl(0, F_GETFD) = 0x1 (flags FD_CLOEXEC)
lseek(0, 0, SEEK_SET) = 0
alarm(0) = 0
rt_sigaction(SIGALRM, {...}, {...}, 8) = 0
alarm(10) = 0
fcntl(0, F_SETLKW, {...}) = 0
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, 0x7fb6a214a080, 384) = 384
read(0, "", 384) = 0
fcntl(0, F_SETLKW, {...}) = 0
alarm(0) = 10
rt_sigaction(SIGALRM, {...}, NULL, 8) = 0
exit_group(1) = ?
+++ exited with 1 +++

Cloudera Employee
Posts: 289
Registered: ‎07-08-2013

Re: cloudera manager will not start - fresh install

[ Edited ]

write(2, 0x7ffeb5d5c650, 54effective uid is not 0, is sudo installed setuid root?) = 54

The above could indicate an issue also, looking into the "stat" output have you change the permission on /usr or only in /usr/java/...; can you check if you sudo works try this command sudo su - ; can you list your mount flags (cat /proc/mounts) ?

Explorer
Posts: 14
Registered: ‎03-20-2016

Re: cloudera manager will not start - fresh install

 

sudo su -
[root@slrs01va6f1clm1 ~]# exit

 

[hduser@slrs01va6f1clm1 ~]$ cat /proc/mounts
rootfs / rootfs rw 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,relatime 0 0
devtmpfs /dev devtmpfs rw,relatime,size=66060492k,nr_inodes=16515123,mode=755 0 0
devpts /dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime 0 0
/dev/mapper/vglocal00-root00 / ext4 rw,relatime,barrier=1,data=ordered 0 0
/proc/bus/usb /proc/bus/usb usbfs rw,relatime 0 0
/dev/sda1 /boot ext4 rw,relatime,barrier=1,data=ordered 0 0
/dev/mapper/vglocal00-home00 /home ext4 rw,relatime,barrier=1,data=ordered 0 0
/dev/mapper/vglocal00-opt00 /opt ext4 rw,relatime,barrier=1,data=ordered 0 0
/dev/mapper/vglocal00-tmp00 /tmp ext4 rw,nosuid,nodev,noexec,relatime,barrier=1,data=ordered 0 0
/dev/mapper/vglocal00-log00 /var/log ext4 rw,relatime,barrier=1,data=ordered 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
/dev/mapper/3600a0980383036566e3f482f48633158 /data ext4 rw,noatime,barrier=1,stripe=16,data=ordered 0 0
cm_processes /var/run/cloudera-scm-agent/process tmpfs rw,relatime,mode=751 0 0
[hduser@slrs01va6f1clm1 ~]$

Explorer
Posts: 14
Registered: ‎03-20-2016

Re: cloudera manager will not start - fresh install

I would like to uninstall the master node and start over.

 

I see multiple uninstall pages/procedures.

 

Can anyone recommend a procedure for uninstalling cloudera manager, agents and all packages for CDH 5.6.x?

 

Thank you

Explorer
Posts: 14
Registered: ‎03-20-2016

Re: cloudera manager will not start - fresh install

I completely removed all packages and re-installed master node.

 

No change in behviour. Server start fails immediately..

 

[hduser@slrs01va6f1clm1 ~]$ sudo service cloudera-scm-server start
[sudo] password for hduser:
Starting cloudera-scm-server: [FAILED]


[hduser@slrs01va6f1clm1 ~]$ cd /var/log/cloudera-scm-server/

 

[hduser@slrs01va6f1clm1 cloudera-scm-server]$ cat cloudera-scm-server.out


JAVA_HOME=/usr/java/jdk1.8.0_72
/usr/share/cmf/bin/cmf-server: line 129: /usr/java/jdk1.8.0_72/bin/java: Permission denied
/usr/share/cmf/bin/cmf-server: line 129: exec: /usr/java/jdk1.8.0_72/bin/java: cannot execute: Permission denied

 

This is pretty frustrating.

 

I would appreciate any suggstions.

 

Thank you

Announcements