About Tulasi

Tulasi · ‎06-28-2020

Hi, I am looking for Cloudera CDP, Hive with Kerberos security and encryption to setup a basic development environment. What kind of license policy is required? Thanks, Tulasi

Tulasi · ‎02-12-2019

Hi Li, Thanks for being on top of it and helping me in solving the problem. usermod -g yarn yarn usermod -a -G hadoop yarn Above two commands fixed my problem. [root@optim-rhel72-uppu meta]# id yarn uid=1007(yarn) gid=1008(yarn) groups=1008(yarn),1010(hadoop) I have no idea how yarn user permissions are changed, all that I am following is that what have been suggested in cloudera instructions to enable encryption. Thanks to all of the folks for providing suggestions. Problems like this sucks lot of time in identifying where to fix and I would request cloudera to improve such situations. Thanks, Tulasi

Tulasi · ‎02-07-2019

This is the content of my /etc/fstab file ----------------------------------- /dev/mapper/rhel_rhel72-root / xfs defaults 0 0 UUID=d762b842-5c87-4e4d-bc0e-7a6bad357604 /boot xfs defaults 0 0 /dev/mapper/rhel_rhel72-home /home xfs defaults 0 0 /dev/mapper/rhel_rhel72-swap swap swap defaults 0 0 ----------------------------------- Do I need to change anything? Thanks.

Tulasi · ‎01-31-2019

Hi Li, This is what I am getting: [root@optim-rhel72-uppu ~]# id yarn uid=1007(yarn) gid=1010(hadoop) groups=1010(hadoop) Thanks, Tulasi

Tulasi · ‎01-30-2019

Hi Li, Everything on a single node. /opt/cloudera/parcels/CDH/lib/hadoop-yarn/bin [root@optim-rhel72-uppu bin]# ls -lrt total 80 -rwxr-xr-x 1 root root 12476 May 24 2018 yarn -rwxr-xr-x 1 root root 5463 May 24 2018 mapred ---Sr-s--- 1 root yarn 53712 May 24 2018 container-executor This is the error from /var/log/hadoop-yarn/hadoop-cmf-yarn-NODEMANAGER-optim-rhel72-uppu.development.unicomglobal.software.log.out 2019-01-30 23:42:45,872 INFO org.apache.hadoop.service.AbstractService: Service NodeManager failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to initialize container executor org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to initialize container executor at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:269) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:562) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:609) Caused by: java.io.IOException: Cannot run program "/opt/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/lib/hadoop-yarn/bin/container-executor": error=13, Permission denied Thanks, Tulasi

Tulasi · ‎01-29-2019

Hi Li, I changed the permission, still it didn't fix the problem. [root@optim-rhel72-uppu bin]# ls -alt /opt/cloudera/parcels/CDH/lib/hadoop-yarn/bin/container-executor ---Sr-s--- 1 root yarn 53712 May 24 2018 /opt/cloudera/parcels/CDH/lib/hadoop-yarn/bin/container-executor In this below, is it something to do with banned.users? [root@optim-rhel72-uppu conf.cloudera.yarn]# cat /etc/hadoop/conf.cloudera.yarn/container-executor.cfg yarn.nodemanager.linux-container-executor.group=yarn min.user.id=1000 allowed.system.users=nobody,impala,hive,llama,hbase banned.users=hdfs,yarn,mapred,bin Thanks, Tulasi

Tulasi · ‎01-29-2019

Hi Jerry, Yes, the value of "Container Executor Group" property is matching with CM, see below: [root@optim-rhel72-uppu conf.cloudera.yarn]# cat container-executor.cfg yarn.nodemanager.linux-container-executor.group=yarn min.user.id=1000 allowed.system.users=nobody,impala,hive,llama,hbase banned.users=hdfs,yarn,mapred,bin Thanks, Tulasi

Tulasi · ‎01-29-2019

Hi Manu, I tried your suggestion and it didn't work. Thanks, Tulasi

Tulasi · ‎01-28-2019

Hi Jerry, Here is what I have on my system: /etc/hadoop/conf.cloudera.yarn -r-------- 1 root hadoop 156 Jan 24 01:00 container-executor.cfg -rw-r--r-- 1 root root 3894 Jan 17 22:56 core-site.xml -rw-r--r-- 1 root root 617 Jan 17 22:56 hadoop-env.sh -rw-r--r-- 1 root root 2729 Jan 17 22:56 hdfs-site.xml Even if I change above file permission, after start, it changes back to the same permission. From manager I have this Container Executor Group = yarn Upgrade also not allowing as it requires all services should be up and running. Let me know if you need any more details. Thanks, Tulasi

Tulasi · ‎01-24-2019

Execute command Start this NodeManager on role NodeManager Failed to start role Supervisor returned FATAL. Please check the role log file, stderr, or stdout. Environment details: Version: Cloudera Express 5.15.0 Java VM Name: Java HotSpot(TM) 64-Bit Server VM Java VM Vendor: Oracle Corporation Java Version: 1.7.0_67 System details: Linux optim-rhel72-uppu.development.unicomglobal.software 3.10.0-327.28.3.el7.x86_64 #1 SMP Fri Aug 12 13:21:05 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux I have followed the steps under "Configuring TLS/SSL for HDFS, YARN and MapReduce" using the link https://www.cloudera.com/documentation/enterprise/5-15-x/topics/sg_hive_encryption.html Service did not start successfully; not all of the required roles started: only 0/1 roles started. Reasons : Service has only 0 NodeManager roles running instead of minimum required 1 I see below error in the role log: Error starting NodeManager org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to initialize container executor at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:269) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:562) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:609) Caused by: java.io.IOException: Linux container executor not configured properly (error=24) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:199) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:267) ... 3 more Caused by: ExitCodeException exitCode=24: Invalid conf file provided : /etc/hadoop/conf.cloudera.yarn/container-executor.cfg at org.apache.hadoop.util.Shell.runCommand(Shell.java:604) at org.apache.hadoop.util.Shell.run(Shell.java:507) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:789) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.init(LinuxContainerExecutor.java:193) ... 4 more SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NodeManager at optim-rhel72-uppu.development.unicomglobal.software/10.1.72.3 ************************************************************/ Any help highly appreciated. Thanks, Tulasi

Online	Offline
Last Visited	‎10-25-2020 01:08 PM

Member Since	‎12-27-2018 08:49 PM
Last Visited	‎10-25-2020 01:08 PM
Posts	24

Cloudera Community

Cloudera CDH License

Re: Unable to start YARN - NodeManager on role Nod...

Re: Unable to start YARN - NodeManager on role Nod...

Re: Unable to start YARN - NodeManager on role Nod...

Re: Unable to start YARN - NodeManager on role Nod...

Re: Unable to start YARN - NodeManager on role Nod...

Re: Unable to start YARN - NodeManager on role Nod...

Re: Unable to start YARN - NodeManager on role Nod...

Re: Unable to start YARN - NodeManager on role Nod...

Unable to start YARN - NodeManager on role NodeMan...