@csguna that JIRA mentions a different log and has been fixed.  So it is likely not the issue reported here.  I saw the JIRA for the FD leak for UDFs.  With that one, the FDs would be to the UDF jar file, which is not what is seen here. ... View more

Start by increasing the heap for the sqoop job JVM. ... View more

@csguna @saranvisa I don't know if those settings will effect the Operations log. I did find this JIRA but it isn't for the operations logs. I only skimmed through it though. https://issues.apache.org/jira/browse/HIVE-4500 It definitely sounds like you have a FD leak in HS2. you could just disable the operation logs to alleviate the issue while you dig into it further. For what it is worth, I am running CDH 5.8.2 in production and don't see this issue with HS2. ... View more

I don't know of a doc or if it is not possible. You should be able to do it all using the CM api. You just need to figure out the order in which to do it. Then you could build a script that does it all with CM in place. This is a lot of work and this is why rolling upgrades is an Enterprise feature. What I would do is get the quickstart VM or install a single node version and use the trial license. Then you can conduct rolling restart, get the ordering, commands from the logs, etc. But then you need to build and test your version. Good luck. ... View more

@csguna OS and kernel version from the dump. OS:Red Hat Enterprise Linux Server release 6.9 (Santiago) uname:Linux 2.6.32-696.1.1.el6.x86_64 #1 SMP Tue Mar 21 12:19:18 EDT 2017 x86_64 libc:glibc 2.12 NPTL 2.12 rlimit: STACK 10240k, CORE 0k, NPROC 1024, NOFILE 4096, AS infinity load average:0.00 0.01 0.00 /proc/meminfo: MemTotal: 24591972 kB MemFree: 10750560 kB The load average and memory usage for the system is not high.  The proc and file limit are the defaults but for a Sqoop job this may be ok. VM Arguments: jvm_args: -Xmx1000m -Dhadoop.log.dir=/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hadoop/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hadoop -Dhadoop.id.str= -Dhadoop.root.logger=INFO,console -Djava.library.path=/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Dhadoop.security.logger=INFO,NullAppender java_command: org.apache.hadoop.util.RunJar /home/progr/oracle_export/sqoop.jar It is running with just shy of 1 GB of heap. So it may simply be that. You can try some of the other recommendation it has. Did this run previously in a different version of CDH or is this a new job? ... View more

@desind No. It was kind of always there but the Impala JIRA mentioned above that fixed a issue caused it to pop up. Those CAM releases have the CM fix to have it communicate with the FQDN. For whatever reason it didn't make it into CM 5.11.1. It should be in a CM 5.11 release in the future. In short either upgrade to one of those CM versions or use the --hostname setting for CM/CDH 5.11.1. ... View more

@desind No. It was kind of always there but the Impala JIRA mentioned above that fixed a issue caused it to pop up. Those CAM releases have the CM fix to have it communicate with the FQDN. For whatever reason it didn't make it into CM 5.11.1. It should be in a CM 5.11 release in the future. In short either upgrade to one of those CM versions or use the --hostname setting for CM/CDH 5.11.1. ... View more

@Lars Volker Thanks for adding this bit of info. I was looking at IMPALA-5631 as a suspect but never thought to look at CM. Lesson Learned: pay as much attention to CM release notes as I do CDH release notes. ... View more