@Nick,   Thanks for the log.  Unfortunately, it does not contain information regarding the stop or start activity that might have led to this condition.   Currently, the supervisor and agent seem to be in an "invalid" state that is not handled by the agent code.   I would recommend restarting the agent on one host and see if that helps prevent the error from occurring. Run:   systemctl restart cloudera-scm-agent   Monitor the agent logs on the host where you restarted the agent to see if there are any errors.   If that doesn't help, we may need to restart the supervisor as well...  Let us know. ... View more

@TCloud,   Can you clarify the problem you are seeing regarding port 7182? ... View more

@manu_009,   Unfortunately, the CM log kind of shows us more of the same:  the server attempts to execute a query but communication with MySQL fails.   If this is a problem where the server cannot start, I'd be interested in seeing the logs from startup just to see if there are any clues when CM attempts to connect to MySQL initially.  If it gets by that part but then fails later on, that could tell us something.   Since the CM log shows it is waiting for packets that never arrive, that means it would be good to look at the network and MySQL to see if it gets connections from CM.   I'd recommend the following:   - Try starting CM and while CM is starting, run the "show full processlist;" to see if any connections are establishjed You could also use netstat -nap or similar to see if connections are established. - Check MySQL documentation... see if there is a way to debug incoming connections - Try running this on the CM host before trying to start:  "tcpdump -i any -w ~/mysql.pcap port 3306" (or whatever port your MySQL is listining on)   This will capture all packets going out to and coming back from the MySQL server. After CM does not start, use Ctrl-c to quit tcpdump and finalize the pcap file.   You can use WireShark to review communication with MySQL.   You could also run tcpdump on the MySQL host (if it is not the CM host) to find out how communication looks on the MySQL host side.   Generally, the errors you are seeing do not indicate a problem with CM itself; rather, CM cannot read information from the database. ... View more

I tested on CM 6.2 and did not encounter the problem on CFM 1.0.1.0. My process directory (created by CM/agent for the process to start) does not include the file. However, the config.zip that is in the process directory does contain it:   [root@host-10-17-100-225 2537-nifitoolkitca-NIFI_TOOLKIT_SERVER]# unzip -l config.zip Archive: config.zip Length Date Time Name --------- ---------- ----- ---- 2094 07-13-2019 13:44 staging/globals.xml 1192 07-13-2019 13:44 logback.xml 828 07-13-2019 13:44 scripts/cc.sh 294 07-13-2019 13:44 cloudera-monitor.properties 238 07-13-2019 13:44 aux/configbuilder/ca-config.json 0 07-13-2019 13:44 redaction-rules.json 308 07-13-2019 13:44 cloudera-stack-monitor.properties 1205 07-13-2019 13:44 aux/templates/env.sh 1280 07-13-2019 13:44 aux/templates/config.json 989 07-13-2019 13:44 aux/templates/hadoop_xml_properties.j2 2332 07-13-2019 13:44 scripts/control.sh 1006 07-13-2019 13:44 aux/configbuilder/control-env.json 809 07-13-2019 13:44 aux/defaults/nifi-ca-client.xml   I saw one hint in an internal discussion that permissions on /var/lib/nifi* could be an issue... can you do an "ls -lrt /var/lib/nifi*" and show the results?   I'll work with the NiFi team on Monday to see if we can get this working for you. ... View more

@anung @Sheldon_G @Nando ,   I'm sorry to hear you are hitting this issue.  Let me walk through the install and see if I can lend a hand.  What instructions are you using?  ... View more

Oops; I forgot ot mention that the line to edit should be 95 or close to there. Also, this would need to be done on all hosts managed by Cloudera Manager as each agent listens on port 9000.     ... View more

@BrettM,   We have a fix to disable TLSv1 that is already slated for upcoming releases of 6.2.x and 6.3. I tested, though, and found that the fix still allowed TLSv1_1.  After some research I was able to work around a limitation with M2Crypto (which does not support the SSL_OP_NO_TLSv1_1 flag that OpenSSL uses to disable TLSv1_1). I opened a new Jira internally at Cloudera so we can review it and check it in as needed.   If you are comfortable editing python files, you can test it out yourself.   Back up /opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py NOTE:  if you are on an el6 platform, the above path may differ.  To find it, you can run: # find /opt/cloudera/cm-agent -name "https.py" Edit the https.py file (on el7 platforms, it will be /opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py Find the line that looks like this: ctx.set_options(m2.SSL_OP_NO_SSLv2 | m2.SSL_OP_NO_SSLv3) Edit the line to look like this (additions in bold): ctx.set_options(m2.SSL_OP_NO_TLSv1 | 0x10000000L | m2.SSL_OP_NO_SSLv2 | m2.SSL_OP_NO_SSLv3) NOTE:  This is Python, so make sure the indentation does not change or the file may not compile during runtime Restart the agent with # service cloudera-scm-agent restart Test by running this on the same host where you made the above edit: #openssl s_client -connect `hostname -f`:9000 -tls1_1 -msg < /dev/null The expected result is that after the ClientHello, nothing is returned from port 9000.  It might look like this: CONNECTED(00000003) >>> TLS 1.1 Handshake [length 0096], ClientHello 01 00 00 92 03 02 5d 29 1e 3b c2 2a 7c 95 49 36 87 e5 8e 82 31 cb af 9a 64 6d a2 1d 2a 2a 95 9f 76 13 3a aa d3 24 00 00 4c c0 14 c0 0a 00 39 00 38 00 88 00 87 c0 0f c0 05 00 35 00 84 c0 13 c0 09 00 33 00 32 00 9a 00 99 00 45 00 44 c0 0e c0 04 00 2f 00 96 00 41 c0 12 c0 08 00 16 00 13 c0 0d c0 03 00 0a 00 07 c0 11 c0 07 c0 0c c0 02 00 05 00 04 00 ff 01 00 00 1d 00 0b 00 04 03 00 01 02 00 0a 00 08 00 06 00 19 00 18 00 17 00 23 00 00 00 0f 00 01 01 If that is what you see, TLSv1 and TLSv1.1 connections will not be serviced.  The "hang" is OK in this case since Cloudera Manager is the only client of port 9000 and it should be using TLSv1.2.   WARNING:  The above steps I tested on my own host, but use at your own risk.  If you prefer to not incur the risk, please await an upcoming release that includes the code changes.  If for some reason the agent doesn't restart, you can copy the backup file in place and restart the agent to recover.  This will not impact data.   Cheers,   Ben ... View more