We found out that CM server establishes some connections to the remote host in the internet and keeps them for a lifetime.
Remote address is:
What such connections are used for? (Send Diagnostic is turned off)
This IP is a translation for archive.cloudera.com, would you please post the output of this command:
$ sudo ps ex | grep 188.8.131.52
Your command returns nothing.
Here are some details
sudo netstat -anpl | grep 184.108.40.206 tcp 0 0 cm_host:48035 220.127.116.11:443 ESTABLISHED 2485/java tcp 0 0 cm_host:4734 18.104.22.168:80 ESTABLISHED 2485/java sudo ps -ef | grep 2485 496 2485 2483 5 Jun06 ? 11:58:21 /usr/java/jdk1.7.0_80/bin/java -cp .:lib/*:/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar -server -Dlog4j.configuration=file:/etc/cloudera-scm-server/log4j.properties -Dfile.encoding=UTF-8 -Dcmf.root.logger=INFO,LOGFILE -Dcmf.log.dir=/var/log/cloudera-scm-server -Dcmf.log.file=cloudera-scm-server.log -Dcmf.jetty.threshhold=WARN -Dcmf.schema.dir=/usr/share/cmf/schema -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Dpython.home=/usr/share/cmf/python -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+HeapDumpOnOutOfMemoryError -Xmx2G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp -XX:OnOutOfMemoryError=kill -9 %p com.cloudera.server.cmf.Main
I have the same "issue" on my cluster cloudera (v5.14). Is it possible to have an explanation about this connections ? What information is sent and why ?
I didn't find anything about this in the documentation.
It's every hour and there are 2 differents IP : 22.214.171.124 and 126.96.36.199
root@serveur-hadoop:~# lsof -Pni | grep 188.8.131.52 java 21591 cloudera-scm 244u IPv4 510376752 0t0 TCP X.X.X.X:58048->184.108.40.206:80 (ESTABLISHED) java 21591 cloudera-scm 301u IPv4 510376750 0t0 TCP X.X.X.X:38753->220.127.116.11:443 (ESTABLISHED)
I believe those connections are for archive.cloudera.com which is where Cloudera Parcels are stored.
I see the same behavior, but with a different IP address (most likely due to my different geographic location).
When Cloudera Manager looks for new parcels, it will open connections to the URLs specified in the Administration --> Settings --> Parcels --> Remote Parcel Repository URLs
By default, you may have a mixture of "https://" and "http://" urls listed in the Parcels configuration. This is why you see connections to both ports 80 and 443.
I observed that the connections would remain for a few minutes after the check, but eventually the connections would be terminated. I am not sure why these are open for a few minutes, but I confirmed via tcpdump that no traffic is active over these ports.
Another thing to check in your Parcels configuration is "Parcel Update Frequency". If you are seeing connections to the parcel repositories open all the time, make sure it is set to 1 day or something bigger.