Can you upload, Kafka and Flume config? ... View more

It's possible, but if you can not upgrade to the last version, you can try my steps to recreate manually. Regards, Marc. ... View more

This command loads environment variables that you will use in the next command: telnet $server_host $server_port You need to check if the problem is for network issue or for application issue. Regards, Marc. ... View more

Please run:   $ source /etc/cloudera-scm-agent/config.ini &>/dev/null $ telnet $server_host $server_port &> without any space between & and >.   Regards,  Marc. ... View more

Hi,    You have any snapshot enabled? I think that with "hdfs fsck /" you are not checking snapshots.   Remove the snapshot with missing blocks and the error will be disappear.   From cloudera you can check in Top Menu > Backup > Snapshot Policies   The BDR replica can use snapshots automatically, so you need the check from command line:   Check Snapshotable directories: $ hdfs lsSnapshottableDir drwxrwx--- 0 user1 group1 0 2017-08-22 04:00 0 655432 /dir1 Remove the snapshot: $ hdfs dfs -ls /dir1/.snapshot drwxrwx--- 0 user1 group1 0 2017-08-22 04:00 0 655432 /dir1/.snapshot/<snapshot_id>/dir1 $ hdfs dfs -deleteSnapshot /dir1 <snapshot_id>   Check HDFS Snapshots for more information.   Regards, Marc Casajus. ... View more

Hi,   I'm not sure what you want.   Which cloudera version you run?   To configure HDFS HA (2 namenodes in the same cluster), you need to configure more things, please check this url: Enabling HDFS HA   If you want to copy data between to clusters using distcp, you don't need to modify any file, even less hdfs-site.xml, please check this url: Copying Cluster Data Using DistCp   Regards, Marc   ... View more

It depends of the usage or type of use.   You will use this cluster for real-time or batch processing?   Which technologies you will use? yarn, spark, hive, impala, flume, kafka…   Typical hadoop cluster or kudu cluster?   Regards,  Marc.   ... View more

Hi cdhhadoop,   Cloudera agent is completely down? It happens in more servers?   Can you provide /var/log/cloudera-scm-agent/cloudera-scm-agent.out output?   Can you provide the output of the next commands?:     $ netstat -ltnp | grep :9000 $ source /etc/cloudera-scm-agent/config.ini &>/dev/null $ ping -w1 $server_host $ telnet $server_host $server_port Regards, Marc Casajus ... View more

Hi  tasch,   /tmp/hadoop-yarn have incorrect owner, it needs to bee yarn.    /var/run/cloudera-scm-agent/cgroups/cpu/hadoop-yarn, needs to be created in all nodemanegers.   Can you try to create this directories?   In HDFS:   hdfs dfs -mkdir /tmp/hadoop-yarn hdfs dfs -chmod 755 /tmp/hadoop-yarn hdfs dfs -chown yarn:supergroup /tmp/hadoop-yarn In Operating System: for NodeManager in $NodeManegerList do ssh $NodeManager 'mkdir --mode=775 /var/run/cloudera-scm-agent/cgroups/cpu/hadoop-yarn' ssh $NodeManager 'chown yarn:hadoop /var/run/cloudera-scm-agent/cgroups/cpu/hadoop-yarn' done   Please if you found another solution, please share it.   It works for me in cdh 5.9.   Regards, Marc Casajús. ... View more

Which Directory are you remove?   # Backup Event Server Index Directory: cp -ra  /var/lib/cloudera-scm-eventserver /var/lib/cloudera-scm-eventserver_backup   # Remove all Event Server Index Directory Content: rm -fr /var/lib/cloudera-scm-eventserver/*   # Restart Event Server Role It should work. ... View more

Also if you use Sentry Synchronization and Sentry Synchronization Path Prefixes  you need to change owner an group for all hive directories and files in hdfs to hive:hive. If you enable this feature you cannot change hive warehouse ACLs manually, because it is controlled by Sentry (with grants).   You can assign this ACLs manually but is no recommended.   Marc. ... View more

Hi,   I work with cdh5.4.7 and I have the same issue, and I resolved it.   When Cloudera Manager server is restarted after upgrade or maintance tasks this starts cloudera server and cloudera agent, but it does not start Cloudera Management Services (mgmt).   The reason is because cloudera-scm-server and cloudera-scm-agent is configured to start at the same time: [ cloudera_server ]: grep chkconfig /etc/init.d/cloudera-scm-* /etc/init.d/cloudera-scm-agent:# chkconfig: 2345 90 10 /etc/init.d/cloudera-scm-server:# chkconfig: 2345 90 10   Cloudera Agent start Cloudera Management Services (mgmt) and it needs to connect to Cloudera Server, Cloudera Server takes more time to start than Cloudera Agent. Cloudera Agent tries to start mgmt 5 times with only 2 seconds between every retry, finally mgmt cannot start (in mgmt role logs I can see "connection refused" errors): 2017-01-02 15:06:44,673 WARN com.cloudera.cmon.firehose.Main: No descriptor fetched from https:// cloudera_server :7183 on after 1 tries, sleeping... 2017-01-02 15:06:44,798 WARN com.cloudera.cmf.event.publish.EventStorePublisherWithRetry: Failed to publish event: SimpleEvent{attributes={ROLE_TYPE=[SERVICEMONITOR], EXCEPTION_TYPES=[java.net.ConnectException], HOST_IDS=[..], STACKTRACE=[java.net.ConnectException: Connection refused [..] 2017-01-02 15:06:46,708 WARN com.cloudera.cmon.firehose.Main: No descriptor fetched from https:// cloudera_server :7183 on after 2 tries, sleeping... [..] 2017-01-02 15:06:52,724 WARN com.cloudera.cmon.firehose.Main: No descriptor fetched from https:// cloudera_server :7183 on after 5 tries, sleeping... To temporally fix this issue I do that:   1. Change start order: I changed to this (server 90 to 89): [ cloudera_server ]: grep chkconfig /etc/init.d/cloudera-scm-* /etc/init.d/cloudera-scm-agent:# chkconfig: 2345 90 10 /etc/init.d/cloudera-scm-server:# chkconfig: 2345 89 10   2. Add cloudera server check in agent init start script. /etc/init.d/cloudera-scm-agent (green line): --- [..] start() { [..] + for i in $(seq 1 30); do curl -k -s -I $(facter cdh_url | awk -F\/api '{print $1}') | grep -q '200 OK' &>/tmp/init_cloudera_agent.out && break; sleep 10; done $CMF_SUDO_CMD /bin/bash -c "nohup $AGENT_SCRIPT $CMF_AGENT_ARGS" >> $AGENT_OUT 2>&1 </dev/null & [..] } [..] --- * cdh_url  is a custom facter that retur https://cloudera_server:7183/api/v10   If I only change start order it doesn't works because when "/etc/init.d/cloudera-scm-server start" is executed it doesn't wait to be completely started, it returns OK immediately (but is starting yet in background). When I reboot this server it starts cloudera-scm-server and immediately starts cloudera-scm-agent, cloudera-scm-agent starts faster than cloudera-scm-server and mgmt cannot connect to cloudera server web, after 5 tries it still down and I need to start mgmt manually.   If I do this changes it works fine, but I think that I should not change this configurations…   Another valid solution is that cloudera-scm-server waits to be successful and completely started to return OK and start first server and then agent with mgmt services, but for the moment it works for me.   Marc. ... View more

Hi  aroraprateek,    I have the same issue and I have resolved.   ... View more