Member since
01-24-2018
13
Posts
1
Kudos Received
0
Solutions
02-06-2018
01:08 AM
hi I had a cloudera manager 5.14 install. I selected all hosts in Admin UI, hit remove from cluster, then chose delete all hosts from cluster. I had stopped the scm agent on all machines. I find that my HDFS drives have some files left behind, how to clean them up ? Does it matter if these files are left lying around, if I try a new installation ? $ sudo ls /data/1/dfs/dn/current/ BP-<something> VERSION ]$ sudo cat /data/1/dfs/dn/current/VERSION #<something> storageID=<something> clusterID=cluster16 cTime=0 datanodeUuid=<something> storageType=DATA_NODE layoutVersion=-56
... View more
Labels:
- Labels:
-
Cloudera Manager
-
HDFS
02-01-2018
01:12 PM
Hi it seems the default log location is under /var/log How can I change this after the installation. Also I want to reduce the amount of logging. Otherwise my disk will run out of space.
... View more
Labels:
- Labels:
-
Cloudera Manager
02-01-2018
12:22 PM
Hi
during role selection, Is there any easy way to say that 1 machine is the master , so it is the Yarn resourcemanager.
Every other machine is a worker ( ie.e node manager).
I do not want to assign roles to every machine for every service I have selected ( hadoop, spark, zookeper, etc).
The master machine should get all the master roles for every service.
Thnaks!
... View more
Labels:
- Labels:
-
Apache YARN
-
Cloudera Manager
02-01-2018
10:26 AM
Hi My install fails when it tries to get heartbeat from newly installed agent. I tried a few things suggested on the interenet by other users, such as cleaning the folders: rm /var/lib/cloudera-scm-agent/ -rf rm /var/lib/cloudera-scm-server/ -rf then I uninstalled and reinstalled: /usr/share/cmf/uninstall-cloudera-manager.sh ./cloudera-manager-installer.bin But the error on step 5 of the install wizard is always : Installation failed. Failed to receive heartbeat from agent. LOG SNIPPET: BEGIN sudo tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/' >>[01/Feb/2018 12:29:01 +0000] 68880 MainThread agent ERROR Error, CM server guid updated, expected eb29c04c-7699-43d6-8896-ea6024c0edc2, received 40d851ea-2dd7-4a6d-8840-410425111154 >>[0 >>[01/Feb/2018 12:49:29 +0000] 71602 MainThread __init__ ERROR Failed to generate and store Agent UUID >>IOError: [Errno 2] No such file or directory: '/var/lib/cloudera-scm-agent/uuid' >>[01/Feb/2018 12:49:29 +0000] 71602 Dummy-1 daemonize WARNING Stopping daemon. >>[01/Feb/2018 13:02:19 +0000] 73786 MainThread __init__ INFO Generating Agent UUID for the first time: '/var/lib/cloudera-scm-agent/uuid' >>[01/Feb/2018 13:02:19 +0000] 73786 MainThread __init__ ERROR Failed to generate and store Agent UUID >>Traceback (most recent call last): >> File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.14.0-py2.7.egg/cmf/util/__init__.py", line 414, in get_host_uuid >> uuid_file = open(uuid_path, 'w') >>IOError: [Errno 2] No such file or directory: '/var/lib/cloudera-scm-agent/uuid' >>[01/Feb/2018 13:02:19 +0000] 73786 Dummy-1 daemonize WARNING Stopping daemon. END (0) end of agent logs.
... View more
Labels:
- Labels:
-
Cloudera Manager
02-01-2018
06:51 AM
thank you for clarifying. I am taking the non-production mode , does this mode allow for cluster size increases like you described ? Or must we have production mode in the first place to allow any sort of cluster size increase ? https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_non_production.html https://www.cloudera.com/documentation/enterprise/latest/topics/installation_installation.html
... View more
02-01-2018
06:47 AM
turns out that my entire lab was behind another firewall, so browsers were not able to pull up the webpage. So basically the firewall problem was not that selinux was running ( it was disabled ), but there was another firewall protecting the entire lab.
... View more
01-31-2018
09:19 AM
Hi I am on centos 7 and cant open this page http://Server host:7180 to begin my installation . I see no errors in /var/log/cloudera-scm-server/cloudera-scm-server.log, iptables has been stopped and sestatus is disabled. sudo netstat -anp | grep 7180 tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 22233/java wget localhost:7180/ return index.html with 200 Ok what could be wrong? Thnaks!
... View more
Labels:
- Labels:
-
Cloudera Manager
01-29-2018
04:41 AM
Hi
Say I have a haddop/spark cluster with 1 master and 5 slave computers.
I am planning to get 3 more new computers, which I want to add to my cluster as 3 more new workers. [ so now I have 1 master and 8 workers]
I would want HDFS installed on all 8 workers. It seems this can be done using the admin UI.
In addition, I also want any hadoop and spark and other packages to be installed on the new workers as well, and configurations updated on the master , so the master knows there are 8 workers now.
How can I use the cloudera manager admin UI to add these new worker computers into my cluster ? The master remains the same.
Thanks !
... View more
Labels:
- Labels:
-
Apache Spark
-
Cloudera Manager
-
HDFS
01-26-2018
04:46 AM
1 Kudo
Hi
I am installing Cloudera Manager , Spark and Hadoop on 1 master and 5 slave computers.
Later on, I am planning to get 3 more slave computers after a few months.
Can I simply increase the size of my cluster using manager UI ? Will that result in all the packages being installed on the new computers and increase in cluster size?
Thnaks.
... View more
Labels:
- Labels:
-
Apache Spark
-
Cloudera Manager
01-24-2018
03:34 PM
thank you, I will try it out.
... View more