Member since
12-10-2014
22
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5794 | 12-11-2014 02:59 AM |
11-25-2015
01:38 PM
Hi, I'm afraid it's been a while since I did this and it's all a blur in my rear view memory now. As it happens, I'm getting back into Cloudera again and will be building some new VM's soon. If I have to do an update I'll let you know how I do it. Sorry for not having an answer for you
... View more
07-18-2015
06:43 AM
Thanks Sean. I appreciate the guidance. I now have Vmware ESXi setup and an ubuntu 14.04 server guest. I'm going through "Installation Path A - Automted Installation by Cloudera Manager". All going well so far.
... View more
07-16-2015
11:20 PM
Hi, I've got the quickstart vm up and running and working pretty well and I'd like to test out a few of the Cloudera administrative tasks using manager. I'm wondering if anyone has successfully deployed additional nodes from the quickstart VM and gotten a cluster (virtual or physical) up and running? Thanks Ty
... View more
Labels:
- Labels:
-
Quickstart VM
02-18-2015
12:56 PM
Thanks. You're a life saver. The cluster now reports as 5.3.1. I guess one thing to learn from this is that it may be worth updating http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/install_upgrade_cdh_maintenance_packages.html to extend the list of example unmanaged services from Mahout, Pig, Whirr to Parquet, HBase, Zookeeper, Solr, Mahout, Pig, Whirr.
... View more
02-18-2015
05:30 AM
1 Kudo
I have the CDH 5.3.0 virtual machine and I'd like to upgrade it to the latest 5.3.1 release in the hope that it will resolve a few configuration issues I have run into. Have followed the instructions here: upgrade CDM & Java: http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ag_upgrade_cm5.html upgrade CDH: http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/install_upgrade_cdh_maintenance_packages.html Unfortunately I end up with the screen in the upgrade wizard at step 6 (Cloudera Manager checks that hosts have the correct software installed. If the packages have not been installed, a warning displays to that effect. Install the packages and click Check Again. When there are no errors, click Continue.): Upgrade the CDH running on Cloudera QuickStart Host Software Detection Checking if the relevant agents are responsive, the relevent hosts are healthy and have the correct software installed on them. Multiple releases detected: [CDH 5.3.1, CDH 5.3.0] The following host(s), quickstart.cloudera, do not have the newly required CDH package installed. They still have the incorrect CDH 5.3.0. It doesn't seem right to me since my Host Inspector looks like everything is fine. Output is this: Inspector Results Validations Inspector ran on all 1 hosts. The following failures were observed in checking hostnames... No errors were found while looking for conflicting init scripts. No errors were found while checking /etc/hosts. All hosts resolved localhost to 127.0.0.1. All hosts checked resolved each other's hostnames correctly and in a timely manner. Host clocks are approximately in sync (within ten minutes). Host time zones are consistent across the cluster. No users or groups are missing. No conflicts detected between packages and parcels. No kernel versions that are known to be bad are running. Cloudera recommends setting /proc/sys/vm/swappiness to 0. Current setting is 1. Use the sysctl command to change this setting at runtime and edit /etc/sysctl.conf for this setting to be saved after a reboot. You may continue with installation, but you may run into issues with Cloudera Manager reporting that your hosts are unhealthy because they are swapping. The following hosts are affected: No performance concerns with Transparent Huge Pages settings. CDH 5 Hue Python version dependency is satisfied. 0 hosts are running CDH 4 and 1 hosts are running CDH5. All checked hosts in each cluster are running the same version of components. All managed hosts have consistent versions of Java. All checked Cloudera Management Daemons versions are consistent with the server. All checked Cloudera Management Agents versions are consistent with the server. Version Summary Cloudera QuickStart — CDH 5 Hosts quickstart.cloudera Component Version Release CDH Version Parquet 1.5.0+cdh5.3.0+52 1.cdh5.3.0.p0.27 CDH 5 Impala 2.1.1+cdh5.3.1+0 1.cdh5.3.1.p0.17 CDH 5 YARN 2.5.0+cdh5.3.1+791 1.cdh5.3.1.p0.17 CDH 5 HDFS 2.5.0+cdh5.3.1+791 1.cdh5.3.1.p0.17 CDH 5 hue-common 3.7.0+cdh5.3.1+135 1.cdh5.3.1.p0.17 CDH 5 Sqoop2 1.99.4+cdh5.3.1+20 1.cdh5.3.1.p0.17 CDH 5 kms Unavailable Unavailable Not installed or path incorrect HBase 0.98.6+cdh5.3.0+73 1.cdh5.3.0.p0.25 CDH 5 Sqoop 1.4.5+cdh5.3.1+61 1.cdh5.3.1.p0.17 CDH 5 Oozie 4.0.0+cdh5.3.1+335 1.cdh5.3.1.p0.17 CDH 5 Zookeeper 3.4.5+cdh5.3.0+81 1.cdh5.3.0.p0.36 CDH 5 Hue 3.7.0+cdh5.3.1+135 1.cdh5.3.1.p0.17 CDH 5 spark 1.2.0+cdh5.3.1+365 1.cdh5.3.1.p0.17 CDH 5 MapReduce 1 2.5.0+cdh5.3.1+791 1.cdh5.3.1.p0.17 CDH 5 Pig 0.12.0+cdh5.3.1+47 1.cdh5.3.1.p0.17 CDH 5 Crunch (CDH 5 only) 0.11.0+cdh5.3.1+17 1.cdh5.3.1.p0.17 CDH 5 Llama (CDH 5 only) 1.0.0+cdh5.3.1+0 1.cdh5.3.1.p0.17 CDH 5 HttpFS 2.5.0+cdh5.3.1+791 1.cdh5.3.1.p0.17 CDH 5 Hadoop 2.5.0+cdh5.3.1+791 1.cdh5.3.1.p0.17 CDH 5 Hive 0.13.1+cdh5.3.1+308 1.cdh5.3.1.p0.17 CDH 5 HCatalog 0.13.1+cdh5.3.1+308 1.cdh5.3.1.p0.17 CDH 5 sentry 1.4.0+cdh5.3.1+127 1.cdh5.3.1.p0.17 CDH 5 MapReduce 2 2.5.0+cdh5.3.1+791 1.cdh5.3.1.p0.17 CDH 5 Lily HBase Indexer 1.5+cdh5.3.1+24 1.cdh5.3.1.p0.17 CDH 5 Solr 4.4.0+cdh5.3.0+314 1.cdh5.3.0.p0.25 CDH 5 Flume NG 1.5.0+cdh5.3.1+80 1.cdh5.3.1.p0.19 CDH 5 Cloudera Manager Management Daemons 5.3.1 1.cm531.p0.191 Not applicable Java 6 JAVA_HOME=/usr/java/jdk1.6.0_31 java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) Unavailable Not applicable Java 7 JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) Unavailable Not applicable Java 8 Unavailable Unavailable Not applicable Cloudera Manager Agent 5.3.1 1.cm531.p0.191.el6 Not applicable Once again, I'm grateful for any assistance. Thanks, Ty
... View more
02-17-2015
04:44 AM
Thanks for the link to your kerberos bootstrap'er. It seems to work for me. Unfortunately I ran into a secondary problem and I'm not sure how to let Cloudera know about it. On a clean CDH5.3 virtual machine I started Cloudera Manager and then ran your kerberos bootstrapper. Then I ran the Kerberos configuration wizard in Cloudera Manager. The restart failed to complete successfully. It seemed like the Yarn NameNode was having trouble with the topology.map amd container-executer.cfg file permissions in the /var/run/cloudera-scm-agent/process/??-yarn-NODEMANAGER/ directory (note: ?? is a number generated each time I tried to restart the NodeManager). On further inspection, and a couple snapshot reverts, I think I found the real problem. the /etc and /etc/hadoop directories have group write permissions set. This was identified in the NodeManager logs. I started again and changed the permissions to remove the group write permission on both directories - before running the Cloudera Manager Kerberos configuration wizard. This time it seems to have worked. note: I hve not yet had a chance to test kerberos actually doing anything from a hdfs/hive/pig user perspective. Only one last glitch in the system. It looks like Spark does not have Kerberos credentials created for it. The Spark History Server is showing as critical health and its log is identifying missing Kerberos credentials. When I look at the Kerberos Credentials screen in Cloudera Manager I see credentials for all the services excpt Spark. I'm not doing anything with Spark and I don't know anything about it so I'll just stop the service for now. I am not sure if changing the directory permissions on /etc and /etc/hadoop will adversely affect other functions; but I hope my little investigation can help others.
... View more
02-16-2015
09:39 PM
ok, I posted too soon. I seem to have solved it. I addedd all the key algorithms that kadmin.local listed when I did a get_principal on the cloudera-scm/admin principal. Restarting the cluster now...
... View more
02-16-2015
09:30 PM
I'm afraid it's not smooth sailing on this one. I found the github project here: https://github.com/esammer/krb-bootstrap It all seems to work ok. I seem to get Kerberos and a realm (CLOUDERA) and a principal (cloudera-scm/admin). After some searching I managed to set the password for cloudera-scm/admin usinf the command line tool kadmin.local Unfortunately when I get to step 5 (import KDC Account Manager Credentials) of the Coudera Manager kerberos setup wizard I get the following message. I'm afraid I'm stuck again and could use some help if anyone knows how to get past this problem. /usr/share/cmf/bin/import_credentials.sh failed with exit code 1 and output of <<
+ export PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/sbin:/usr/sbin:/bin:/usr/bin
+ PATH=/usr/kerberos/bin:/usr/kerberos/sbin:/usr/lib/mit/sbin:/usr/sbin:/sbin:/usr/sbin:/bin:/usr/bin
+ KEYTAB_OUT=/var/run/REDACTED-scm-server/cmf242896655772090475.keytab
+ USER=REDACTED-scm/admin@CLOUDERA
+ PASSWD=REDACTED
+ KVNO=1
+ SLEEP=0
+ RHEL_FILE=/etc/redhat-release
+ '[' -f /etc/redhat-release ']'
+ set +e
+ grep Tikanga /etc/redhat-release
+ '[' 1 -eq 0 ']'
+ '[' 0 -eq 0 ']'
+ grep 'CentOS release 5' /etc/redhat-release
+ '[' 1 -eq 0 ']'
+ '[' 0 -eq 0 ']'
+ grep 'Scientific Linux release 5' /etc/redhat-release
+ '[' 1 -eq 0 ']'
+ set -e
+ '[' -z /etc/krb5.conf ']'
+ echo 'Using custom config path '\''/etc/krb5.conf'\'', contents below:'
+ cat /etc/krb5.conf
+ IFS=' '
+ read -a ENC_ARR
+ for ENC in '"${ENC_ARR[@]}"'
+ echo 'addent -password -p REDACTED-scm/admin@CLOUDERA -k 1 -e des-hmac-sha1'
+ '[' 0 -eq 1 ']'
+ echo REDACTED
+ echo 'wkt /var/run/REDACTED-scm-server/cmf242896655772090475.keytab'
+ ktutil
+ chmod 600 /var/run/REDACTED-scm-server/cmf242896655772090475.keytab
+ kinit -k -t /var/run/REDACTED-scm-server/cmf242896655772090475.keytab REDACTED-scm/admin@CLOUDERA
kinit: Key table entry not found while getting initial credentials
>>
... View more
02-16-2015
04:12 PM
Just to be 100% sure are you saying that it is not possible to implement Sentry with the virtual machine alone since it does not have any kerberos functionality inbuilt?
... View more