Cloudera Community
Announcements
Now Live: Explore expert insights and technical deep dives on the new Cloudera Community Blogs — Read the Announcement
Shelton
user rank
Master Mentor

Re: ambari metrics collector restarting again and ...

by Master Mentor in Support Questions
‎08-10-2017 07:37 AM
‎08-10-2017 07:37 AM
@Anurag Mishra First check the value of `zookeeper.znode.parent` in HBase. Set it to the same value in Ambari, Kill all the metrics processes running on the node. `ps -ef | grep metrics` and kill all of them as they were caching the `/hbase` value. Watch the ambari metrics collector logs ( /var/log/ambari-metrics-collector/ambari-metrics-collector.log) while you do the below steps Steps: 0. tail -f /var/log/ambari-metrics-collector/ambari-metrics-collector.log 1. Stop Ambari 2. Kill all the metrics processes 3. curl --user admin:admin -i -H "X-Requested-By: ambari" -X DELETE http://`hostname -f`:8080/api/v1/clusters/CLUSTERNAME/services/AMBARI_METRICS => Make sure you replace CLUSTERNAME with your cluster name 4. Refresh Ambari UI 5. Add Service 6. Select Ambari Metrics 7. In the configuration screen, make sure to set the value of `zookeeper.znode.parent` to what is configured in the HBase service. By default in Ambari Metrics it is set to empty value. 8. Deploy In embedded mode then hbase.cluster.distributed should be false, and hbase.rootdir set to a local directory using the "file://" scheme. ... View more

Re: hadoop + ambari cluster change configuration

by Master Mentor in Support Questions
‎08-09-2017 09:14 PM
‎08-09-2017 09:14 PM
@uri ben-ari You are opening many threads for the same issue be reminded this is unproductive https://community.hortonworks.com/questions/118414/ambari-change-the-parameters-of-the-ambari-service.html?childToView=118423#comment-118423 Members take their valuable time to answer and once your problem has been resolved ,you need to accept the best answer hence rewarding the user. Reward the user with th best answer and close the thread ... View more

Re: Ambari not using available repos

by Master Mentor in Support Questions
‎08-09-2017 08:42 PM
‎08-09-2017 08:42 PM
@John Wright Looking at your HDP.repo I see 2 different URL's one is private and the other is the official public is that a coincidence? baseurl=http://private-repo-1.hortonworks.com/HDP/centos7-ppc/2.x/updates/2.6.0.0-598 baseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ppc64le Please correct that with the hortonworks publc repos and run the below commands yum clean all yum repolist Hope that works ... View more

Re: hadoop + ambari cluster change configuration

by Master Mentor in Support Questions
‎08-09-2017 08:33 PM
‎08-09-2017 08:33 PM
@uri ben-ari The curl command you have run is a GET not POST. To register a blueprint with Ambari you should have done this while in the directory where the blueprint is located. Below is an example curl -H "X-Requested-By: ambari"-X POST -u admin:admin http://<ambari-server>:8080/api/v1/blueprints/multinode-hdp -d @1-HDP6_blueprint.json Hope that helps ... View more

Re: Kerberos Setup on HDP 2.6

by Master Mentor in Support Questions
‎08-09-2017 02:47 PM
‎08-09-2017 02:47 PM
Assumption REALM=GOLD.COM REALM Password =welcome1 KDC_HOST =KDC_SERVER_FQDN Install kerberos package yum install -y krb5-server krb5-libs krb5-workstation Edit Server Configuration Files The configuration files for the server are located in the directory /var/kerberos/krb5kdc edit the 2 files kdc.conf kadm5.acl The conf file is the server configuration and the acl file, well the ACL. The ACL grants all privileges to anyone one with the admin role. Change the Kerberos GOLD.COM realm cd /var/kerberos/krb5kdc/ vi kadm5.acl replace the REALM after the @ with the below */admin@GOLD.COM * Edit the kdc.conf [kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88 [realms] GOLD.COM = { #master_key_type = aes256-cts acl_file = /var/kerberos/krb5kdc/kadm5.acl dict_file = /usr/share/dict/words admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal } Edit the Client Configuration Update the krb.conf on the KDC server copy this file to all the host in the cluster in the below directory /etc/krb5.conf vi /etc/krb5.conf # Configuration snippets may be placed in this directory as well includedir /etc/krb5.conf.d/ [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] dns_lookup_realm = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true rdns = false default_realm = GOLD.COM default_ccache_name = KEYRING:persistent:%{uid} [realms] GOLD.COM = { kdc = KDC_SERVER_FQDN admin_server = KDC_SERVER_FQDN } [domain_realm] .gold.com = GOLD.COM gold.com = GOLD.COM Create the DC database When creating the KDC database we will need an entropy pool of random data sudo kdb5_util create -s -r GOLD.COM Loading random data Initializing database '/var/kerberos/krb5kdc/principal' for realm 'GOLD.COM', master key name 'K/M@GOLD.COM' You will be prompted for the database Master Password. It is important that you NOT FORGET this password.Enter KDC database master key: welcome1 Re-enter KDC database master key to verify: welcome1 Start and Enable Kerberos Start and enable the two services ensure you run status command to validate that kadmin is running # sudo systemctl start krb5kdc kadmin # sudo systemctl enable krb5kdc kadmin Created symlink from /etc/systemd/system/multi-user.target.wants/krb5kdc.service to /usr/lib/systemd/system/krb5kdc.service. Created symlink from /etc/systemd/system/multi-user.target.wants/kadmin.service to /usr/lib/systemd/system/kadmin.service. Check the Principals # sudo kadmin.local Authenticating as principal root/admin@GOLD.COM with password. kadmin.local: listprincs K/M@GOLD.COM kadmin/admin@GOLD.COM kadmin/changepw@GOLD.COM kadmin/KDC_SERVER_FQDN@GOLD.COM kiprop/KDC_SERVER_FQDN@GOLD.COM krbtgt/UPUTEST.CH@UPUTEST.CH kadmin.local: Must create a root principal for kerberization kadmin.local: addprinc root/admin WARNING: no policy specified for root/admin@GOLD.COM; defaulting to no policy Enter password for principal "root/admin@GOLD.COM": Re-enter password for principal "root/admin@GOLD.COM": Principal "root/admin@GOLD.COM" created. Install kerberos Client package on tall the hosts in the cluster install the client yum install -y krb5-libs krb5-workstation This will deliver the necessary files copy the krb5.conf from the KDC server to all the client hosts Invoke kerberos tool in Ambari When running the kerberization tool in Ambari use root/admin@GOLD.COM password welcome1 Follow the instructions take care that you installed JCE on the KDC server ... View more

Re: Beeline connection throws error

by Master Mentor in Support Questions
‎08-09-2017 06:03 AM
‎08-09-2017 06:03 AM
@pv poreddy Here are the different methods to connect. Assumptions: HS2 Host(s): test01.com and test02.com HS2 Binary Port: 10010 HS2 HTTP Port: 10011 ZooKeeper Quorom: test01.com:2181,test02.com:2181:test03.com:2181 HttpPath: cliservice HS2 ZooKeeper Namespace: hiveserver2 User: hiveuser Password: hivepswd NOTE: <db> is the database in the examples below and is optional. The leading slash '/' is required. WARNING: When using 'beeline' and specifying the connection url (-u) at the command line, be sure to quote the url. Non-Secure Environments Direct - Binary Transport Mode beeline -n hiveuser -p hivepswd -u "jdbc:hive2://test01.com:10010/<db>" Direct - HTTP Transport Mode beeline -n hiveuser -p hivepswd -u "jdbc:hive2://test01.com:10011/<db>;transportMode=http;httpPath=cliservice" ZooKeeper - Binary Transport Mode beeline -n hiveuser -p hivepswd -u "jdbc:hive2://test01.com:2181,test02.com:2181,test03.com:2181/<db>" ZooKeeper - Http Transport Mode beeline -n hiveuser -p hivepswd -u "jdbc:hive2://test01.com:2181,test02.com:2181,test03.com:2181/<db>;transportMode=http;httpPath=cliservice" Alternate Connectivity Thru Knox jdbc:hive2://<knox_host>:8443/;ssl=true;sslTrustStore=/var/lib/knox/data/security/keystores/gateway.jks;trustStorePassword=<password>?hive.server2.transport.mode=http;hive.server2.thrift.http.path=gateway/<CLUSTER>/hive Hope that helps ... View more

Re: Beeline connection throws error

by Master Mentor in Support Questions
‎08-08-2017 10:31 PM
‎08-08-2017 10:31 PM
@pv poreddy see my response ! ... View more

Re: Beeline connection throws error

by Master Mentor in Support Questions
‎08-08-2017 10:04 PM
‎08-08-2017 10:04 PM
@pv poreddy Here is a coonect string I just created in a non kerberized cluster. Assumptions hive host=bombay.new.com Default port = 10000 Hive database= hive Hive user =hive Hive password =hive Here you go [hive@bombo ~]$ beeline WARNING: Use "yarn jar" to launch YARN applications. Beeline version 1.2.1.2.3.6.0-3796 by Apache Hive beeline> !connect jdbc:hive2://bombay.new.com:10000/hive Connecting to jdbc:hive2://bombay.new.com:10000/hive Enter username for jdbc:hive2://bombo.new.com:10000/hive: hive Enter password for jdbc:hive2://bombo.new.com:10000/hive: **** Connected to: Apache Hive (version 1.2.1.2.3.6.0-3796) Driver: Hive JDBC (version 1.2.1.2.3.6.0-3796) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://bombo.uganda.com:10000/hive> show databases; +----------------+--+ | database_name | +----------------+--+ | default | +----------------+--+ 1 row selected (5.577 seconds) 0: jdbc:hive2://bombo.uganda.com:10000/hive> So you should have a successful login ... View more

Re: Beeline connection throws error

by Master Mentor in Support Questions
‎08-08-2017 08:41 PM
‎08-08-2017 08:41 PM
@pv poreddy The below output is for a keberized cluster you should replace the principal and REALM with the correct values derived from $ klist -ket /etc/security/keytabs/hive.keytab Keytab name: FILE:/etc/security/keytabs/hive.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 12/28/16 22:25:31 hive-{hostname}@REALM.COM (des3-cbc-sha1) 1 12/28/16 22:25:31 hive-{hostname}@REALM.COM (aes128-cts-hmac-sha1-96) 1 12/28/16 22:25:31 hive-{hostname}@REALM.COM (arcfour-hmac) 1 12/28/16 22:25:31 hive-{hostname}@REALM.COM (des-cbc-md5) 1 12/28/16 22:25:31 hive-{hostname}@REALM.COM (aes256-cts-hmac-sha1-96) To see if you grabbed a validate kerberos ticket [hive@{hostname} ~]$ klist Ticket cache: FILE:/tmp/krb5cc_507 Default principal: hive-{hostname}@REALM.COM Valid starting Expires Service principal 12/28/16 22:57:11 12/29/16 22:57:11 krbtgt/REALM.COM@REALM.COM renew until 12/28/16 22:57:11 12/28/16 22:57:11 12/29/16 22:57:11 HTTP/{hostname}.REALM.COM@renew until 12/28/16 22:57:11 12/28/16 22:57:11 12/29/16 22:57:11 HTTP/{hostname}.REALM.COM@REALM.COM renew until 12/28/16 22:57:11 Then use the above to connect and just PRESS enter at the username and password prompt Beeline version 1.2.1000.2.5.3.0-37 by Apache Hive beeline> ! connect jdbc:hive2://test.com:10000/;principal=hive/test.com@REALM.COM Connecting to jdbc:hive2://test.com:10000/;principal=hive/test.com@REALM.COM Enter username for jdbc:hive2://test.com:10000/;principal=hive/test.com@REALM.COM: Enter password for jdbc:hive2://test.com:10000/;principal=hive/test.com@REALM.COM: Connected to: Apache Hive (version 1.2.1000.2.5.3.0-37) Driver: Hive JDBC (version 1.2.1000.2.5.3.0-37) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://test.com:10000/> show databases; +----------------+--+ | database_name | +----------------+--+ | default | | moroto | | olum | +----------------+--+ 3 rows selected (2.863 seconds) You should now connect successfully otherwise in un kerberized environment [hive@host~]$ beeline .... ..... !connect jdbc:hive2://test.com:10000/<db> Cheers ... View more

Re: how to update ambari cluster with blueprint.js...

by Master Mentor in Support Questions
‎08-08-2017 04:01 PM
‎08-08-2017 04:01 PM
@uri ben-ari Setup YARN HA for example ... View more
Contact Me
Online
Offline
Last Visited
‎12-11-2025 11:50 PM
Community Statistics
Member Since ‎01-19-2017 04:35 AM
Last Visited ‎12-11-2025 11:50 PM
Posts 3,679
Kudos received 627