Member since
01-19-2017
3679
Posts
632
Kudos Received
372
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 794 | 06-04-2025 11:36 PM | |
| 1372 | 03-23-2025 05:23 AM | |
| 683 | 03-17-2025 10:18 AM | |
| 2471 | 03-05-2025 01:34 PM | |
| 1608 | 03-03-2025 01:09 PM |
09-30-2017
08:55 AM
@ashim sinha Hive CLI is now a legacy tool.HiveServer1 is already deprecated it has been a while that Hive community has been recommending Beeline + HS2 configuration, ideally because of the deprecation Hive CLI, use the below example to achieve the same goal. Assuming you have the correct hive principal replace the below with appropriate values. This is how it works on my cluster ###########################################################
# Access HiveServer2 in kerberized cluster
# hive CLI has been deprecated in HS2
###########################################################
# su - hive
[hive@london ~]$ beeline
Beeline version 1.2.1000.2.5.3.0-37 by Apache Hive
beeline> ! connect jdbc:hive2://london.uk.com:10000/;principal=hive/london.uk.com@TEST.COM
Connecting to jdbc:hive2://london.uk.com:10000/;principal=hive/london.uk.com@TEST.COM
Enter username for jdbc:hive2://london.uk.com:10000/;principal=hive/london.uk.com@TEST.COM:
Enter password for jdbc:hive2://london.uk.com:10000/;principal=hive/london.uk.com@TEST.COM:
Connected to: Apache Hive (version 1.2.1000.2.5.3.0-37)
Driver: Hive JDBC (version 1.2.1000.2.5.3.0-37)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://london.uk.com:10000/> show databases;
+----------------+--+
| database_name |
+----------------+--+
| default |
| moroto |
| jair |
+----------------+--+
3 rows selected (2.863 seconds)
0: jdbc:hive2://london.uk.com:10000> use jair;
No rows affected (0.097 seconds)
0: jdbc:hive2://london.uk.com:10000> show tables;
+-----------+--+
| tab_name |
+-----------+--+
| employee |
+-----------+--+
1 row selected (0.162 seconds)
0: jdbc:hive2://london.uk.com:10000/> select * from employee;
+--------------+----------------+------------------+-----------------------+--+
| employee.id | employee.name | employee.salary | employee.destination |
+--------------+----------------+------------------+-----------------------+--+
| 1201 | Gopal | 45000 | Technical manager |
| 1202 | Manisha | 45000 | Proof reader |
| 1203 | Masthanvali | 40000 | Technical writer |
| 1204 | Kiran | 40000 | Hr Admin |
| 1205 | Kranthi | 30000 | Op Admin |
| 1206 | Geoffrey | 45000 | DevOPS |
| 1207 | Salvatore | 30000 | IT Lead |
| 1208 | Dave | 40000 | Cleaner |
| 1206 | Geoffrey | 45000 | DevOPS |
| 1207 | Shelly | 30000 | IT Lead |
| 1208 | Dave | 40000 | Cleaner |
| 1209 | fid | 40000 | Cleaner |
| 1210 | Grid | 40000 | Clerk |
| 1211 | Dred | 40000 | Masseuse |
| 1212 | stad | 40000 | boss |
| 1213 | jair | 40000 | cook |
| 1214 | Jenelle | 40000 | handyman |
+--------------+----------------+------------------+-----------------------+--+
17 rows selected (7.172 seconds) Please let me know if you have any issue I will gladly help
... View more
09-30-2017
08:36 AM
@Benjamin Hopp Can you try to validate by doing the below steps $ kdestroy The grab a ticket as the nifi user [root@host ~]# sudo su - nifi_user
$ kinit
Password for nifi_user@NAM.xxxxxxxx.COM:
$ klist
Ticket cache: FILE:/tmp/krb5cc_49393
Default principal: nifi_user@NAM.xxxxxxxx.COM
Valid starting Expires Service principal
09/28/17 17:06:44 09/29/17 03:06:44 krbtgt/ NAM.xxxxxxxx.COM@NAM.xxxxxxxx.COM Tell me if that works
... View more
09-24-2017
01:08 PM
@Anish Gupta Here is an extract of a HCC solution https://nicholasmaillard.wordpress.com/2015/07/20/formatting-hdfs/ The above article describes right steps to format hdfs for HA enabled clusters as below: "The initial steps are very close
Stop the Hdfs service Start only the journal nodes (as they will need to be made aware of the formatting) On the first namenode (as user hdfs)
hadoop namenode -format hdfs namenode -initializeSharedEdits -force (for the journal nodes) hdfs zkfc -formatZK -force (to force zookeeper to reinitialise) restart that first namenode On the second namenode
hdfs namenode -bootstrapStandby -force (force synch with first namenode) On every datanode clear the data directory Restart the HDFS service These steps should help you to overcome the issue.
... View more
09-23-2017
11:47 AM
@Pathan Shehbaz Have you enabled the plugins? Lineage will show for tables you created after the Altas setup not before. Have you configured the cross component lineage see doc Can you create a new table "as select" from 2 tables and check whether you can see the lineage in Atlas?
... View more
09-21-2017
10:29 AM
@Ashnee Sharma Notice krb5-auth-dialog is optional Assuming you installed the KDC server yum -y install krb5-server krb5-libs krb5-auth-dialog Assuming you installed the KDC clients yum -y install krb5-auth-dialog krb5-workstation Your /etc/krb5.conf looks like below and copied to all the hosts in the cluster [logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = EXAMPLE.COM
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
[realms]
EXAMPLE.COM = {
kdc = kdc.examplecom
admin_server = kdc.examplecom
}
[domain_realm]
.example.com = EXAMPLE.COM
example.com = EXAMPLE.COM Your kdc.conf should resemble this [kdcdefaults]
kdc_ports = 88
kdc_tcp_ports = 88
[realms]
EXAMPLE.COM = {
#master_key_type = aes256-cts
acl_file = /var/kerberos/krb5kdc/kadm5.acl
dict_file = /usr/share/dict/words
admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
} You kadm5.acl in /var/kerberos/krb5kdc as below */admin@EAMPLE.COM * Can you create an admin principal as suit # kadmin.local -q "addprinc admin/admin"
Authenticating as principal admin/admin@EXAMPLE.COM with password.
WARNING: no policy specified for admin/admin@EXAMPLE.COM; defaulting to no policy
Enter password for principal "admin/admin@EXAMPLE.COM":
Re-enter password for principal "admin/admin@EXAMPLE.COM":
Principal "admin/admin@EXAMPLE.COM" created. This is the principal you should use for the Ambari Kerberos,make sure you started the appropriate daemons below Centos7/RHEL7 # systemctl start krb5kdc
# systemctl start kadmin Centos6/RHEL6 # systemctl start krb5kdc
# systemctl start kadmin All should be fine please let me know
... View more
09-20-2017
08:35 PM
@Jacqualin jasmin The simple answer is YES for MR This whole depends on the number of nodes in your cluster and the desired setup HA or NOT. Usually, there are Masternodes ,Datanodes and edge node (Clients Only) Masternode (if HA the recommended setup at least 3 nodes) - NAMENODE.
- JOURNALNODE.
- ZKFC.
- NODEMANAGER.
- HBASE_MASTER(Optional).
- HISTORYSERVER.
- APP_TIMELINE_SERVER.
- RESOURCEMANAGER.
- ZOOKEEPER_SERVER.(at least 2 )
- JOURNALNODE.
- MYSQL_SERVER.
- ZKFC.
- HIVE_SERVER.
- HIVE_METASTORE.
- WEBHCAT_SERVER.
- ZOOKEEPER_CLIENT. Datanode (As desired) - JOURNALNODE.
- NODEMANAGER.
- HBASE_REGIONSERVER(if Hbase installed).
- ZOOKEEPER_CLIENT.
- DATANODE Edgenode ( At least one) - YARN_CLIENT.
- ZOOKEEPER_CLIENT.
- RANGER_ADMIN.
- RANGER_USERSYNC.
- METRICS_MONITOR.
- METRICS_COLLECTOR.
- ZEPPELIN_MASTER.
- INFRA_SOLR.
- INFRA_SOLR_CLIENT.
- HBASE_CLIENT.
- HDFS_CLIENT.
- TEZ_CLIENT.
- YARN_CLIENT.
- HIVE_CLIENT.
- MAPREDUCE2_CLIENT In the above setup, some components are optional. See official Hortonworks document
... View more
09-20-2017
07:23 AM
@Pooja Kamle Do you have some error messages in hiveserver2.log?
... View more
09-19-2017
02:30 PM
@PJ Check and ensure the ambari-agent is stopped on the dead datanode.
... View more