I am having the same error but I didn't understand the solution, can you please explain it. Thank you. ... View more

Did you know the solution for that problem? @renuka20  ... View more

Thanks for replying @Shelton @EricL .   We only have MIT Kerberos and doesn't have any Active Directory.   These are the outputs, we have two KDC setup for each cluster but they are not replicating to each other. We have one more cluster with same REALM NAME but for them also there are two KDC but there is no replication happening. Not only Hive service, even if I want to install  extra Node Manager I am getting the same error.           [root@spectra-xx-z15p xxxxxxx]# klist -kt /etc/security/keytabs/hive.service.keytabKeytab name: FILE:/etc/security/keytabs/hive.service.keytab klist: Key table file '/etc/security/keytabs/hive.service.keytab' not found while starting keytab scan                 [root@spectra-xx-z15p xxxxxxx]# cat /etc/krb5.conf # Other applications require this directory to perform krb5 configuration. includedir /etc/krb5.conf.d/ # This file is provided by the CADA client package # Previous versions of this file can be found in /opt/cada/backups/ # $Id: krb5.conf 10925 2010-05-14 19:55:23Z xxxxxxx $ [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] default_realm = SPECTRA.XXXXXXX.NET dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h forwardable = yes renew_lifetime = 180d [realms] SPECTRA.XXXXXXX.NET = { kdc = spectra-xx-z39p.sys.xxxxxxx.net kdc = spectra-xx-z40p.sys.xxxxxxx.net admin_server = spectra-po-z39p.sys.xxxxxxx.net } XXXXXXX.NET = { kdc = kdc-m.xxxxxxx.net:88 kdc = kdc.xxxxxxx.net:88 admin_server = kdc-m.xxxxxxx.net:749 } [domain_realm] .xxxxxxx.net = XXXXXXX.NET xxxxxxx.net = XXXXXXX.NET .xxxxxxx.com = XXXXXXX.NET xxxxxxx.com = XXXXXXX.NET .sys.xxxxxxx.net = SPECTRA.XXXXXXX.NET sys.xxxxxx.net = SPECTRA.xxxxxx.NET [appdefaults] pam = { debug = false forwardable = true krb4_convert = false chpw_prompt = sshd } pkinit = { allow_pkinit = false } Below are the hive.keytab outputs from hive metastore and hive server. [root@spectra-xx-z15p process]# cd /var/run/cloudera-scm-agent/process/17710-hive-HIVEMETXXTORE/ [root@spectra-xx-z15p 17710-hive-HIVEMETASTORE]# ls cloudera-monitor.properties core-site.xml hive.keytab hive-site.xml process_timestamp sentry-site.xml yarn-conf cloudera-stack-monitor.properties creds.localjceks hive-log4j.properties logs redaction-rules.json service-metrics.properties [root@spectra-xx-z15p 17710-hive-HIVEMETASTORE]# klist -kt hive.keytab Keytab name: FILE:hive.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET [root@spectra-xx-z15p 17710-hive-HIVEMETASTORE]# cd /var/run/cloudera-scm-agent/process/17709-hive-HIVESERVER2/ [root@spectra-xx-z15p 17709-hive-HIVESERVER2]# ls cloudera-monitor.properties hive.keytab logs process_timestamp service-metrics.properties cloudera-stack-monitor.properties hive-log4j.properties navigator.client.properties redaction-rules.json yarn-conf core-site.xml hive-site.xml navigator.lineage.client.properties sentry-site.xml [root@spectra-xx-z15p 17709-hive-HIVESERVER2]# klist -kt hive.keytab Keytab name: FILE:hive.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET [root@spectra-xx-z15p 17709-hive-HIVESERVER2]#   ... View more

Hello @bgooley    I am talking about /var/lib/cloudera-host-monitor and /var/lib/cloudera-service-monitor. They occupied almost 20GB of the space and I want to delete those files. By below screenshot you will know what these folders contain. I need space, if charts history is gone I am fine. Thank you. ... View more

Hello @bgooley      I am talking about /var/lib/cloudera-service-monitor and /var/lib/clouder-host-monitor, both of these occupied almost 20GB, I want to delete the files in both of them. But not sure what will happen. Its a production cluster, so dont want impact on it by my actions.   cloudera-service-monitor has impala, reports, subject_record, ts and yarn folders. I want to delete the files in subject_record and ts floders, because they occupied 9GB. Is that ok to delete them?    The ts folder has below files: stream               ts_stream_rollup_PT21600S   ts_stream_rollup_PT600S     ts_stream_rollup_PT86400S   ts_type_rollup_PT3600S   ts_type_rollup_PT604800S   type ts_entity_metadata   ts_stream_rollup_PT3600S   ts_stream_rollup_PT604800S   ts_type_rollup_PT21600S     ts_type_rollup_PT600S   ts_type_rollup_PT86400S   The subject_record folder has below files: subject_ts   ts_subject   The cloudera-host-monitor has below floders: subject_record   ts Like in cloudera-service-monitor, almost same files are present in both the subject_record and ts floders.   Is that ok to manually delete them? if chart data is gone it is fine but we need space.  ... View more