I am having the same error but I didn't understand the solution, can you please explain it. Thank you. ... View more

Thanks for replying @Shelton @EricL .   We only have MIT Kerberos and doesn't have any Active Directory.   These are the outputs, we have two KDC setup for each cluster but they are not replicating to each other. We have one more cluster with same REALM NAME but for them also there are two KDC but there is no replication happening. Not only Hive service, even if I want to install  extra Node Manager I am getting the same error.           [root@spectra-xx-z15p xxxxxxx]# klist -kt /etc/security/keytabs/hive.service.keytabKeytab name: FILE:/etc/security/keytabs/hive.service.keytab klist: Key table file '/etc/security/keytabs/hive.service.keytab' not found while starting keytab scan                 [root@spectra-xx-z15p xxxxxxx]# cat /etc/krb5.conf # Other applications require this directory to perform krb5 configuration. includedir /etc/krb5.conf.d/ # This file is provided by the CADA client package # Previous versions of this file can be found in /opt/cada/backups/ # $Id: krb5.conf 10925 2010-05-14 19:55:23Z xxxxxxx $ [logging] default = FILE:/var/log/krb5libs.log kdc = FILE:/var/log/krb5kdc.log admin_server = FILE:/var/log/kadmind.log [libdefaults] default_realm = SPECTRA.XXXXXXX.NET dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h forwardable = yes renew_lifetime = 180d [realms] SPECTRA.XXXXXXX.NET = { kdc = spectra-xx-z39p.sys.xxxxxxx.net kdc = spectra-xx-z40p.sys.xxxxxxx.net admin_server = spectra-po-z39p.sys.xxxxxxx.net } XXXXXXX.NET = { kdc = kdc-m.xxxxxxx.net:88 kdc = kdc.xxxxxxx.net:88 admin_server = kdc-m.xxxxxxx.net:749 } [domain_realm] .xxxxxxx.net = XXXXXXX.NET xxxxxxx.net = XXXXXXX.NET .xxxxxxx.com = XXXXXXX.NET xxxxxxx.com = XXXXXXX.NET .sys.xxxxxxx.net = SPECTRA.XXXXXXX.NET sys.xxxxxx.net = SPECTRA.xxxxxx.NET [appdefaults] pam = { debug = false forwardable = true krb4_convert = false chpw_prompt = sshd } pkinit = { allow_pkinit = false } Below are the hive.keytab outputs from hive metastore and hive server. [root@spectra-xx-z15p process]# cd /var/run/cloudera-scm-agent/process/17710-hive-HIVEMETXXTORE/ [root@spectra-xx-z15p 17710-hive-HIVEMETASTORE]# ls cloudera-monitor.properties core-site.xml hive.keytab hive-site.xml process_timestamp sentry-site.xml yarn-conf cloudera-stack-monitor.properties creds.localjceks hive-log4j.properties logs redaction-rules.json service-metrics.properties [root@spectra-xx-z15p 17710-hive-HIVEMETASTORE]# klist -kt hive.keytab Keytab name: FILE:hive.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET [root@spectra-xx-z15p 17710-hive-HIVEMETASTORE]# cd /var/run/cloudera-scm-agent/process/17709-hive-HIVESERVER2/ [root@spectra-xx-z15p 17709-hive-HIVESERVER2]# ls cloudera-monitor.properties hive.keytab logs process_timestamp service-metrics.properties cloudera-stack-monitor.properties hive-log4j.properties navigator.client.properties redaction-rules.json yarn-conf core-site.xml hive-site.xml navigator.lineage.client.properties sentry-site.xml [root@spectra-xx-z15p 17709-hive-HIVESERVER2]# klist -kt hive.keytab Keytab name: FILE:hive.keytab KVNO Timestamp Principal ---- ------------------- ------------------------------------------------------ 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 hive/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET 2 08/29/2019 16:57:43 HTTP/spectra-xx-z15p.sys.xxxxxxx.net@SPECTRA.XXXXXXX.NET [root@spectra-xx-z15p 17709-hive-HIVESERVER2]#   ... View more

Hello @bgooley    I am talking about /var/lib/cloudera-host-monitor and /var/lib/cloudera-service-monitor. They occupied almost 20GB of the space and I want to delete those files. By below screenshot you will know what these folders contain. I need space, if charts history is gone I am fine. Thank you. ... View more

Hello @bgooley      I am talking about /var/lib/cloudera-service-monitor and /var/lib/clouder-host-monitor, both of these occupied almost 20GB, I want to delete the files in both of them. But not sure what will happen. Its a production cluster, so dont want impact on it by my actions.   cloudera-service-monitor has impala, reports, subject_record, ts and yarn folders. I want to delete the files in subject_record and ts floders, because they occupied 9GB. Is that ok to delete them?    The ts folder has below files: stream              ts_stream_rollup_PT21600S  ts_stream_rollup_PT600S     ts_stream_rollup_PT86400S  ts_type_rollup_PT3600S  ts_type_rollup_PT604800S  type ts_entity_metadata  ts_stream_rollup_PT3600S   ts_stream_rollup_PT604800S  ts_type_rollup_PT21600S    ts_type_rollup_PT600S   ts_type_rollup_PT86400S   The subject_record folder has below files: subject_ts  ts_subject   The cloudera-host-monitor has below floders: subject_record  ts Like in cloudera-service-monitor, almost same files are present in both the subject_record and ts floders.   Is that ok to manually delete them? if chart data is gone it is fine but we need space.  ... View more