question Re: nifi manages the supervisor.conf file? in Support Questions

nifi manages the supervisor.conf file?

yagoaparecidoti — Wed, 13 Apr 2022 20:27:08 GMT

we need to know if nifi is responsible for the file below:

/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE/supervisor.conf

because this file contains information that we are not finding in the nifi settings in the CDM

Re: nifi manages the supervisor.conf file?

mszurap — Thu, 14 Apr 2022 08:45:54 GMT

Hi @yagoaparecidoti , in general, the "supervisor.conf" in the process directory (actually the whole process directory) is prepared by Cloudera Manager (server) before starting a process (CM server sends the whole package of information including config files to the CM agent which extracts it in a new process directory). The supervisor.conf file contains all the environment and command related information which is needed for the Supervisor daemon to start the process. There might be some default values taken from the cluster or from the service type.
Do you have some specific questions about it?

Re: nifi manages the supervisor.conf file?

yagoaparecidoti — Thu, 14 Apr 2022 13:57:30 GMT

hi @mszurap, how are you?

right, in the file "supervisor.conf" there is a parameter called "KRB5CCNAME=/tmp/kbr5cc_dffe", I would like to know where this information is coming from.

Re: nifi manages the supervisor.conf file?

mszurap — Thu, 14 Apr 2022 14:42:22 GMT

Hi, I'm doing well, thank you, hope you're good too.

That property usually points to a relative path - which exists in the process directory:

KRB5CCNAME='krb5cc_cldr'

if that's not the case, I would look into whether the root user's (or maybe the "cloudera-scm" user's) .bashrc file has overridden that KRB5CCNAME environment variable by any chance.

Re: nifi manages the supervisor.conf file?

yagoaparecidoti — Thu, 14 Apr 2022 15:16:35 GMT

hi @mszurap ,

in the file "/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE/supervisor.conf" has exactly the parameter "KRB5CCNAME='krb5cc_cldr'"

do we need to know where he is getting this information from?

we already looked in the .bashrc and .bash_profile of the root user and other users, nothing was found.

Re: nifi manages the supervisor.conf file?

mszurap — Thu, 14 Apr 2022 16:16:55 GMT

So is it "/tmp/kbr5cc_dffe" or "krb5cc_cldr"? Or where do you see the "KRB5CCNAME=/tmp/kbr5cc_dffe"?

The "krb5cc_cldr" is used for all (? not sure, but all which I've quickly verified had that) services - we can say it's hardcoded - it is anyways "private" to the process itself, that holds the kerberos ticket cache which only that process is using (and renewing if needed).

Re: nifi manages the supervisor.conf file?

yagoaparecidoti — Thu, 14 Apr 2022 16:30:01 GMT

hi, @mszurap,

in the file "/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE/supervisor.conf" has the following parameter "KRB5CCNAME=krb5cc_cldr"

the "KRB5CCNAME" parameter is in the "environment" line of the "/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE/supervisor.conf" file as shown below:

environment=CDH_PARQUET_HOME='/usr/lib/parquet',CDH_MR2_HOME='/usr/lib/hadoop-mapreduce',CMF_PACKAGE_DIR='/opt/cloudera/cm-agent/service',CM_STATUS_CODES='STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE',MGMT_HOME='/opt/cloudera/cm',CGROUP_ROOT_CPU='/sys/fs/cgroup/cpu,cpuacct',CDH_IMPALA_HOME='/usr/lib/impala',CDH_YARN_HOME='/usr/lib/hadoop-yarn',nifi_service_principal='nifi/host-nifi-16.domain.local@TESTE.INT.BR',CGROUP_GROUP_CPU='system.slice/cloudera-scm-agent.service',CSD_JAVA_OPTS='-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/nifi_nifi-NIFI_NODE-ae57679df54eef6fea3afc54d3cb40ad_pid{{PID}}.hprof -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh',CDH_HDFS_HOME='/usr/lib/hadoop-hdfs',CSD_HOST='host-nifi-16.domain.local',HADOOP_CREDSTORE_PASSWORD='6d1ddiyksatrp1nwubz27h5jp',PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin',CDH_HUE_PLUGINS_HOME='/usr/lib/hadoop',PARCEL_DIRNAMES='CDH-6.3.4-1.cdh6.3.4.p0.6751098:CFM-1.0.1.0',CDH_HADOOP_BIN='/usr/bin/hadoop',KEYTRUSTEE_KP_HOME='/usr/share/keytrustee-keyprovider',CDH_KUDU_HOME='/usr/lib/kudu',CLOUDERA_ORACLE_CONNECTOR_JAR='/usr/share/java/oracle-connector-java.jar',CONF_DIR='/var/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE',CDH_SQOOP2_HOME='/usr/lib/sqoop2',KEYTRUSTEE_SERVER_HOME='/usr/lib/keytrustee-server',JSVC_HOME='/usr/libexec/bigtop-utils',HOME='/var/lib/nifi',HIVE_DEFAULT_XML='/etc/hive/conf.dist/hive-default.xml',CUSTOM_JAVA_HOME='/usr/java/default',JAVA_HOME='/usr/java/jdk1.8.0_162',CLOUDERA_POSTGRESQL_JDBC_JAR='/opt/cloudera/cm/lib/postgresql-42.2.14.jre7.jar',CGROUP_GROUP_CPUACCT='system.slice/cloudera-scm-agent.service',CM_CSD_SCRIPT='scripts/control.sh',PARCELS_ROOT='/opt/cloudera/parcels',CDH_KMS_HOME='/usr/lib/hadoop-kms',ZK_QUORUM='host-nifi-14.domain.local:2183,host-nifi-15.domain.local:2183,host-nifi-16.domain.local:2183',CDH_HBASE_HOME='/usr/lib/hbase',CDH_SQOOP_HOME='/usr/lib/sqoop',CDH_HCAT_HOME='/usr/lib/hcatalog',CDH_OOZIE_HOME='/usr/lib/oozie',CDH_HIVE_HOME='/usr/lib/hive',CDH_ZOOKEEPER_HOME='/usr/lib/zookeeper',CDH_HUE_HOME='/usr/lib/hue',nifi_spnego_principal='HTTP/host-nifi-16.domain.local@TESTE.INT.BR',CGROUP_GROUP_DEVICES='system.slice',CGROUP_ROOT_MEMORY='/sys/fs/cgroup/memory',CLOUDERA_MYSQL_CONNECTOR_JAR='/usr/share/java/mysql-connector-java.jar',CDH_SPARK_HOME='/usr/lib/spark',CGROUP_GROUP_MEMORY='system.slice/cloudera-scm-agent.service',KRB5CCNAME='krb5cc_cldr',CDH_MR1_HOME='/usr/lib/hadoop-0.20-mapreduce',TOMCAT_HOME='/usr/lib/bigtop-tomcat',CDH_PIG_HOME='/usr/lib/pig',CDH_SENTRY_HOME='/usr/lib/sentry',CDH_VERSION='6',CGROUP_GROUP_BLKIO='system.slice/cloudera-scm-agent.service',NIFI_LOG_DIR='/var/log/nifi',CGROUP_ROOT_BLKIO='/sys/fs/cgroup/blkio',CDH_CRUNCH_HOME='/usr/lib/crunch',CDH_LLAMA_HOME='/usr/lib/llama/',CDH_HTTPFS_HOME='/usr/lib/hadoop-httpfs',CDH_HADOOP_HOME='/usr/lib/hadoop',ORACLE_HOME='/usr/share/oracle/instantclient',CMF_SERVER_ROOT='/opt/cloudera/cm',WEBHCAT_DEFAULT_XML='/etc/hive-webhcat/conf.dist/webhcat-default.xml',CDH_KAFKA_HOME='/usr/lib/kafka',CGROUP_ROOT_DEVICES='/sys/fs/cgroup/devices',CDH_HBASE_INDEXER_HOME='/usr/lib/hbase-solr',SCM_DEFINES_SCRIPTS='/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/meta/cdh_env.sh:/opt/cloudera/parcels/CFM-1.0.1.0/meta/nifi_bundle_env.sh',CDH_SOLR_HOME='/usr/lib/solr',CGROUP_ROOT_CPUACCT='/sys/fs/cgroup/cpu,cpuacct',CDH_FLUME_HOME='/usr/lib/flume-ng'

so the cloudera manager itself is distributing this file "krb5cc_cldr"?

PS: this file is in /tmp/krb5cc_cldr

Re: nifi manages the supervisor.conf file?

bbahamondes — Thu, 14 Apr 2022 19:51:16 GMT

KRB5CCNAME is an environment variable that Cloudera Manager sets, it'd change the filename for newly created kerberos tickets from the default "krbc5cc_<uid>" to krb5cc_cldr.

https://web.mit.edu/kerberos/krb5-1.6/krb5-1.6/doc/klist.html

If needed, you can unset it by adding the following to CM > NiFi > Configs > Filter for "nifi-env.sh"
unset KRB5CCNAME

Re: nifi manages the supervisor.conf file?

yagoaparecidoti — Thu, 14 Apr 2022 20:35:32 GMT

hi @bbahamondes,

thank you for the explanation

Re: nifi manages the supervisor.conf file?

cjervis — Fri, 15 Apr 2022 13:48:42 GMT

@yagoaparecidoti If your issue is resolved, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.