Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

nifi manages the supervisor.conf file?

avatar
Rising Star

we need to know if nifi is responsible for the file below:

 

/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE/supervisor.conf

 

because this file contains information that we are not finding in the nifi settings in the CDM

9 REPLIES 9

avatar

Hi @yagoaparecidoti , in general, the "supervisor.conf" in the process directory (actually the whole process directory) is prepared by Cloudera Manager (server) before starting a process (CM server sends the whole package of information including config files to the CM agent which extracts it in a new process directory). The supervisor.conf file contains all the environment and command related information which is needed for the Supervisor daemon to start the process. There might be some default values taken from the cluster or from the service type.
Do you have some specific questions about it?

avatar
Rising Star

hi @mszurap, how are you?

 

right, in the file "supervisor.conf" there is a parameter called "KRB5CCNAME=/tmp/kbr5cc_dffe", I would like to know where this information is coming from.

avatar

Hi, I'm doing well, thank you, hope you're good too.

That property usually points to a relative path - which exists in the process directory:

KRB5CCNAME='krb5cc_cldr'

if that's not the case, I would look into whether the root user's (or maybe the "cloudera-scm" user's) .bashrc file has overridden that KRB5CCNAME environment variable by any chance.

avatar
Rising Star

hi @mszurap , 

 

in the file "/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE/supervisor.conf" has exactly the parameter "KRB5CCNAME='krb5cc_cldr'"

 

do we need to know where he is getting this information from?

 

we already looked in the .bashrc and .bash_profile of the root user and other users, nothing was found.

avatar

So is it "/tmp/kbr5cc_dffe" or "krb5cc_cldr"? Or where do you see the "KRB5CCNAME=/tmp/kbr5cc_dffe"? 

The "krb5cc_cldr" is used for all (? not sure, but all which I've quickly verified had that) services - we can say it's hardcoded - it is anyways "private" to the process itself, that holds the kerberos ticket cache which only that process is using (and renewing if needed).

avatar
Rising Star

hi, @mszurap,

 

in the file "/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE/supervisor.conf" has the following parameter "KRB5CCNAME=krb5cc_cldr"

 

the "KRB5CCNAME" parameter is in the "environment" line of the "/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE/supervisor.conf" file as shown below:

 

environment=CDH_PARQUET_HOME='/usr/lib/parquet',CDH_MR2_HOME='/usr/lib/hadoop-mapreduce',CMF_PACKAGE_DIR='/opt/cloudera/cm-agent/service',CM_STATUS_CODES='STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MODE',MGMT_HOME='/opt/cloudera/cm',CGROUP_ROOT_CPU='/sys/fs/cgroup/cpu,cpuacct',CDH_IMPALA_HOME='/usr/lib/impala',CDH_YARN_HOME='/usr/lib/hadoop-yarn',nifi_service_principal='nifi/host-nifi-16.domain.local@TESTE.INT.BR',CGROUP_GROUP_CPU='system.slice/cloudera-scm-agent.service',CSD_JAVA_OPTS='-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/nifi_nifi-NIFI_NODE-ae57679df54eef6fea3afc54d3cb40ad_pid{{PID}}.hprof -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh',CDH_HDFS_HOME='/usr/lib/hadoop-hdfs',CSD_HOST='host-nifi-16.domain.local',HADOOP_CREDSTORE_PASSWORD='6d1ddiyksatrp1nwubz27h5jp',PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin',CDH_HUE_PLUGINS_HOME='/usr/lib/hadoop',PARCEL_DIRNAMES='CDH-6.3.4-1.cdh6.3.4.p0.6751098:CFM-1.0.1.0',CDH_HADOOP_BIN='/usr/bin/hadoop',KEYTRUSTEE_KP_HOME='/usr/share/keytrustee-keyprovider',CDH_KUDU_HOME='/usr/lib/kudu',CLOUDERA_ORACLE_CONNECTOR_JAR='/usr/share/java/oracle-connector-java.jar',CONF_DIR='/var/run/cloudera-scm-agent/process/24750-nifi-NIFI_NODE',CDH_SQOOP2_HOME='/usr/lib/sqoop2',KEYTRUSTEE_SERVER_HOME='/usr/lib/keytrustee-server',JSVC_HOME='/usr/libexec/bigtop-utils',HOME='/var/lib/nifi',HIVE_DEFAULT_XML='/etc/hive/conf.dist/hive-default.xml',CUSTOM_JAVA_HOME='/usr/java/default',JAVA_HOME='/usr/java/jdk1.8.0_162',CLOUDERA_POSTGRESQL_JDBC_JAR='/opt/cloudera/cm/lib/postgresql-42.2.14.jre7.jar',CGROUP_GROUP_CPUACCT='system.slice/cloudera-scm-agent.service',CM_CSD_SCRIPT='scripts/control.sh',PARCELS_ROOT='/opt/cloudera/parcels',CDH_KMS_HOME='/usr/lib/hadoop-kms',ZK_QUORUM='host-nifi-14.domain.local:2183,host-nifi-15.domain.local:2183,host-nifi-16.domain.local:2183',CDH_HBASE_HOME='/usr/lib/hbase',CDH_SQOOP_HOME='/usr/lib/sqoop',CDH_HCAT_HOME='/usr/lib/hcatalog',CDH_OOZIE_HOME='/usr/lib/oozie',CDH_HIVE_HOME='/usr/lib/hive',CDH_ZOOKEEPER_HOME='/usr/lib/zookeeper',CDH_HUE_HOME='/usr/lib/hue',nifi_spnego_principal='HTTP/host-nifi-16.domain.local@TESTE.INT.BR',CGROUP_GROUP_DEVICES='system.slice',CGROUP_ROOT_MEMORY='/sys/fs/cgroup/memory',CLOUDERA_MYSQL_CONNECTOR_JAR='/usr/share/java/mysql-connector-java.jar',CDH_SPARK_HOME='/usr/lib/spark',CGROUP_GROUP_MEMORY='system.slice/cloudera-scm-agent.service',KRB5CCNAME='krb5cc_cldr',CDH_MR1_HOME='/usr/lib/hadoop-0.20-mapreduce',TOMCAT_HOME='/usr/lib/bigtop-tomcat',CDH_PIG_HOME='/usr/lib/pig',CDH_SENTRY_HOME='/usr/lib/sentry',CDH_VERSION='6',CGROUP_GROUP_BLKIO='system.slice/cloudera-scm-agent.service',NIFI_LOG_DIR='/var/log/nifi',CGROUP_ROOT_BLKIO='/sys/fs/cgroup/blkio',CDH_CRUNCH_HOME='/usr/lib/crunch',CDH_LLAMA_HOME='/usr/lib/llama/',CDH_HTTPFS_HOME='/usr/lib/hadoop-httpfs',CDH_HADOOP_HOME='/usr/lib/hadoop',ORACLE_HOME='/usr/share/oracle/instantclient',CMF_SERVER_ROOT='/opt/cloudera/cm',WEBHCAT_DEFAULT_XML='/etc/hive-webhcat/conf.dist/webhcat-default.xml',CDH_KAFKA_HOME='/usr/lib/kafka',CGROUP_ROOT_DEVICES='/sys/fs/cgroup/devices',CDH_HBASE_INDEXER_HOME='/usr/lib/hbase-solr',SCM_DEFINES_SCRIPTS='/opt/cloudera/parcels/CDH-6.3.4-1.cdh6.3.4.p0.6751098/meta/cdh_env.sh:/opt/cloudera/parcels/CFM-1.0.1.0/meta/nifi_bundle_env.sh',CDH_SOLR_HOME='/usr/lib/solr',CGROUP_ROOT_CPUACCT='/sys/fs/cgroup/cpu,cpuacct',CDH_FLUME_HOME='/usr/lib/flume-ng'

 

so the cloudera manager itself is distributing this file "krb5cc_cldr"?

 

PS: this file is in /tmp/krb5cc_cldr

avatar
Expert Contributor

KRB5CCNAME is an environment variable that Cloudera Manager sets, it'd change the filename for newly created kerberos tickets from the default "krbc5cc_<uid>" to krb5cc_cldr.

 https://web.mit.edu/kerberos/krb5-1.6/krb5-1.6/doc/klist.html

If needed, you can unset it by adding the following to CM > NiFi > Configs > Filter for "nifi-env.sh"
unset KRB5CCNAME
 

avatar
Rising Star

hi @bbahamondes

 

thank you for the explanation

avatar
Community Manager

@yagoaparecidoti If your issue is resolved, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

 

 

 

Screen Shot 2019-08-06 at 1.54.47 PM.png

 

 


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.