Created 01-30-2016 10:53 AM
Dear Experts,
I installed HDF (nifi-1.1.1.0-12) using a user nifi (group hadoop) under /opt/nifi-1.1.1.0-12
Starting/Stopping the nifi service from bash works fine.
Afterwards I installed the Ambari Service for Nifi as outlined on https://github.com/abajwa-hw/ambari-nifi-service
Unfortunately whenever I start Nifi under Ambari it goes down without showing an error in '/var/lib/ambari-agent/data/errors-xxxx.txt'
the stout of ambari looks like the following:
---------------------------
stdout: 2016-01-30 10:31:01,040 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-01-30 10:31:01,040 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-01-30 10:31:01,040 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-01-30 10:31:01,061 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-01-30 10:31:01,062 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-01-30 10:31:01,082 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-01-30 10:31:01,082 - Ensuring that hadoop has the correct symlink structure 2016-01-30 10:31:01,083 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf 2016-01-30 10:31:01,192 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-01-30 10:31:01,192 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-01-30 10:31:01,192 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-01-30 10:31:01,214 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-01-30 10:31:01,214 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} 2016-01-30 10:31:01,235 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0') 2016-01-30 10:31:01,235 - Ensuring that hadoop has the correct symlink structure 2016-01-30 10:31:01,235 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf 2016-01-30 10:31:01,237 - Group['hadoop'] {} 2016-01-30 10:31:01,237 - Group['nifi'] {} 2016-01-30 10:31:01,238 - Group['users'] {} 2016-01-30 10:31:01,238 - User['zookeeper'] {'gid': 'hadoop', 'groups': [u'hadoop']} 2016-01-30 10:31:01,238 - User['ams'] {'gid': 'hadoop', 'groups': [u'hadoop']} 2016-01-30 10:31:01,239 - User['ambari-qa'] {'gid': 'hadoop', 'groups': [u'users']} 2016-01-30 10:31:01,239 - User['kafka'] {'gid': 'hadoop', 'groups': [u'hadoop']} 2016-01-30 10:31:01,240 - User['hdfs'] {'gid': 'hadoop', 'groups': [u'hadoop']} 2016-01-30 10:31:01,240 - User['yarn'] {'gid': 'hadoop', 'groups': [u'hadoop']} 2016-01-30 10:31:01,241 - User['nifi'] {'gid': 'hadoop', 'groups': [u'hadoop']} 2016-01-30 10:31:01,242 - User['mapred'] {'gid': 'hadoop', 'groups': [u'hadoop']} 2016-01-30 10:31:01,242 - User['hbase'] {'gid': 'hadoop', 'groups': [u'hadoop']} 2016-01-30 10:31:01,243 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2016-01-30 10:31:01,244 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'} 2016-01-30 10:31:01,248 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if 2016-01-30 10:31:01,248 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'recursive': True, 'mode': 0775, 'cd_access': 'a'} 2016-01-30 10:31:01,249 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555} 2016-01-30 10:31:01,250 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'} 2016-01-30 10:31:01,254 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if 2016-01-30 10:31:01,254 - Group['hdfs'] {'ignore_failures': False} 2016-01-30 10:31:01,254 - User['hdfs'] {'ignore_failures': False, 'groups': [u'hadoop', u'hdfs']} 2016-01-30 10:31:01,255 - Directory['/etc/hadoop'] {'mode': 0755} 2016-01-30 10:31:01,266 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'} 2016-01-30 10:31:01,267 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 0777} 2016-01-30 10:31:01,277 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'} 2016-01-30 10:31:01,285 - Skipping Execute[('setenforce', '0')] due to only_if 2016-01-30 10:31:01,285 - Directory['/var/log/hadoop'] {'owner': 'root', 'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'} 2016-01-30 10:31:01,287 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True, 'cd_access': 'a'} 2016-01-30 10:31:01,288 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True, 'cd_access': 'a'} 2016-01-30 10:31:01,291 - File['/usr/hdp/current/hadoop-client/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'} 2016-01-30 10:31:01,292 - File['/usr/hdp/current/hadoop-client/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'} 2016-01-30 10:31:01,293 - File['/usr/hdp/current/hadoop-client/conf/log4j.properties'] {'content': ..., 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644} 2016-01-30 10:31:01,300 - File['/usr/hdp/current/hadoop-client/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'} 2016-01-30 10:31:01,301 - File['/usr/hdp/current/hadoop-client/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755} 2016-01-30 10:31:01,305 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'} 2016-01-30 10:31:01,308 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755} 2016-01-30 10:31:01,488 - File['/opt/nifi-1.1.1.0-12/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'} 2016-01-30 10:31:01,491 - File['/opt/nifi-1.1.1.0-12/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'} 2016-01-30 10:31:01,495 - File['/opt/nifi-1.1.1.0-12/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'} 2016-01-30 10:31:01,496 - Execute['echo pid file /var/run/nifi/nifi.pid'] {} 2016-01-30 10:31:01,498 - Execute['echo JAVA_HOME=/usr/jdk64/jdk1.8.0_60'] {} 2016-01-30 10:31:01,501 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_60;/opt/nifi-1.1.1.0-12/bin/nifi.sh start >> /var/log/nifi/nifi-setup.log'] {'user': 'nifi'} 2016-01-30 10:31:04,558 - Execute['cat /opt/nifi-1.1.1.0-12/bin/nifi.pid | grep pid | sed 's/pid=\(\.*\)/\1/' > /var/run/nifi/nifi.pid'] {} 2016-01-30 10:31:04,567 - Execute['chown nifi:nifi /var/run/nifi/nifi.pid'] {} ---------------------------
Again, when I start nifi from bash it works as expacted.
Any help on how to fix this or better trace the problem is highly appreciated!
br,
Rainer
Created 01-30-2016 06:56 PM
@Rainer Geissendoerfer Others have encountered similar issues with the Nifi service (usually on Java 😎 as well but I have not been able to consistently reproduce it yet. From what I have seen of this issue, if you run the steps to start Nifi manually via CLI, it works (but for some reason not from Ambari). You can try below manual commands to start Nifi and populate the pid file so that Ambari can track its status:
su - nifi export JAVA_HOME=/usr/java/default /opt/nifi-1.1.1.0-12/bin/nifi.sh start >> /var/log/nifi/nifi-setup.log cat /opt/nifi-1.1.1.0-12/bin/nifi.pid | grep pid | sed 's/pid=\(\.*\)/\1/' > /var/run/nifi/nifi.pid #run below as root chown nifi:nifi /var/run/nifi/nifi.pid
If you are encountering the same issue, could you provide the below details so we can try to reproduce it:
Created 02-15-2016 01:46 AM