Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

HDF (nifi-1.1.1.0-12) doesn't start using ambari nifi service

avatar
author-rank rainer_geissend
Expert Contributor

Created ‎01-30-2016 10:53 AM

Dear Experts,

I installed HDF (nifi-1.1.1.0-12) using a user nifi (group hadoop) under /opt/nifi-1.1.1.0-12

Starting/Stopping the nifi service from bash works fine.

Afterwards I installed the Ambari Service for Nifi as outlined on https://github.com/abajwa-hw/ambari-nifi-service

Unfortunately whenever I start Nifi under Ambari it goes down without showing an error in '/var/lib/ambari-agent/data/errors-xxxx.txt'

the stout of ambari looks like the following:

---------------------------

stdout:
2016-01-30 10:31:01,040 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485
2016-01-30 10:31:01,040 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0
2016-01-30 10:31:01,040 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-01-30 10:31:01,061 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '')
2016-01-30 10:31:01,062 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-01-30 10:31:01,082 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0')
2016-01-30 10:31:01,082 - Ensuring that hadoop has the correct symlink structure
2016-01-30 10:31:01,083 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-01-30 10:31:01,192 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485
2016-01-30 10:31:01,192 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0
2016-01-30 10:31:01,192 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-01-30 10:31:01,214 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '')
2016-01-30 10:31:01,214 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-01-30 10:31:01,235 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0')
2016-01-30 10:31:01,235 - Ensuring that hadoop has the correct symlink structure
2016-01-30 10:31:01,235 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-01-30 10:31:01,237 - Group['hadoop'] {}
2016-01-30 10:31:01,237 - Group['nifi'] {}
2016-01-30 10:31:01,238 - Group['users'] {}
2016-01-30 10:31:01,238 - User['zookeeper'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,238 - User['ams'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,239 - User['ambari-qa'] {'gid': 'hadoop', 'groups': [u'users']}
2016-01-30 10:31:01,239 - User['kafka'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,240 - User['hdfs'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,240 - User['yarn'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,241 - User['nifi'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,242 - User['mapred'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,242 - User['hbase'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,243 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-01-30 10:31:01,244 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2016-01-30 10:31:01,248 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2016-01-30 10:31:01,248 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'recursive': True, 'mode': 0775, 'cd_access': 'a'}
2016-01-30 10:31:01,249 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-01-30 10:31:01,250 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2016-01-30 10:31:01,254 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if
2016-01-30 10:31:01,254 - Group['hdfs'] {'ignore_failures': False}
2016-01-30 10:31:01,254 - User['hdfs'] {'ignore_failures': False, 'groups': [u'hadoop', u'hdfs']}
2016-01-30 10:31:01,255 - Directory['/etc/hadoop'] {'mode': 0755}
2016-01-30 10:31:01,266 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2016-01-30 10:31:01,267 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 0777}
2016-01-30 10:31:01,277 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2016-01-30 10:31:01,285 - Skipping Execute[('setenforce', '0')] due to only_if
2016-01-30 10:31:01,285 - Directory['/var/log/hadoop'] {'owner': 'root', 'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2016-01-30 10:31:01,287 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True, 'cd_access': 'a'}
2016-01-30 10:31:01,288 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True, 'cd_access': 'a'}
2016-01-30 10:31:01,291 - File['/usr/hdp/current/hadoop-client/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2016-01-30 10:31:01,292 - File['/usr/hdp/current/hadoop-client/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2016-01-30 10:31:01,293 - File['/usr/hdp/current/hadoop-client/conf/log4j.properties'] {'content': ..., 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2016-01-30 10:31:01,300 - File['/usr/hdp/current/hadoop-client/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
2016-01-30 10:31:01,301 - File['/usr/hdp/current/hadoop-client/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2016-01-30 10:31:01,305 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'}
2016-01-30 10:31:01,308 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2016-01-30 10:31:01,488 - File['/opt/nifi-1.1.1.0-12/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 10:31:01,491 - File['/opt/nifi-1.1.1.0-12/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 10:31:01,495 - File['/opt/nifi-1.1.1.0-12/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 10:31:01,496 - Execute['echo pid file /var/run/nifi/nifi.pid'] {}
2016-01-30 10:31:01,498 - Execute['echo JAVA_HOME=/usr/jdk64/jdk1.8.0_60'] {}
2016-01-30 10:31:01,501 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_60;/opt/nifi-1.1.1.0-12/bin/nifi.sh start >> /var/log/nifi/nifi-setup.log'] {'user': 'nifi'}
2016-01-30 10:31:04,558 - Execute['cat /opt/nifi-1.1.1.0-12/bin/nifi.pid | grep pid | sed 's/pid=\(\.*\)/\1/' > /var/run/nifi/nifi.pid'] {}
2016-01-30 10:31:04,567 - Execute['chown nifi:nifi /var/run/nifi/nifi.pid'] {}
---------------------------

Again, when I start nifi from bash it works as expacted.

Any help on how to fix this or better trace the problem is highly appreciated!

br,

Rainer

8,532 Views
1 ACCEPTED SOLUTION

avatar
author-rank abajwa author-rank
Guru

Created ‎01-30-2016 06:56 PM

@Rainer Geissendoerfer Others have encountered similar issues with the Nifi service (usually on Java 😎 as well but I have not been able to consistently reproduce it yet. From what I have seen of this issue, if you run the steps to start Nifi manually via CLI, it works (but for some reason not from Ambari). You can try below manual commands to start Nifi and populate the pid file so that Ambari can track its status:

su - nifi
export JAVA_HOME=/usr/java/default
/opt/nifi-1.1.1.0-12/bin/nifi.sh start >> /var/log/nifi/nifi-setup.log
cat /opt/nifi-1.1.1.0-12/bin/nifi.pid | grep pid | sed 's/pid=\(\.*\)/\1/' > /var/run/nifi/nifi.pid
#run below as root
chown nifi:nifi /var/run/nifi/nifi.pid

If you are encountering the same issue, could you provide the below details so we can try to reproduce it:

  • java version
  • OS/version
  • Ambari + HDP version (is this sandbox?)
  • are you installing Nifi service on same node as Ambari?

View solution in original post

7,716 Views
20 REPLIES 20

avatar
author-rank aervits
Master Mentor

Created ‎01-30-2016 12:06 PM

@Rainer Geissendoerfer please paste the nifi logs in n /var/log/nifi

4,943 Views
0 Kudos

avatar
author-rank rainer_geissend
Expert Contributor

Created ‎01-30-2016 03:04 PM

Based on Neerjas comment below I enabled user nifi to write into /var/log/ ... no at least I am getting an error in the Ambari Console when starting the service that I attached in errors-1795.txt.

In addition I posted the logs of /var/log/nifi.

Again, when I am starting "nifi.sh start" from bash, nifi starts successfully.

Thanks, in advance, for any further advise.

br,

Rainer

logs.tar.gz
10 KB
4,943 Views

avatar
author-rank nsabharwal
Master Mentor

Created ‎01-30-2016 01:35 PM

@Rainer Geissendoerfer

Couple of things:

1) Ambari service for NiFi is not officially released yet. You are using git code.

2) This is based on existing issue. I don't know how your setup looks like so you may want to check this.

In git code, user nifi is starting nifi ..make sure that nifi user has privilieges to write to /var/log

code link

4,943 Views
0 Kudos

avatar
author-rank rainer_geissend
Expert Contributor

Created ‎01-30-2016 03:05 PM

Hi Neeray, thanks for your feedback ... I am now getting more detailed logs ... even though the problem didn't go aways, please see above. Thanks, Rainer

4,943 Views

avatar
author-rank nsabharwal
Master Mentor

Created ‎01-30-2016 03:45 PM

@Rainer Geissendoerfer If you look deeply 😉 it's still permission issue. Make sure that you create directories for nifi user under /var/log, /var/run etc.

4,943 Views
0 Kudos

avatar
author-rank nsabharwal
Master Mentor

Created ‎01-30-2016 03:46 PM

@Rainer Geissendoerfer You can search git code to get an idea on directory structure

https://github.com/abajwa-hw/ambari-nifi-service/search?utf8=%E2%9C%93&q=%2Fvar%2F

4,943 Views
0 Kudos

avatar
author-rank nsabharwal
Master Mentor

Created ‎02-12-2016 09:51 AM

@Rainer Geissendoerfer Could you please accept the best the answer to close this thread?

4,943 Views
0 Kudos

avatar
author-rank aervits
Master Mentor

Created ‎01-30-2016 03:25 PM

@Rainer Geissendoerfer create directory /var/run/nifi and make sure permissions are set correctly on the directory

4,943 Views
0 Kudos

avatar
author-rank rainer_geissend
Expert Contributor

Created ‎01-30-2016 03:40 PM

I did that already in order to trick Ambari after starting nifi through bash (and then I inserted a valid PID in /var/run/nifi) and restarted the service through Amari. Obviously Ambari found that PID and got green ... but still no luck starting the service through Ambari 😞

4,943 Views
Announcements
Community Announcements
April 2025 Cloudera Customer Advisory: Cloudera’s response t...
What's New @ Cloudera
[RELEASED] Cloudera Streaming Analytics - Kubernetes Operato...
What's New @ Cloudera
[RELEASED] Cloudera Streams Messaging - Kubernetes Operator ...
Community Announcements
February 2025 Community Highlights
What's New @ Cloudera
3 Benefits of External IDE Connectivity, Now Available in Cl...
View More Announcements