Support Questions
Find answers, ask questions, and share your expertise

Installed Nifi using Ambari but the service is not starting.

Installed Nifi using Ambari but the service is not starting.

tderr: /var/lib/ambari-agent/data/errors-41.txt

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 360, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 164, in start
    self.configure(env, is_starting = True)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 145, in configure
    params.nifi_flow_config_dir, params.nifi_sensitive_props_key, is_starting)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 298, in encrypt_sensitive_properties
    Execute(encrypt_config_script_prefix, user=nifi_user,logoutput=False)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'JAVA_HOME=/usr/jdk64/jdk1.8.0_77 /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.2.0-10/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -p '[PROTECTED]'' returned 4. 2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of nifi.properties
2017/07/09 01:58:34 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source nifi.properties and destination nifi.properties are identical [/usr/hdf/current/nifi/conf/nifi.properties] so the original will be overwritten
2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool:        bootstrap.conf:               	/usr/hdf/current/nifi/conf/bootstrap.conf
2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  nifi.properties:              	/usr/hdf/current/nifi/conf/nifi.properties
2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) nifi.properties:              	/usr/hdf/current/nifi/conf/nifi.properties
2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  login-identity-providers.xml: 	null
2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) login-identity-providers.xml: 	null
2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  flow.xml.gz: 					null
2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) flow.xml.gz: 					null
2017/07/09 01:58:34 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 116 properties from /usr/hdf/current/nifi/conf/nifi.properties
2017/07/09 01:58:35 ERROR [main] org.apache.nifi.properties.ConfigEncryptionTool: Encountered an error
java.security.KeyException: Cannot derive key from empty/short password -- password must be at least 12 characters
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:80)
	at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:74)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
	at org.apache.nifi.properties.ConfigEncryptionTool.deriveKeyFromPassword(ConfigEncryptionTool.groovy:1042)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite$StaticMetaMethodSiteNoUnwrapNoCoerce.invoke(StaticMetaMethodSite.java:151)
	at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:102)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:206)
	at org.apache.nifi.properties.ConfigEncryptionTool.getKeyInternal(ConfigEncryptionTool.groovy:362)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190)
	at org.apache.nifi.properties.ConfigEncryptionTool.getKey(ConfigEncryptionTool.groovy:377)
	at org.apache.nifi.properties.ConfigEncryptionTool.getKey(ConfigEncryptionTool.groovy:376)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
	at org.apache.nifi.properties.ConfigEncryptionTool.main(ConfigEncryptionTool.groovy:1105)
Cannot derive key from empty/short password -- password must be at least 12 characters

usage: org.apache.nifi.properties.ConfigEncryptionTool [-A <arg>] [-b <arg>] [-e <arg>] [-f <arg>] [-g <arg>] [-h] [-i <arg>] [-k <arg>] [-l <arg>] [-m] [-n
       <arg>] [-o <arg>] [-p <arg>] [-P <arg>] [-r] [-s <arg>] [-v] [-w <arg>] [-x]

This tool reads from a nifi.properties and/or login-identity-providers.xml file with plain sensitive configuration values, prompts the user for a master key,
and encrypts each value. It will replace the plain value with the protected value in the same file (or write to a new file if specified). It can also be used to
migrate already-encrypted values in those files or in flow.xml.gz to be encrypted with a new key.

 -A,--newFlowAlgorithm <arg>               The algorithm to use to encrypt the sensitive processor properties in flow.xml.gz
 -b,--bootstrapConf <arg>                  The bootstrap.conf file to persist master key
 -e,--oldKey <arg>                         The old raw hexadecimal key to use during key migration
 -f,--flowXml <arg>                        The flow.xml.gz file currently protected with old password (will be overwritten)
 -g,--outputFlowXml <arg>                  The destination flow.xml.gz file containing protected config values (will not modify input flow.xml.gz)
 -h,--help                                 Prints this usage message
 -i,--outputLoginIdentityProviders <arg>   The destination login-identity-providers.xml file containing protected config values (will not modify input
                                           login-identity-providers.xml)
 -k,--key <arg>                            The raw hexadecimal key to use to encrypt the sensitive properties
 -l,--loginIdentityProviders <arg>         The login-identity-providers.xml file containing unprotected config values (will be overwritten)
 -m,--migrate                              If provided, the nifi.properties and/or login-identity-providers.xml sensitive properties will be re-encrypted with a
                                           new key
 -n,--niFiProperties <arg>                 The nifi.properties file containing unprotected config values (will be overwritten)
 -o,--outputNiFiProperties <arg>           The destination nifi.properties file containing protected config values (will not modify input nifi.properties)
 -p,--password <arg>                       The password from which to derive the key to use to encrypt the sensitive properties
 -P,--newFlowProvider <arg>                The security provider to use to encrypt the sensitive processor properties in flow.xml.gz
 -r,--useRawKey                            If provided, the secure console will prompt for the raw key value in hexadecimal form
 -s,--propsKey <arg>                       The password or key to use to encrypt the sensitive processor properties in flow.xml.gz
 -v,--verbose                              Sets verbose mode (default false)
 -w,--oldPassword <arg>                    The old password from which to derive the key during migration
 -x,--encryptFlowXmlOnly                   If provided, the properties in flow.xml.gz will be re-encrypted with a new key but the nifi.properties and/or
                                           login-identity-providers.xml files will not be modified

Java home: /usr/jdk64/jdk1.8.0_77
NiFi Toolkit home: /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.2.0-10

stdout: /var/lib/ambari-agent/data/output-41.txt

2017-07-09 01:58:28,162 - Group['hadoop'] {}
2017-07-09 01:58:28,198 - Group['nifi'] {}
2017-07-09 01:58:28,198 - User['storm'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-09 01:58:28,199 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-09 01:58:28,199 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-09 01:58:28,200 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-09 01:58:28,200 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2017-07-09 01:58:28,201 - User['kafka'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-09 01:58:28,201 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'nifi']}
2017-07-09 01:58:28,202 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-07-09 01:58:28,224 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-07-09 01:58:28,324 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2017-07-09 01:58:28,344 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2017-07-09 01:58:28,460 - Skipping Execute[('setenforce', '0')] due to only_if
2017-07-09 01:58:28,890 - Stack Feature Version Info: stack_version=2.1, version=2.1.4.0-5, current_cluster_version=2.1.4.0-5 -> 2.1.4.0-5
2017-07-09 01:58:28,934 - Directory['/var/run/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:28,936 - Directory['/var/log/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:28,949 - Directory['/var/lib/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:29,018 - Directory['/var/lib/nifi/database_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:29,018 - Directory['/var/lib/nifi/flowfile_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:29,019 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:29,019 - Directory['/var/lib/nifi/provenance_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:29,020 - Directory['/usr/hdf/current/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:29,079 - Directory['/var/lib/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:29,079 - Directory['/var/lib/nifi/state/local'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-07-09 01:58:29,080 - Directory['/usr/hdf/current/nifi/lib'] {'owner': 'nifi', 'group': 'nifi', 'create_parents': True, 'recursive_ownership': True}
2017-07-09 01:58:29,226 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2017-07-09 01:58:29,282 - File['/etc/security/limits.d/nifi.conf'] {'content': Template('nifi.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2017-07-09 01:58:29,293 - PropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'group': 'nifi', 'mode': 0600, 'properties': ...}
2017-07-09 01:58:29,311 - Generating properties file: /usr/hdf/current/nifi/conf/nifi.properties
2017-07-09 01:58:29,311 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-07-09 01:58:29,482 - Writing File['/usr/hdf/current/nifi/conf/nifi.properties'] because contents don't match
2017-07-09 01:58:29,486 - File['/usr/hdf/current/nifi/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-07-09 01:58:29,501 - File['/usr/hdf/current/nifi/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-07-09 01:58:29,515 - File['/usr/hdf/current/nifi/conf/state-management.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-07-09 01:58:29,532 - File['/usr/hdf/current/nifi/conf/authorizers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-07-09 01:58:29,552 - File['/usr/hdf/current/nifi/conf/login-identity-providers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-07-09 01:58:29,564 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2017-07-09 01:58:29,577 - File['/usr/hdf/current/nifi/conf/bootstrap-notification-services.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-07-09 01:58:29,588 - Encrypting NiFi sensitive configuration properties
2017-07-09 01:58:29,588 - File['/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.2.0-10/bin/encrypt-config.sh'] {'mode': 0755}
2017-07-09 01:58:29,803 - Execute[(u'JAVA_HOME=/usr/jdk64/jdk1.8.0_77', '/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.2.0-10/bin/encrypt-config.sh', '-v', '-b', u'/usr/hdf/current/nifi/conf/bootstrap.conf', '-n', u'/usr/hdf/current/nifi/conf/nifi.properties', '-p', [PROTECTED])] {'logoutput': False, 'user': 'nifi'}
 

Command failed after 1 tries

That is the error log.

2 REPLIES 2

Re: Installed Nifi using Ambari but the service is not starting.

Super Mentor

@Rahul P

Error:

2017/07/09 01:58:35 ERROR [main] org.apache.nifi.properties.ConfigEncryptionTool: Encountered an error
java.security.KeyException: Cannot derive key from empty/short password -- password must be at least 12 characters

Base don't he error details, it looks like You have not given the password based on the password policy rules. May be during the NiFi installation the password mentioned in the property "Encrypt Configuration Master Key Password" is not respecting the password policy. Please use 12 characters at least to define the password/key.

.

Re: Installed Nifi using Ambari but the service is not starting.

@Jay SenSharma I changed the password size and it worked. Thanks