Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NIFI wont start on HDF

avatar
Expert Contributor

Hi, i have just installed HDF, Nifi wont start. I get the following error. Any help would be appreciated. Thanks

stderr: 
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 360, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 720, in restart
    self.start(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 164, in start
    self.configure(env, is_starting = True)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 145, in configure
    params.nifi_flow_config_dir, params.nifi_sensitive_props_key, is_starting)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 298, in encrypt_sensitive_properties
    Execute(encrypt_config_script_prefix, user=nifi_user,logoutput=False)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'JAVA_HOME=/usr/jdk64/jdk1.8.0_77 /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.1.0-2/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -p '[PROTECTED]'' returned 4. 2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of nifi.properties
2017/01/08 19:46:19 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source nifi.properties and destination nifi.properties are identical [/usr/hdf/current/nifi/conf/nifi.properties] so the original will be overwritten
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool:        bootstrap.conf:                   /usr/hdf/current/nifi/conf/bootstrap.conf
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  nifi.properties:                  /usr/hdf/current/nifi/conf/nifi.properties
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) nifi.properties:                  /usr/hdf/current/nifi/conf/nifi.properties
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  login-identity-providers.xml:     null
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) login-identity-providers.xml:     null
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  flow.xml.gz:                     null
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) flow.xml.gz:                     null
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 116 properties from /usr/hdf/current/nifi/conf/nifi.properties
2017/01/08 19:46:19 ERROR [main] org.apache.nifi.properties.ConfigEncryptionTool: Encountered an error
java.security.KeyException: Cannot derive key from empty/short password -- password must be at least 12 characters
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:80)
    at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:74)
    at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
    at org.apache.nifi.properties.ConfigEncryptionTool.deriveKeyFromPassword(ConfigEncryptionTool.groovy:1042)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite$StaticMetaMethodSiteNoUnwrapNoCoerce.invoke(StaticMetaMethodSite.java:151)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:102)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:206)
    at org.apache.nifi.properties.ConfigEncryptionTool.getKeyInternal(ConfigEncryptionTool.groovy:362)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190)
    at org.apache.nifi.properties.ConfigEncryptionTool.getKey(ConfigEncryptionTool.groovy:377)
    at org.apache.nifi.properties.ConfigEncryptionTool.getKey(ConfigEncryptionTool.groovy:376)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
    at org.apache.nifi.properties.ConfigEncryptionTool.main(ConfigEncryptionTool.groovy:1105)
Cannot derive key from empty/short password -- password must be at least 12 characters

usage: org.apache.nifi.properties.ConfigEncryptionTool [-A <arg>] [-b <arg>] [-e <arg>] [-f <arg>] [-g <arg>] [-h] [-i <arg>] [-k <arg>] [-l <arg>] [-m] [-n
       <arg>] [-o <arg>] [-p <arg>] [-P <arg>] [-r] [-s <arg>] [-v] [-w <arg>] [-x]

This tool reads from a nifi.properties and/or login-identity-providers.xml file with plain sensitive configuration values, prompts the user for a master key,
and encrypts each value. It will replace the plain value with the protected value in the same file (or write to a new file if specified). It can also be used to
migrate already-encrypted values in those files or in flow.xml.gz to be encrypted with a new key.

 -A,--newFlowAlgorithm <arg>               The algorithm to use to encrypt the sensitive processor properties in flow.xml.gz
 -b,--bootstrapConf <arg>                  The bootstrap.conf file to persist master key
 -e,--oldKey <arg>                         The old raw hexadecimal key to use during key migration
 -f,--flowXml <arg>                        The flow.xml.gz file currently protected with old password (will be overwritten)
 -g,--outputFlowXml <arg>                  The destination flow.xml.gz file containing protected config values (will not modify input flow.xml.gz)
 -h,--help                                 Prints this usage message
 -i,--outputLoginIdentityProviders <arg>   The destination login-identity-providers.xml file containing protected config values (will not modify input
                                           login-identity-providers.xml)
 -k,--key <arg>                            The raw hexadecimal key to use to encrypt the sensitive properties
 -l,--loginIdentityProviders <arg>         The login-identity-providers.xml file containing unprotected config values (will be overwritten)
 -m,--migrate                              If provided, the nifi.properties and/or login-identity-providers.xml sensitive properties will be re-encrypted with a
                                           new key
 -n,--niFiProperties <arg>                 The nifi.properties file containing unprotected config values (will be overwritten)
 -o,--outputNiFiProperties <arg>           The destination nifi.properties file containing protected config values (will not modify input nifi.properties)
 -p,--password <arg>                       The password from which to derive the key to use to encrypt the sensitive properties
 -P,--newFlowProvider <arg>                The security provider to use to encrypt the sensitive processor properties in flow.xml.gz
 -r,--useRawKey                            If provided, the secure console will prompt for the raw key value in hexadecimal form
 -s,--propsKey <arg>                       The password or key to use to encrypt the sensitive processor properties in flow.xml.gz
 -v,--verbose                              Sets verbose mode (default false)
 -w,--oldPassword <arg>                    The old password from which to derive the key during migration
 -x,--encryptFlowXmlOnly                   If provided, the properties in flow.xml.gz will be re-encrypted with a new key but the nifi.properties and/or
                                           login-identity-providers.xml files will not be modified

Java home: /usr/jdk64/jdk1.8.0_77
NiFi Toolkit home: /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.1.0-2
 stdout:
2017-01-08 19:46:12,153 - Group['hadoop'] {}
2017-01-08 19:46:12,154 - Group['nifi'] {}
2017-01-08 19:46:12,155 - User['storm'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,156 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,157 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,157 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,158 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2017-01-08 19:46:12,159 - User['kafka'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,159 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'nifi']}
2017-01-08 19:46:12,160 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-01-08 19:46:12,161 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-01-08 19:46:12,174 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2017-01-08 19:46:12,205 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2017-01-08 19:46:12,217 - Skipping Execute[('setenforce', '0')] due to not_if
2017-01-08 19:46:12,408 - Stack Feature Version Info: stack_version=2.1, version=2.1.1.0-2, current_cluster_version=2.1.1.0-2 -> 2.1.1.0-2
2017-01-08 19:46:12,430 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2017-01-08 19:46:12,432 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_77;/usr/hdf/current/nifi/bin/nifi.sh stop >> /var/log/nifi/nifi-setup.log'] {'user': 'nifi'}
2017-01-08 19:46:16,632 - Pid file /var/run/nifi/nifi.pid is empty or does not exist
2017-01-08 19:46:16,634 - Directory['/var/run/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,634 - Directory['/var/log/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,635 - Directory['/var/lib/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,635 - Directory['/var/lib/nifi/database_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,636 - Directory['/var/lib/nifi/flowfile_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,639 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,640 - Directory['/var/lib/nifi/provenance_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,640 - Directory['/usr/hdf/current/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,641 - Directory['/var/lib/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,641 - Directory['/var/lib/nifi/state/local'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,642 - Directory['/usr/hdf/current/nifi/lib'] {'owner': 'nifi', 'group': 'nifi', 'create_parents': True, 'recursive_ownership': True}
2017-01-08 19:46:16,644 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2017-01-08 19:46:16,650 - File['/etc/security/limits.d/nifi.conf'] {'content': Template('nifi.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2017-01-08 19:46:16,651 - PropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'group': 'nifi', 'mode': 0600, 'properties': ...}
2017-01-08 19:46:16,655 - Generating properties file: /usr/hdf/current/nifi/conf/nifi.properties
2017-01-08 19:46:16,656 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-01-08 19:46:16,779 - Writing File['/usr/hdf/current/nifi/conf/nifi.properties'] because contents don't match
2017-01-08 19:46:16,784 - File['/usr/hdf/current/nifi/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-01-08 19:46:16,788 - File['/usr/hdf/current/nifi/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-01-08 19:46:16,791 - File['/usr/hdf/current/nifi/conf/state-management.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-01-08 19:46:16,797 - File['/usr/hdf/current/nifi/conf/authorizers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-01-08 19:46:16,802 - File['/usr/hdf/current/nifi/conf/login-identity-providers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-01-08 19:46:16,805 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2017-01-08 19:46:16,807 - File['/usr/hdf/current/nifi/conf/bootstrap-notification-services.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-01-08 19:46:16,808 - Encrypting NiFi sensitive configuration properties
2017-01-08 19:46:16,808 - File['/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.1.0-2/bin/encrypt-config.sh'] {'mode': 0755}
2017-01-08 19:46:16,812 - Execute[(u'JAVA_HOME=/usr/jdk64/jdk1.8.0_77', '/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.1.0-2/bin/encrypt-config.sh', '-v', '-b', u'/usr/hdf/current/nifi/conf/bootstrap.conf', '-n', u'/usr/hdf/current/nifi/conf/nifi.properties', '-p', [PROTECTED])] {'logoutput': False, 'user': 'nifi'}

Command failed after 1 tries


1 ACCEPTED SOLUTION

avatar

Hi @Roger Young,

Based on the stacktrace, it seems that during the NiFi installation the password provided for the property "Encrypt Configuration Master Key Password" is not respecting the conditions:

Cannot derive key from empty/short password -- password must be at least 12 characters

View solution in original post

4 REPLIES 4

avatar

Hi @Roger Young,

Based on the stacktrace, it seems that during the NiFi installation the password provided for the property "Encrypt Configuration Master Key Password" is not respecting the conditions:

Cannot derive key from empty/short password -- password must be at least 12 characters

avatar
Expert Contributor

Hi, thanks, where could i change the password?

avatar
Expert Contributor

Got it, thank you. I changed the pasword on the Ambari GUI. Under NiFi > configs >

Advanced nifi-ambari-config.

avatar
New Contributor

Thank you Roger, your solution helped me.