Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NIFI wont start on HDF

SOLVED Go to solution
Highlighted

NIFI wont start on HDF

Expert Contributor

Hi, i have just installed HDF, Nifi wont start. I get the following error. Any help would be appreciated. Thanks

stderr: 
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 360, in <module>
    Master().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
    method(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 720, in restart
    self.start(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 164, in start
    self.configure(env, is_starting = True)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 145, in configure
    params.nifi_flow_config_dir, params.nifi_sensitive_props_key, is_starting)
  File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 298, in encrypt_sensitive_properties
    Execute(encrypt_config_script_prefix, user=nifi_user,logoutput=False)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'JAVA_HOME=/usr/jdk64/jdk1.8.0_77 /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.1.0-2/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -p '[PROTECTED]'' returned 4. 2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of nifi.properties
2017/01/08 19:46:19 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source nifi.properties and destination nifi.properties are identical [/usr/hdf/current/nifi/conf/nifi.properties] so the original will be overwritten
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool:        bootstrap.conf:                   /usr/hdf/current/nifi/conf/bootstrap.conf
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  nifi.properties:                  /usr/hdf/current/nifi/conf/nifi.properties
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) nifi.properties:                  /usr/hdf/current/nifi/conf/nifi.properties
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  login-identity-providers.xml:     null
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) login-identity-providers.xml:     null
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src)  flow.xml.gz:                     null
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) flow.xml.gz:                     null
2017/01/08 19:46:19 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 116 properties from /usr/hdf/current/nifi/conf/nifi.properties
2017/01/08 19:46:19 ERROR [main] org.apache.nifi.properties.ConfigEncryptionTool: Encountered an error
java.security.KeyException: Cannot derive key from empty/short password -- password must be at least 12 characters
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:80)
    at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:74)
    at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
    at org.apache.nifi.properties.ConfigEncryptionTool.deriveKeyFromPassword(ConfigEncryptionTool.groovy:1042)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite$StaticMetaMethodSiteNoUnwrapNoCoerce.invoke(StaticMetaMethodSite.java:151)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:102)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:206)
    at org.apache.nifi.properties.ConfigEncryptionTool.getKeyInternal(ConfigEncryptionTool.groovy:362)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:190)
    at org.apache.nifi.properties.ConfigEncryptionTool.getKey(ConfigEncryptionTool.groovy:377)
    at org.apache.nifi.properties.ConfigEncryptionTool.getKey(ConfigEncryptionTool.groovy:376)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
    at org.apache.nifi.properties.ConfigEncryptionTool.main(ConfigEncryptionTool.groovy:1105)
Cannot derive key from empty/short password -- password must be at least 12 characters

usage: org.apache.nifi.properties.ConfigEncryptionTool [-A <arg>] [-b <arg>] [-e <arg>] [-f <arg>] [-g <arg>] [-h] [-i <arg>] [-k <arg>] [-l <arg>] [-m] [-n
       <arg>] [-o <arg>] [-p <arg>] [-P <arg>] [-r] [-s <arg>] [-v] [-w <arg>] [-x]

This tool reads from a nifi.properties and/or login-identity-providers.xml file with plain sensitive configuration values, prompts the user for a master key,
and encrypts each value. It will replace the plain value with the protected value in the same file (or write to a new file if specified). It can also be used to
migrate already-encrypted values in those files or in flow.xml.gz to be encrypted with a new key.

 -A,--newFlowAlgorithm <arg>               The algorithm to use to encrypt the sensitive processor properties in flow.xml.gz
 -b,--bootstrapConf <arg>                  The bootstrap.conf file to persist master key
 -e,--oldKey <arg>                         The old raw hexadecimal key to use during key migration
 -f,--flowXml <arg>                        The flow.xml.gz file currently protected with old password (will be overwritten)
 -g,--outputFlowXml <arg>                  The destination flow.xml.gz file containing protected config values (will not modify input flow.xml.gz)
 -h,--help                                 Prints this usage message
 -i,--outputLoginIdentityProviders <arg>   The destination login-identity-providers.xml file containing protected config values (will not modify input
                                           login-identity-providers.xml)
 -k,--key <arg>                            The raw hexadecimal key to use to encrypt the sensitive properties
 -l,--loginIdentityProviders <arg>         The login-identity-providers.xml file containing unprotected config values (will be overwritten)
 -m,--migrate                              If provided, the nifi.properties and/or login-identity-providers.xml sensitive properties will be re-encrypted with a
                                           new key
 -n,--niFiProperties <arg>                 The nifi.properties file containing unprotected config values (will be overwritten)
 -o,--outputNiFiProperties <arg>           The destination nifi.properties file containing protected config values (will not modify input nifi.properties)
 -p,--password <arg>                       The password from which to derive the key to use to encrypt the sensitive properties
 -P,--newFlowProvider <arg>                The security provider to use to encrypt the sensitive processor properties in flow.xml.gz
 -r,--useRawKey                            If provided, the secure console will prompt for the raw key value in hexadecimal form
 -s,--propsKey <arg>                       The password or key to use to encrypt the sensitive processor properties in flow.xml.gz
 -v,--verbose                              Sets verbose mode (default false)
 -w,--oldPassword <arg>                    The old password from which to derive the key during migration
 -x,--encryptFlowXmlOnly                   If provided, the properties in flow.xml.gz will be re-encrypted with a new key but the nifi.properties and/or
                                           login-identity-providers.xml files will not be modified

Java home: /usr/jdk64/jdk1.8.0_77
NiFi Toolkit home: /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.1.0-2
 stdout:
2017-01-08 19:46:12,153 - Group['hadoop'] {}
2017-01-08 19:46:12,154 - Group['nifi'] {}
2017-01-08 19:46:12,155 - User['storm'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,156 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,157 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,157 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,158 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2017-01-08 19:46:12,159 - User['kafka'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-01-08 19:46:12,159 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'nifi']}
2017-01-08 19:46:12,160 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-01-08 19:46:12,161 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-01-08 19:46:12,174 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2017-01-08 19:46:12,205 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2017-01-08 19:46:12,217 - Skipping Execute[('setenforce', '0')] due to not_if
2017-01-08 19:46:12,408 - Stack Feature Version Info: stack_version=2.1, version=2.1.1.0-2, current_cluster_version=2.1.1.0-2 -> 2.1.1.0-2
2017-01-08 19:46:12,430 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2017-01-08 19:46:12,432 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_77;/usr/hdf/current/nifi/bin/nifi.sh stop >> /var/log/nifi/nifi-setup.log'] {'user': 'nifi'}
2017-01-08 19:46:16,632 - Pid file /var/run/nifi/nifi.pid is empty or does not exist
2017-01-08 19:46:16,634 - Directory['/var/run/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,634 - Directory['/var/log/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,635 - Directory['/var/lib/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,635 - Directory['/var/lib/nifi/database_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,636 - Directory['/var/lib/nifi/flowfile_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,639 - Directory['/var/lib/nifi/content_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,640 - Directory['/var/lib/nifi/provenance_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,640 - Directory['/usr/hdf/current/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,641 - Directory['/var/lib/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,641 - Directory['/var/lib/nifi/state/local'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-01-08 19:46:16,642 - Directory['/usr/hdf/current/nifi/lib'] {'owner': 'nifi', 'group': 'nifi', 'create_parents': True, 'recursive_ownership': True}
2017-01-08 19:46:16,644 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2017-01-08 19:46:16,650 - File['/etc/security/limits.d/nifi.conf'] {'content': Template('nifi.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2017-01-08 19:46:16,651 - PropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'group': 'nifi', 'mode': 0600, 'properties': ...}
2017-01-08 19:46:16,655 - Generating properties file: /usr/hdf/current/nifi/conf/nifi.properties
2017-01-08 19:46:16,656 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-01-08 19:46:16,779 - Writing File['/usr/hdf/current/nifi/conf/nifi.properties'] because contents don't match
2017-01-08 19:46:16,784 - File['/usr/hdf/current/nifi/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-01-08 19:46:16,788 - File['/usr/hdf/current/nifi/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-01-08 19:46:16,791 - File['/usr/hdf/current/nifi/conf/state-management.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-01-08 19:46:16,797 - File['/usr/hdf/current/nifi/conf/authorizers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-01-08 19:46:16,802 - File['/usr/hdf/current/nifi/conf/login-identity-providers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-01-08 19:46:16,805 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2017-01-08 19:46:16,807 - File['/usr/hdf/current/nifi/conf/bootstrap-notification-services.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-01-08 19:46:16,808 - Encrypting NiFi sensitive configuration properties
2017-01-08 19:46:16,808 - File['/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.1.0-2/bin/encrypt-config.sh'] {'mode': 0755}
2017-01-08 19:46:16,812 - Execute[(u'JAVA_HOME=/usr/jdk64/jdk1.8.0_77', '/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.1.0.2.1.1.0-2/bin/encrypt-config.sh', '-v', '-b', u'/usr/hdf/current/nifi/conf/bootstrap.conf', '-n', u'/usr/hdf/current/nifi/conf/nifi.properties', '-p', [PROTECTED])] {'logoutput': False, 'user': 'nifi'}

Command failed after 1 tries


1 ACCEPTED SOLUTION

Accepted Solutions

Re: NIFI wont start on HDF

Hi @Roger Young,

Based on the stacktrace, it seems that during the NiFi installation the password provided for the property "Encrypt Configuration Master Key Password" is not respecting the conditions:

Cannot derive key from empty/short password -- password must be at least 12 characters
4 REPLIES 4

Re: NIFI wont start on HDF

Hi @Roger Young,

Based on the stacktrace, it seems that during the NiFi installation the password provided for the property "Encrypt Configuration Master Key Password" is not respecting the conditions:

Cannot derive key from empty/short password -- password must be at least 12 characters

Re: NIFI wont start on HDF

Expert Contributor

Hi, thanks, where could i change the password?

Re: NIFI wont start on HDF

Expert Contributor

Got it, thank you. I changed the pasword on the Ambari GUI. Under NiFi > configs >

Advanced nifi-ambari-config.

Re: NIFI wont start on HDF

New Contributor

Thank you Roger, your solution helped me.