Hi Team,
We are seeing some weird error in HDF 3.0.1.0-43 NIFI it not starting using ambari but it starting without any issues in commandline.
It something failing in the encryption, kindly guide me to resolve this thread.
resource_management.core.exceptions.ExecutionFailed: Execution of 'JAVA_HOME=/usr/jdk64/jdk1.8.0_60 /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.2.0.3.0.1.0-43/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -f /data/nifi/conf/flow.xml.gz -s '[PROTECTED]' -l /usr/hdf/current/nifi/conf/login-identity-providers.xml -p '[PROTECTED]'' returned 1.
stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 309, in <module>
Master().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
method(env)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 850, in restart
self.start(env, upgrade_type=upgrade_type)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 177, in start
self.configure(env, is_starting = True)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 119, in locking_configure
original_configure(obj, *args, **kw)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 150, in configure
params.nifi_flow_config_dir, params.nifi_sensitive_props_key, is_starting)
File "/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/scripts/nifi.py", line 304, in encrypt_sensitive_properties
Execute(encrypt_config_script_prefix, user=nifi_user,logoutput=False)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'JAVA_HOME=/usr/jdk64/jdk1.8.0_60 /var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.2.0.3.0.1.0-43/bin/encrypt-config.sh -v -b /usr/hdf/current/nifi/conf/bootstrap.conf -n /usr/hdf/current/nifi/conf/nifi.properties -f /data/nifi/conf/flow.xml.gz -s '[PROTECTED]' -l /usr/hdf/current/nifi/conf/login-identity-providers.xml -p '[PROTECTED]'' returned 1.
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of login-identity-providers.xml
2017/10/12 23:54:22 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source login-identity-providers.xml and destination login-identity-providers.xml are identical [/usr/hdf/current/nifi/conf/login-identity-providers.xml] so the original will be overwritten
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of nifi.properties
2017/10/12 23:54:22 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source nifi.properties and destination nifi.properties are identical [/usr/hdf/current/nifi/conf/nifi.properties] so the original will be overwritten
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Handling encryption of flow.xml.gz
2017/10/12 23:54:22 WARN [main] org.apache.nifi.properties.ConfigEncryptionTool: The source flow.xml.gz and destination flow.xml.gz are identical [/data/nifi/conf/flow.xml.gz] so the original will be overwritten
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: bootstrap.conf: /usr/hdf/current/nifi/conf/bootstrap.conf
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src) nifi.properties: /usr/hdf/current/nifi/conf/nifi.properties
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) nifi.properties: /usr/hdf/current/nifi/conf/nifi.properties
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src) login-identity-providers.xml: /usr/hdf/current/nifi/conf/login-identity-providers.xml
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) login-identity-providers.xml: /usr/hdf/current/nifi/conf/login-identity-providers.xml
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (src) flow.xml.gz: /data/nifi/conf/flow.xml.gz
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: (dest) flow.xml.gz: /data/nifi/conf/flow.xml.gz
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 128 properties from /usr/hdf/current/nifi/conf/nifi.properties
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.NiFiPropertiesLoader: Loaded 128 properties from /usr/hdf/current/nifi/conf/nifi.properties
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Loaded NiFiProperties instance with 128 properties
2017/10/12 23:54:22 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Loaded LoginIdentityProviders content (108 lines)
2017/10/12 23:54:22 WARN [main] org.apache.nifi.properties.AESSensitivePropertyProvider: JCE Unlimited Strength Cryptography Jurisdiction policies are not available, so the max key length is 128 bits
2017/10/12 23:54:22 WARN [main] org.apache.nifi.properties.AESSensitivePropertyProvider: JCE Unlimited Strength Cryptography Jurisdiction policies are not available, so the max key length is 128 bits
2017/10/12 23:54:23 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: No encrypted password property elements found in login-identity-providers.xml
2017/10/12 23:54:23 WARN [main] org.apache.nifi.properties.AESSensitivePropertyProvider: JCE Unlimited Strength Cryptography Jurisdiction policies are not available, so the max key length is 128 bits
2017/10/12 23:54:23 WARN [main] org.apache.nifi.properties.AESSensitivePropertyProvider: JCE Unlimited Strength Cryptography Jurisdiction policies are not available, so the max key length is 128 bits
2017/10/12 23:54:23 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Attempting to encrypt property
2017/10/12 23:54:23 INFO [main] org.apache.nifi.properties.AESSensitivePropertyProvider: AES Sensitive Property Provider encrypted a sensitive value successfully
2017/10/12 23:54:23 INFO [main] org.apache.nifi.properties.ConfigEncryptionTool: Updated XML content: <?xml version="1.0" encoding="UTF-8"?><loginIdentityProviders>
<provider>
<identifier>ldap-provider</identifier>
<class>org.apache.nifi.ldap.LdapProvider</class>
<property name="Identity Strategy">USE_DN</property>
<property name="Authentication Strategy">SIMPLE</property>
<property name="Manager DN">svc.haasdev.001@zone1.scbdev.net</property>
<property name="Manager Password" encryption="aes/gcm/128">cYAgEgKZAsfFiVjR||jB6aUU2eBkjpqUx2q7DvmE3AclRJZPdkKvQ4</property>
<property name="TLS - Keystore"/>
<property name="TLS - Keystore Password"/>
<property name="TLS - Keystore Type"/>
<property name="TLS - Truststore"/>
<property name="TLS - Truststore Password"/>
<property name="TLS - Truststore Type"/>
<property name="TLS - Client Auth"/>
<property name="TLS - Protocol">TLS</property>
<property name="TLS - Shutdown Gracefully"/>
<property name="Referral Strategy">FOLLOW</property>
<property name="Connect Timeout">10 secs</property>
<property name="Read Timeout">10 secs</property>
<property name="Url">ldap://HKWVADIDM05.zone1.scbdev.net:389</property>
<property name="User Search Base">OU=ITSC,dc=zone1,dc=scbdev,dc=net</property>
<property name="Identity Strategy">USE_USERNAME</property>
<property name="User Search Filter">sAMAccountName={0}</property>
<property name="Authentication Expiration">12 hours</property>
</provider>
</loginIdentityProviders>
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3332)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:569)
at java.lang.StringBuilder.append(StringBuilder.java:190)
at org.apache.commons.io.output.StringBuilderWriter.write(StringBuilderWriter.java:143)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2370)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2348)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2325)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2273)
at org.apache.commons.io.IOUtils.toString(IOUtils.java:1041)
at org.apache.commons.io.IOUtils$toString.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
at org.apache.nifi.properties.ConfigEncryptionTool$_loadFlowXml_closure2$_closure19.doCall(ConfigEncryptionTool.groovy:488)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
at org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019)
at groovy.lang.Closure.call(Closure.java:426)
at groovy.lang.Closure.call(Closure.java:442)
at org.codehaus.groovy.runtime.IOGroovyMethods.withCloseable(IOGroovyMethods.java:1622)
at org.codehaus.groovy.runtime.NioGroovyMethods.withCloseable(NioGroovyMethods.java:1754)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
stdout:
2017-10-12 23:53:50,252 - Group['ranger'] {}
2017-10-12 23:53:50,253 - Group['hadoop'] {}
2017-10-12 23:53:50,253 - Group['nifi'] {}
2017-10-12 23:53:50,253 - User['logsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-10-12 23:53:50,255 - User['infra-solr'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-10-12 23:53:50,255 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-10-12 23:53:50,256 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-10-12 23:53:50,257 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2017-10-12 23:53:50,257 - User['ranger'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'ranger']}
2017-10-12 23:53:50,258 - User['nifi'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'nifi']}
2017-10-12 23:53:50,259 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-10-12 23:53:50,260 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-10-12 23:53:50,265 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2017-10-12 23:53:50,278 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2017-10-12 23:53:50,284 - Skipping Execute[('setenforce', '0')] due to not_if
2017-10-12 23:53:50,501 - Stack Feature Version Info: stack_version=3.0, version=3.0.1.0-43, current_cluster_version=3.0.1.0-43 -> 3.0.1.0-43
2017-10-12 23:53:50,546 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2017-10-12 23:53:50,548 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_60;/usr/hdf/current/nifi/bin/nifi.sh stop >> /data/nifi/log/nifi-setup.log'] {'user': 'nifi'}
2017-10-12 23:54:20,757 - Pid file /var/run/nifi/nifi.pid is empty or does not exist
2017-10-12 23:54:20,759 - Directory['/var/run/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,759 - Directory['/data/nifi/log'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,763 - Directory['/data/nifi'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,936 - Directory['/data/nifi/database_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,937 - Directory['/data/nifi/flowfile_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,947 - Directory['/data/nifi/provenance_repository'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,957 - Directory['/usr/hdf/current/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,958 - Directory['/data/nifi/conf'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,958 - Directory['/data/nifi/state/local'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,959 - Directory['/usr/hdf/current/nifi/lib'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,961 - Directory['{{nifi_content_repo_dir_default}}'] {'owner': 'nifi', 'create_parents': True, 'group': 'nifi', 'recursive_ownership': True}
2017-10-12 23:54:20,962 - Directory['/data/nifi/content_repository'] {'owner': 'nifi', 'group': 'nifi', 'create_parents': True, 'recursive_ownership': True}
2017-10-12 23:54:21,027 - Directory['/etc/security/limits.d'] {'owner': 'root', 'create_parents': True, 'group': 'root'}
2017-10-12 23:54:21,030 - File['/etc/security/limits.d/nifi.conf'] {'content': Template('nifi.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2017-10-12 23:54:21,174 - PropertiesFile['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'group': 'nifi', 'mode': 0600, 'properties': ...}
2017-10-12 23:54:21,178 - Generating properties file: /usr/hdf/current/nifi/conf/nifi.properties
2017-10-12 23:54:21,178 - File['/usr/hdf/current/nifi/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-10-12 23:54:21,258 - Writing File['/usr/hdf/current/nifi/conf/nifi.properties'] because contents don't match
2017-10-12 23:54:21,261 - File['/usr/hdf/current/nifi/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-10-12 23:54:21,265 - File['/usr/hdf/current/nifi/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-10-12 23:54:21,268 - File['/usr/hdf/current/nifi/conf/state-management.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-10-12 23:54:21,273 - File['/usr/hdf/current/nifi/conf/authorizers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-10-12 23:54:21,277 - File['/usr/hdf/current/nifi/conf/login-identity-providers.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0600}
2017-10-12 23:54:21,278 - File['/usr/hdf/current/nifi/bin/nifi-env.sh'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0755}
2017-10-12 23:54:21,280 - File['/usr/hdf/current/nifi/conf/bootstrap-notification-services.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi', 'mode': 0400}
2017-10-12 23:54:21,281 - Encrypting NiFi sensitive configuration properties
2017-10-12 23:54:21,281 - File['/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.2.0.3.0.1.0-43/bin/encrypt-config.sh'] {'mode': 0755}
2017-10-12 23:54:21,429 - Execute[(u'JAVA_HOME=/usr/jdk64/jdk1.8.0_60', '/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.2.0.3.0.1.0-43/bin/encrypt-config.sh', '-v', '-b', u'/usr/hdf/current/nifi/conf/bootstrap.conf', '-n', u'/usr/hdf/current/nifi/conf/nifi.properties', '-f', u'/data/nifi/conf/flow.xml.gz', '-s', [PROTECTED], '-l', u'/usr/hdf/current/nifi/conf/login-identity-providers.xml', '-p', [PROTECTED])] {'logoutput': False, 'user': 'nifi'}
Command failed after 1 tries
