NIFI service doesn't start after SSL configuration

[#1]

Hello guys please i need some help here. A former colleague of mine installed some HDF services into an existing HDP cluster. The NIFI service was running fine but the UI wasnt open because the ssl feature was not completely configured. I followed the guide by configuring the Initial Admin entity and Node identities but after saving it, NIFI stopped working and till now i cannot start it. The error below comes up whenever i start the NIFI service.

org.apache.nifi.toolkit.tls.<wbr>service.client.<wbr>TlsCertificateAuthorityClient: Unable to open existing keystore, it can be reused by specifiying both configJson and useConfigJson
Service client error: Keystore was tampered with, or password was incorrect

Kindly help me.

Also can i use the Ambari to remove a service from the cluster?

Thanks

[#2 · Accepted solution]

@Biliyaminu Umar

The Error indicates that the Keystore is corrupted or the Password of the keystore is changed.

 Keystore was tampered with, or password was incorrect

.

You can use the keytool -list command to see if with the keystore password it is working or not?

# $JAVA_HOME/bin/keytool -list -v -keystore /PATH/TO/your_keystore.jks

.

Regarding your query: "can i use the Ambari to remove a service from the cluster?"

From Ambari Server UI we need to stop the service and then we can delete it as following:

Example: (It can be any service "NiFi is just for example here)

Ambari UI --> NiFi --> Service Actions (Drop Down) --> Stop
Ambari UI --> NiFi --> Service Actions (Drop Down) --> Delete Service

.

Also the services can be deleted using Ambari API as mentioned in the following Doc:

https://cwiki.apache.org/confluence/display/AMBARI/Using+APIs+to+delete+a+service+or+all+host+compon...

.

[#3]

Thanks @Jay SenSharma. I will try this out. To paint a better scenario, its an HDP cluster of 4 nodes but nifi is running on just one of them. Also, NIFI Certificate Authority is also installed. I thought with CA, i wont need to make any changes to the keystore.

[#4]

Thanks @Jay Kumar SenSharma i tried what you told me and the keystore password worked. But when i start the NIFI service, i still get the same error. So i tried to reinstall the nifi but i get the error below.

17/10/31 15:57:49 INFO properties.AESSensitivePropertyProvider: AES Sensitive Property Provider decrypted a sensitive value successfully
Attempt to contact NiFi Node https://iot-node07.netvisor.local:9091/nifi did not complete due to exception: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors. Sleeping for 20 sec(s)
2017-10-31 15:58:10,335 - File['/var/lib/ambari-agent/cache/common-services/NIFI/1.0.0/package/files/nifi-toolkit-1.2.0.3.0.0.0-453/bin/node-manager.sh'] {'mode': 0755}
2017-10-31 15:58:13,004 - Will retry 14 time(s), caught exception: Error connecting to one or more nifi nodes: 17/10/31 15:58:11 INFO properties.NiFiPropertiesLoader: Loaded 124 properties from /usr/hdf/current/nifi/conf/nokedli/nifi.properties
17/10/31 15:58:12 INFO properties.ProtectedNiFiProperties: There are 4 protected properties of 5 sensitive properties (100%)
17/10/31 15:58:12 INFO properties.AESSensitivePropertyProvider: AES Sensitive Property Provider decrypted a sensitive value successfully
17/10/31 15:58:12 INFO properties.AESSensitivePropertyProvider: AES Sensitive Property Provider decrypted a sensitive value successfully
17/10/31 15:58:12 INFO properties.AESSensitivePropertyProvider: AES Sensitive Property Provider decrypted a sensitive value successfully
17/10/31 15:58:12 INFO properties.AESSensitivePropertyProvider: AES Sensitive Property Provider decrypted a sensitive value successfully
Attempt to contact NiFi Node https://iot-node07.netvisor.local:9091/nifi did not complete due to exception: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors. Sleeping for 20 sec(s

[#5]

@Biliyaminu Umar

On the Nifi Host can you please check what is the path of nifi keystore & truststore?

# grep 'nifi.security' /etc/nifi/conf/nifi.properties 

.
Also please check the permission of these files.
And please check the certs if those are OK using the following command:

# openssl s_client -showcerts -connect iot-node07.xxxxxxxxxxxx.local:9091

.

What is your JDK version?

[#6]

Hello @Jay Kumar SenSharma since these certificates were generated by someone else i will like to delete them and create a fresh set of certificates together with readding the NIFI component to HDP.

Kindly help will the proper certificate clearing and creation process.

Thanks

[#7]

@Biliyaminu Umar

The following link can be referred to know about setting the certificates:

https://community.hortonworks.com/articles/17293/how-to-create-user-generated-keys-for-securing-nif....

.

[#8]

Hello @Jay Kumar SenSharma my jdk version is as below.

openjdk version "1.8.0_131"

When i ran the command to check the certificates, i got the message below before the certificate chain message.

CONNECTED(00000003) depth=1 OU = NIFI, CN = iot-node07.netvisor.local verify error:num=19:self signed certificate in certificate chain verify return:0 140187436160928:error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate:s3_pkt.c:1259:SS L alert number 42 140187436160928:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:184: ---

Finally the permissions of the certificates were -rw-------

Thanks

[#9]

Thank you very much @Jay Kumar SenSharma

I am able to resolve it now

Umar