Error while using fetchs3object nifi processor to fetch data from aws s3

[#1]

I am using Nifi Version 1.5.0.3.1.2.0-7.I am trying to fetch data from aws s3 storage by using fetchs3object processor.I had attached the screenshot of processor.I am trying to connect through aws access key and security key which i had already provided in the configuration details of fetchs3object prcessor.I had also provided bucket name,object key,region as shown in image.I am trying to connect to aws from nifi first time.It is showing error 'unable to find valid certification path to requested target' while starting the processor.

Please help me in resolving the error.

[#2]

@mayank_tripathi The solution here is to setup the SSL context Service for the processor.  To do this you will need to create a new SSL Context Service and provide it a path to keystore and/or truststore files which contain the s3 bucket’s SSL CERT.  The files should be on all nifi nodes, owned by nifi user.

[#3]

Thanks for replying.

Please help me regarding these points are:

1)Does we need keystore and truststore file both or using any of them would work.

2)How we can create keystore and truststore files.

[#4]

For your example I would use truststore to "Trust Amazon's Cert".  You get the Amazon Cert, and create truststore with it.   There are many ways to get the Amazon Cert.  Recently, I found that a very easy way (windows) is to use a browser.  Visit an https url to your s3 host.  Click the SSL link, view certificate, download, etc.   On Nifi node add this cert file to /etc/nifi/ssl/ as amazon.cer.  Next the truststore command looks like:

/usr/jdk64/jdk1.8.0_112/bin/keytool -import -file /etc/nifi/ssl/amazon.cer -alias amazon -keystore /etc/nifi/ssl/truststore-amazon.jks

Make sure the file is chown nifi:nifi (owned to nifi user) and copied to all nodes.

The SSLContext controller service is then configured as:

[#5]

Thanks for replying.

1)I didn't understand exactly the way you suggested to download ssl certification.Please clear the steps once .

2)Please confirm that this is a free service or paid service.

[#6]

There are many ways to get a public cert.  I just recommend one I used recently.  You basically just click the lock in your browser.  In windows you can then download files from there.    If you are not on windows, or using command line you can reference this link for some other ways to get the public cert for amazon s3 urls:

https://serverfault.com/questions/661978/displaying-a-remote-ssl-certificate-details-using-cli-tools

Not sure what your last question refers to?

[#7]

Thanks for replying.

I had attached the screenshot of .cer file i had downloaded.

1)Please confirm the below steps

Now i will add .cer file at /etc/nifi/ssl/ as amazon.cer and trustore file at /trial as truststore-amazon.jks.I will change the jdk version as per version present on my cluster in truststore file.The content of truststore file will be like

/usr/jdk64/jdk1.8.0_77/bin/keytool -import -file /etc/nifi/ssl/amazon.cer -alias amazon -keystore /etc/nifi/ssl/truststore-amazon.jks

I will mention the truststore filename as /trial/truststore-amazon.jks in the configuration of fecths3 processor.

2)Using ssl certification is free service or i need to pay to amazon for it.

[#8]

@mayank_tripathi  It's free.  No issues there.

Yes I believe your summary is correct.  If you are using /trial folder, you can put the cer in there, and then execute the trustore command.  I like to keep all my files in the same place.   Just make sure when you are done that /trial folder is right permissions so nifi user can read the files and the files are copied to all nifi nodes.    If you do not do correct ownership and copy to all nodes part, the controller service will throw an error.  

[#9]

Hi to all. 

i'm having the same issue, even after following the procedure.

What i did:

1) get the ssl certificate from AWS using the ssl:

penssl s_client -showcerts -connect <source>:443 </dev/null | openssl x509 -text | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'

2) Copy the result to a *.crt file

3) convert the file to DEM

openssl x509  -in aws_cert.crt  -inform PEM  -out aws_cert.der -outform DER

4) Create the jks file using keytools

keytool  -import  -trustcacerts -alias aws3buckets  -file aws_cert.der  -keystore truststore-amazon.jks

5) change the permissions to be accessible from nifi.

6) add the file in the StandardSSLContextService and set password.

I receive the same e

SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Do you have any clue how to solve this?

I can use the crt file on the keytool?

There is any version of keytool that we need to use?

Thanks