Member since
03-05-2020
3
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
7104 | 03-19-2020 02:56 AM |
03-19-2020
02:56 AM
A co-worker found the 'simple solution' to the problem I asked and I would like to share so it can help others. What was missing was the property sslmode = require. After including that, the service worked perfectly. Actually, I am not sure why it didn't work with sslmode = prefer, since it is what my pgadmin is using for the same database and there it works perfectly. It seems like we must 'force' nifi to use ssl in this case - see documentation here: https://jdbc.postgresql.org/documentation/head/ssl-client.html. Moreover, some insights: 1) It worked with certificate in '.der' and key in '.pk8' formats (didn't have to use trust-store and key-store as needed in other services). 2) One can add in the 'plus' button the properties and give them the right name as we would do in java code, instead of concatenating every property in the connection string (see second option in the question above). Make it helps others as well.
... View more
03-09-2020
03:53 AM
Thank you, for your answer. Unfortunately, not the DBCPConnectionPool service, nor the ExecuteSQL processor provide option for SSLContextService. Therefore, I need another solution and it suppose it must be in the url connection string, but not sure.
... View more
03-05-2020
01:36 AM
I have a NIFI image running in openshift and a postgres in the cloud "owned" by another department.
They sent to us a certificate(".crt") and a key(".key"), since the log-in is made trough client certificate, instead of username and password. I have succeeded to log-in in pgAdmin 4, but not to connect the NIFI to the Postgres with certificate and key.
I have uploaded the certificate and the key to the image (using a secret and mounting it) so if I go to the pod terminal I can access it.
But, when I pass to a DBCPConnectionPoll service the connection string bellow and activates an ExecuteSQL processor, I receive an exception that the certificate is not valid, as follows:
ERRORExecuteSQL[id=...] Unable to execute SQL query <...>;due to java.sql.SQLException: Cannot create a PoolableConnectionFactory (FATAL: connection requires a valid client certificate).
No FlowFile to route to failure: org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException: Cannot create a PoolableConnectionFactory (FATAL: connection requires a valid client certificate)
I have tried to pass the certificate in two ways to the DBCPConnectionPoll service:
1) as parameters in the connection string ("database connection url" property:
jdbc:postgresql://<ip>:<port>/<username>?user=<username>&sslTrue&sslcert=/etc/.../mycerts/mycert.der&sslkey=/etc/.../mycerts/mykey.key.pk8
2) adding properties in the DBCPConnectionPoll service (+ button and the just adding parameter name and the path as the value) and just passing this as url:
jdbc:postgresql://<ip>:<port>/<username>
Both seems to work generally speaking, since I can connect to another postgres I have which not requires ssl certification.
Some considerations:
1) My assumption here is that the connection string in the NIFI does not know to read properly the file path for the certificate and key.
2) I have converted the certificates a bunch of times to different types that java can receive in order to see if that was the problem, but I still receive the same exception. So it seems that the connection pool just does not "achieve" the files at all. Nevertheless, if some one has a say in this topic, it can be handy, after the main problem is solved. So appreciate some tips here as well.
3) I have also read the NIFI source code and it seems that NIFI uses normally JDBC classes to create the connection pool, so a connection string as I passed would have worked in java code, but somehow doesn't work in NIFI (which is written in java).
4) The jdbc driver and everything else is configured properly, since I can work with a non-secure postgres in NIFI.
Thank you very much.
... View more
Labels:
- Labels:
-
Apache NiFi