Member since
11-05-2019
25
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1297 | 01-15-2020 01:36 PM |
10-14-2020
05:34 AM
I ended up executing unzip from execute process and then a ListFiles to get the new files created by unzip command.
... View more
07-05-2020
03:16 PM
Sorry for the bump, but I am very courious of how everyone handles this kind of *possible* security issues. Thanks!
... View more
06-25-2020
06:39 PM
I have my credential stored in AWS Secret Manager, use a PutLamda to retrieve it and send it to a jsonextract and then invokehttp processor where I use the credentials in the body of a post message to get a bearer token. Between those processors the secret travels in plaintext. Is there a way to hide the credentials? The idea is to rotate the credentials on aws and the only way to view those is using aws console, doing this we can log the access to the secret, when and who. Thank you!
... View more
Labels:
- Labels:
-
Apache NiFi
06-04-2020
05:24 AM
Great! thanks!
... View more
06-03-2020
01:51 PM
Hi! I have a DBCPConnectionPool on NiFi 1.11.3 with this configuration: Normally it works ok, but sometimes the query get stuck on a PutSQL, no errors for minutes and then a timeout (very basic queries) I was trying some configurations on the pool but it seems I didn't quite understand the best way to troubleshoot this, I was thinking in each processor add a retry+counter+error. But don't know where to set the time out for queries from start of it and no per connection. Maybe a way is to not keep connections open until the query starts, how should I set the pool to generate the connection on request from a processor and terminate it before execution. I think it's a Athena problem with pools because I have no problem with another databases like postgres and mysql. Any ideas?? Thank you!
... View more
Labels:
- Labels:
-
Apache NiFi
05-24-2020
05:50 AM
Hi, it seems I forgot to set that varialble for the clean test I made to make the post, this is current setting: and this is the error I get with the context added: Thank you!
... View more
05-21-2020
01:27 PM
Hi, sorry for the bump, I don't know what other things to try, any lead will help. Thank you!
... View more
05-15-2020
11:36 AM
Hi i'm having a problem with self S2S reports, it's a NIFI 1.11.3 standalone (no cluster) linux deployment, this is the related configuration: # Site to Site properties
nifi.remote.input.host=localhost
nifi.remote.input.secure=true
nifi.remote.input.socket.port=10443
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs # web properties #
nifi.web.war.directory=./lib
nifi.web.http.host=
nifi.web.http.port=
nifi.web.http.network.interface.default=
nifi.web.https.host=my.domain.com
nifi.web.https.port=8443
nifi.web.https.network.interface.default=
nifi.web.jetty.working.directory=./work/jetty
nifi.web.jetty.threads=200
nifi.web.max.header.size=16 KB
nifi.web.proxy.context.path=
nifi.web.proxy.host= nifi.security.keystore=./conf/keystore.jks
nifi.security.keystoreType=jks
nifi.security.keystorePasswd=xxxxxxxxxxxx
nifi.security.keyPasswd=xxxxxxxxxxxxxxxx
nifi.security.truststore=./conf/truststore.jks
nifi.security.truststoreType=jks
nifi.security.truststorePasswd=xxxxxxxxxxxxxxxxx
nifi.security.user.authorizer=managed-authorizer
nifi.security.user.login.identity.provider=
nifi.security.ocsp.responder.url=
nifi.security.ocsp.responder.certificate= And the processor configuration, StandardRestrictedSSLContextService (using the self signed keystore from nifi that allows it to work securely) And the SiteToSiteBulletinReportingTask: with that configuration I receive this errors: SiteToSiteBulletinReportingTask[id=017111a7-83c2-1c18-25d3-ad4d5f780eb1] Error running task SiteToSiteBulletinReportingTask[id=017111a7-83c2-1c18-25d3-ad4d5f780eb1] due to org.apache.nifi.processor.exception.ProcessException: Failed to send Bulletins to destination due to IOException:null SiteToSiteBulletinReportingTask[id=017111a7-83c2-1c18-25d3-ad4d5f780eb1] org.apache.nifi.remote.client.PeerSelector@1e7445c6 Unable to refresh Remote Group's peers due to null If i change http to https I receive: SiteToSiteBulletinReportingTask[id=017111a7-83c2-1c18-25d3-ad4d5f780eb1] Error running task SiteToSiteBulletinReportingTask[id=017111a7-83c2-1c18-25d3-ad4d5f780eb1] due to org.apache.nifi.processor.exception.ProcessException: Failed to send Bulletins to destination due to IOException:sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target To test if Nifi could see itself I tried this, it's about the self signed certificate, so it seems ok: curl https://host123.internal:8443/nifi
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above. Ignoring the certificate I reach Jetty: curl --insecure -I https://host123.internal:8443/nifi
HTTP/1.1 302 Found
Date: Fri, 15 May 2020 18:21:49 GMT
Location: https://host123.internal:8443/nifi/
Content-Length: 0
Server: Jetty(9.4.19.v20190610) To secure the connection on installation (http to https) I used the security toolkit and copied the files to their locations under the nifi/conf folder as seen on the configuration file at top of the post: ./bin/tls-toolkit.sh standalone -n 'localhost(1)' -C 'CN=my_user,OU=NIFI' -O -o ../security_output Any idea of what can I be doing wrong with the S2S configuration? Thanks!
... View more
Labels:
- Labels:
-
Apache NiFi
03-24-2020
03:43 PM
Ok, this is all me for not understanding permissions correctly, I thought that if a permission wasn't configured it heredates the the permissions of NiFi. So: I'm Admin, I generated a group I should have access. You made me check again and I thank you for that!
... View more
03-23-2020
11:38 AM
Sorry for the bump, have more time than usual to solve this kind of problems, any idea? It's becoming hard to debug flows without this. Thank you!
... View more