Member since
03-06-2018
18
Posts
1
Kudos Received
0
Solutions
06-20-2018
10:10 PM
Hey Guys ( @Matt Clarke ) , Does anyone know why the SelectHiveQL processor does *not* copy the flowfile attributes of the incoming flowfile to the outgoing flowfile (unlike other processors)? I've had to do some crafty things to workaround this. For example, modify my SQL to embed the flowfile attributes I would like to propogate... ie: select interesting_stuff, 'attribute_1_val' as attribute_1_key from table. I was thinking of creating a custom processor by subclassing SelectHiveQL processor to fix this issue but I wanted to ask: 1) Does anyone know if this is a known issue with a known fix?
... View more
- Tags:
- nifi-processor
Labels:
- Labels:
-
Apache NiFi
05-27-2018
10:48 PM
Thanks Matt, that did the trick!
... View more
05-27-2018
05:55 PM
Hey Guys, Is it possible to access the Processor Group variables within the ExecuteScript Processor? I would specifically be interested in the syntax for doing that with Ruby. Any guidance would be appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
05-17-2018
10:26 PM
Hey Guys, In addition to sending back the status code (which I do with HandleHttpResponse => Http Response Code"), I would like to set a custom status message. For example: Status: 400 Status Message: "You are not authorized" Any pointers would be appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
05-17-2018
09:08 PM
Hey Guys, I have a situation where occasionally flow files appear to be stuck in a queue before an InvokeHttp processor. The flow file attributes show that none of the files are Penalized. I'm not sure how to diagnose the issue, any pointers would be appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
05-12-2018
10:28 PM
Hey Guys, We are running Nifi on a cluster of two nodes. In the last 5 weeks, the flow.xml.gz file has gotten out of sync twice. What are some situations that will defeat Nifi's replication of this file? Is there anything we can do to mitigate the likelihood of this happening again?
... View more
Labels:
- Labels:
-
Apache NiFi
05-12-2018
10:27 PM
Hey Guys, We are running Nifi on a cluster of two nodes. In the last 5 weeks, the flow.xml.gz file has gotten out of sync twice. What are some situations that will defeat Nifi's replication of this file? Is there anything we can do to mitigate the likelihood of this happening again?
... View more
Labels:
- Labels:
-
Apache NiFi
04-02-2018
04:46 PM
Thank you for the clarification.
... View more
04-02-2018
03:27 PM
Thank you for your detailed response, Matt @Matt Clarke. Yes, we are using HDF-3.1.0.0 and it is secured and using Ranger for NiFi authorizations. I see that this issue is resolved in Nifi 1.6.0. Is the recommended short term fix to upgrade Nifi?
... View more
04-01-2018
06:27 PM
1 Kudo
Hey Guys, We have a Nifi cluster that 5 days ago hung - the flow.xml.gz file got out of sync between the nodes due to prolonged connectivity issue. Ambari was also reporting a 98% JVM memory utilization (we have 32 Gigs). Once we resync'ed the flow.xml.gz file and restarted the nodes, the JVM memory dropped back down to 5%. Since then, the JVM memory has slowly climbed back up to 82% - which Ambari is warning with a color-coded orange. We don't have 'alot' of flows nor queued flowfiles (~300 flowfiles of less than 10k size), so I'm puzzled at the high JVM memory utilization. Any help diagnosing this issue would be greatly appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
03-11-2018
01:54 AM
Hey Guys, Is it possible to interpolate an attribute value into the regular expression for ExtractText processor? For example.. is it possible to rewrite the following so that Nifi accepts it? I would like to replace the placeholder ${key_value} with the attribute value for key_name... ${key_name}\s*:(\s*.+\n)
... View more
Labels:
- Labels:
-
Apache NiFi
03-06-2018
05:53 PM
@Matt Clarke Yes, my requirement is to encrypt passwords at rest. I notice that property values also appear in the database_repository/ files as well...
... View more
03-06-2018
04:49 PM
Thanks for the detailed response, @Matt Clarke. In addition to flow.xml.gz, can I assume that sensitive properties will be encrypted in all locations (logs, repositories)?
... View more
03-06-2018
02:37 AM
To whom it may concern: I am attempting to encrypt sensitive processor properties with the following snippet from my nifi.properties file: nifi.sensitive.props.key=1756eb0b-4eb3-46d6-98a4-a350b46c7459 nifi.sensitive.props.key.protected= nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL nifi.sensitive.props.provider=BC nifi.sensitive.props.additional.keys=my_secret I restarted Nifi but the value of the my_secret attribute remains unencrypted in flow.xml.gz. I've modified the value of my_secret several times but it is still saved in cleartext. Any help with this would be greatly appreciated. In addition, does anyone know what the value of "BC" means for the nifi.sensitive.props.provider key? Equivalently, anyone know where I can get a list of default sensitive properties?
... View more
Labels:
- Labels:
-
Apache NiFi