Member since
05-01-2018
5
Posts
0
Kudos Received
0
Solutions
05-01-2018
08:21 PM
@Abdelkrim Hadjidj Your article was helpful as well: https://community.hortonworks.com/articles/140231/data-flow-enrichment-with-nifi-part-2-lookupattrib.html
... View more
05-01-2018
07:48 PM
I ended up getting my whitelist to work. Here is what I did: Put whitelist into csv format Create a SimpleCSVLookupService Use LookupAttribute to pull value of ${someid}. Returns null if nothing matched Use RouteAttribute to test whither ${someid} equal the LookupAttribute There might be a better way where I don't have to use RouteAttribute to compare two variables but this is working for me as of now.
... View more
05-01-2018
04:24 PM
Here is what I am doing currently. Essentially based on a field from the EvaluateJsonPath, lets call it ${someid}, I need to match that variable to a whitelist. That whitelist exists in HDFS but it could be loaded to local file or some cache if needed. If there is a match then I want to HTTPPost if not then I will drop the message. The issue is that ScanAttribute will only accept a regular expression and not a variable using expression language ${someid}. I'm wondering if there is a better way. Maybe read in the whitelist to a CSV service and then lookup record... not sure how best to use a whitelist in Nifi.
... View more
05-01-2018
03:49 PM
Hi there, I have broken out the import piece now so it doesn't run in line which is fine for now. The problem I have now is that ScanAttribute doesn't support expression language so I can't give it a variable to scan for... any suggestions?
... View more
05-01-2018
03:22 AM
I am ingesting data from Kafka and then splitting out a JSON field. I then want to check that said fields content is in a whitelist that is kept in HDFS. I am looking at ScanAttribute and it seems like the thing I need except it can't read from HDFS. I tried reading from HDFS and then copying the file to local disk but that had some problems. Is there a better way?
KafkaConsume SplitJsonPaths ($.someid) ScanAttribute from HDFS (${someid})
Drop if ${someid} text isn't in HDFS file (new-line-delimited text file) Succeed if ${someid} text is in HDFS file Chad
... View more
Labels:
- Labels:
-
Apache NiFi