NiFi processor to fetch attribute value dynamically from file or table

[#1]

I want processor to fetch attribute value on run time. Example - if I am filtering twitter feeds by specific keywords, i want to maintain the list of keywords in a separate repository like file or table and not confined as a text box value. In that case, how will NiFi processor fetch that values from external file or table to attribute value.

[#2]

It sounds like you might be able to use the ScanAttribute Processor to meet your needs.

[#3]

Hi. Appreciate your answer. Sorry for delay in response - I was away for a while. ScanAttribute processor sounds to be the answer to my question and I did try a sample but its not picking up value and filtering the incoming tweets accordingly. Can you provide an example with snapshots.

Incoming tweets via getTwitter Processor >> ScanAttribute processor. (dictionary file : <location of text file with keywords), attribute pattern - $.text, Match criteria - atleast one value must match)

[#4]

ScanAttribute processor works fine for matching exact value and not substring of attribute value. I looked into the code {{if (dictionary.contains(entry.getValue())) }} which means matching condition if dictionary words contain any of attribute words. In my case attribute is a tweet - collection of words and would fail all as unmatched.

Need to look at a logic to see if attribute : tweet (collection of words) contain dictionary words. Customization of processor or other ways of extract text from tweet attribute.

[#5]

@Timothy Spann : Greetings. I need your assistance in customizing processor - ScanAttribute.

Flow file data considering it as tweet_msg attribute needs to be checked if it contains dictionary file words.

[#6]

Scan attribute should work just put in a regular expression that does what you need and just use the one attribute for . maybe it's updated in the new one.

You can create your own processor, starting with the code in ScanAttribute if it doesn't meet your needs then deploy your NAR to the nifi/lib dir and restart nifi.

This is the formatting

https://github.com/tspannhw/nifi-nlp-processor

copy this code

https://github.com/apache/nifi/blob/f8cad0f8c94869e2fd2a30896033c3c48eadbf14/nifi-nar-bundles/nifi-s...

https://github.com/apache/nifi/blob/d838f61291d2582592754a37314911b701c6891b/nifi-nar-bundles/nifi-s...

[#7 · Accepted solution]

Thanks @Timothy Spann. Looked at your comments just now after i did some modifications to existing nar file. I edited the current ScanAttribute file to iterate the dictionary words and check for its presence in twitter message. Customizing is good and look forward to make more solutions integrated into NiFi.

[#8]

cool, put a JIRA ticket out for that and post it to your own github. perhaps that will get mainlined. Accept your answer.