Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi processor to fetch attribute value dynamically from file or table

avatar
Expert Contributor

I want processor to fetch attribute value on run time. Example - if I am filtering twitter feeds by specific keywords, i want to maintain the list of keywords in a separate repository like file or table and not confined as a text box value. In that case, how will NiFi processor fetch that values from external file or table to attribute value.

1 ACCEPTED SOLUTION

avatar
Expert Contributor

Thanks @Timothy Spann. Looked at your comments just now after i did some modifications to existing nar file. I edited the current ScanAttribute file to iterate the dictionary words and check for its presence in twitter message. Customizing is good and look forward to make more solutions integrated into NiFi.

View solution in original post

7 REPLIES 7

avatar
Contributor

It sounds like you might be able to use the ScanAttribute Processor to meet your needs.

avatar
Expert Contributor

Hi. Appreciate your answer. Sorry for delay in response - I was away for a while. ScanAttribute processor sounds to be the answer to my question and I did try a sample but its not picking up value and filtering the incoming tweets accordingly. Can you provide an example with snapshots.

Incoming tweets via getTwitter Processor >> ScanAttribute processor. (dictionary file : <location of text file with keywords), attribute pattern - $.text, Match criteria - atleast one value must match)

avatar
Expert Contributor

ScanAttribute processor works fine for matching exact value and not substring of attribute value. I looked into the code {{if (dictionary.contains(entry.getValue())) }} which means matching condition if dictionary words contain any of attribute words. In my case attribute is a tweet - collection of words and would fail all as unmatched.

Need to look at a logic to see if attribute : tweet (collection of words) contain dictionary words. Customization of processor or other ways of extract text from tweet attribute.

avatar
Expert Contributor

@Timothy Spann : Greetings. I need your assistance in customizing processor - ScanAttribute.

Flow file data considering it as tweet_msg attribute needs to be checked if it contains dictionary file words.

avatar
Master Guru

Scan attribute should work just put in a regular expression that does what you need and just use the one attribute for . maybe it's updated in the new one.

You can create your own processor, starting with the code in ScanAttribute if it doesn't meet your needs then deploy your NAR to the nifi/lib dir and restart nifi.

This is the formatting

https://github.com/tspannhw/nifi-nlp-processor

copy this code

https://github.com/apache/nifi/blob/f8cad0f8c94869e2fd2a30896033c3c48eadbf14/nifi-nar-bundles/nifi-s...

https://github.com/apache/nifi/blob/d838f61291d2582592754a37314911b701c6891b/nifi-nar-bundles/nifi-s...

avatar
Expert Contributor

Thanks @Timothy Spann. Looked at your comments just now after i did some modifications to existing nar file. I edited the current ScanAttribute file to iterate the dictionary words and check for its presence in twitter message. Customizing is good and look forward to make more solutions integrated into NiFi.

avatar
Master Guru

cool, put a JIRA ticket out for that and post it to your own github. perhaps that will get mainlined. Accept your answer.