Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi Code to Description mapping

avatar
Expert Contributor

Hello,

Wanted to find out what's the NiFi best practice for a use case I am working on;

The incoming data has codes and the codes should be replaced with the corresponding code descriptions, from a manually maintained static (almost static, rarely changes) list of codes and their corresponding descriptions in a file.

Is ReplaceTextWithMapping the right processor to implement this.

Thanks.

1 ACCEPTED SOLUTION

avatar

@Raj B I laid out some of the options in a recent mailing list discussion [1]. These include both of the suggestions above, including a link to a work-in-progress implementation for the ControllerService approach.

1. http://mail-archives.apache.org/mod_mbox/nifi-dev/201611.mbox/%3c31b8fe4f-95f6-4419-80a9-f9a728a9cb7...

View solution in original post

4 REPLIES 4

avatar

Depending on the format of your data, you might be better off using the ScanAttribute https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ScanAttribute/... It will reference an external file for a match and route the flowfile accordingly. E.g. follow up with an UpdateAttribute to update/add to the metadata.

avatar

This is a pretty common pattern and something we should add more out of the box support for. This is a lot like how the GeoEnrich processor works. There is some reference dataset and some incoming data which needs to be enriched/altered based on pulling keys from the data and looking up their values in the dataset.

I'd recommend you provide a ControllerService that loads/monitors changes to your reference dataset that offers a 'get(key)' type lookup where the value returned is the value you want to place into your data.

Then provide a custom processor that uses that controller service against your data. You could do both of these using the scripting support in Groovy, for example.

You could also just do this in a single processor as well.

Hope that helps

avatar

@Raj B I laid out some of the options in a recent mailing list discussion [1]. These include both of the suggestions above, including a link to a work-in-progress implementation for the ControllerService approach.

1. http://mail-archives.apache.org/mod_mbox/nifi-dev/201611.mbox/%3c31b8fe4f-95f6-4419-80a9-f9a728a9cb7...

avatar
Expert Contributor

@jfrazee @Andrew Grande @jwitt thank you all for the ideas and information.