question Re: NiFi Code to Description mapping in Archives of Support Questions (Read Only)

NiFi Code to Description mapping

Raj_B — Wed, 23 Nov 2016 01:07:24 GMT

Hello,

Wanted to find out what's the NiFi best practice for a use case I am working on;

The incoming data has codes and the codes should be replaced with the corresponding code descriptions, from a manually maintained static (almost static, rarely changes) list of codes and their corresponding descriptions in a file.

Is ReplaceTextWithMapping the right processor to implement this.

Thanks.

Re: NiFi Code to Description mapping

andrewg — Wed, 23 Nov 2016 01:11:21 GMT

Depending on the format of your data, you might be better off using the ScanAttribute https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ScanAttribute/index.html It will reference an external file for a match and route the flowfile accordingly. E.g. follow up with an UpdateAttribute to update/add to the metadata.

Re: NiFi Code to Description mapping

JoeWitt — Wed, 23 Nov 2016 01:13:39 GMT

This is a pretty common pattern and something we should add more out of the box support for. This is a lot like how the GeoEnrich processor works. There is some reference dataset and some incoming data which needs to be enriched/altered based on pulling keys from the data and looking up their values in the dataset.

I'd recommend you provide a ControllerService that loads/monitors changes to your reference dataset that offers a 'get(key)' type lookup where the value returned is the value you want to place into your data.

Then provide a custom processor that uses that controller service against your data. You could do both of these using the scripting support in Groovy, for example.

You could also just do this in a single processor as well.

Hope that helps

Re: NiFi Code to Description mapping

jfrazee — Wed, 23 Nov 2016 01:19:40 GMT

@Raj B I laid out some of the options in a recent mailing list discussion [1]. These include both of the suggestions above, including a link to a work-in-progress implementation for the ControllerService approach.

1. http://mail-archives.apache.org/mod_mbox/nifi-dev/201611.mbox/%3c31b8fe4f-95f6-4419-80a9-f9a728a9cb7c@me.com%3e

Re: NiFi Code to Description mapping

Raj_B — Wed, 23 Nov 2016 02:14:36 GMT

@jfrazee @Andrew Grande @jwitt thank you all for the ideas and information.