- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
NiFi Code to Description mapping
- Labels:
-
Apache NiFi
Created 11-22-2016 05:07 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Wanted to find out what's the NiFi best practice for a use case I am working on;
The incoming data has codes and the codes should be replaced with the corresponding code descriptions, from a manually maintained static (almost static, rarely changes) list of codes and their corresponding descriptions in a file.
Is ReplaceTextWithMapping the right processor to implement this.
Thanks.
Created 11-22-2016 05:19 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Raj B I laid out some of the options in a recent mailing list discussion [1]. These include both of the suggestions above, including a link to a work-in-progress implementation for the ControllerService approach.
Created 11-22-2016 05:11 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Depending on the format of your data, you might be better off using the ScanAttribute https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ScanAttribute/... It will reference an external file for a match and route the flowfile accordingly. E.g. follow up with an UpdateAttribute to update/add to the metadata.
Created 11-22-2016 05:13 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is a pretty common pattern and something we should add more out of the box support for. This is a lot like how the GeoEnrich processor works. There is some reference dataset and some incoming data which needs to be enriched/altered based on pulling keys from the data and looking up their values in the dataset.
I'd recommend you provide a ControllerService that loads/monitors changes to your reference dataset that offers a 'get(key)' type lookup where the value returned is the value you want to place into your data.
Then provide a custom processor that uses that controller service against your data. You could do both of these using the scripting support in Groovy, for example.
You could also just do this in a single processor as well.
Hope that helps
Created 11-22-2016 05:19 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Raj B I laid out some of the options in a recent mailing list discussion [1]. These include both of the suggestions above, including a link to a work-in-progress implementation for the ControllerService approach.
Created 11-22-2016 06:14 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@jfrazee @Andrew Grande @jwitt thank you all for the ideas and information.
