I did explore the way to hash and dynamically can extract data as csv or avro as default by developing a custom processor. You can download the processor from HERE. The full source code is shared from here, feel free to contribute for further functionalities. https://github.com/vanducng/hashing-columns-nifi-processor   ... View more

Matt, thanks a lot for all your help. I was able to refactor my dataflow, reducing the number of groups and keeping everything simple in a single dynamic flow. Just to elaborate a little bit better, here's what I did. Data coming in CSV format separated by pipes. e.g.: (transaction #, sequence #, table code) 123|456|35| 123|456|36| 123|456|100| First I split the flowfile into multiple ones using SplitText >> then I used the ExtractText processor to grab the 3rd field (table code) >> LookupAttribute setting the user-defined-field schema.name (to be used by AvroSchemaRegistry controller service) >> Push the data to Kafka and Hive using the appropriate processors. Thanks a lot! ... View more

What issues are you having? That flow description seems like it should work. Perhaps your regular expression or other config of ExtractText needs tweaking? ... View more

@Shu, thank you very much. It worked perfectly! ... View more