Member since
02-02-2018
20
Posts
0
Kudos Received
0
Solutions
11-23-2019
08:51 AM
I did explore the way to hash and dynamically can extract data as csv or avro as default by developing a custom processor. You can download the processor from HERE. The full source code is shared from here, feel free to contribute for further functionalities. https://github.com/vanducng/hashing-columns-nifi-processor
... View more
07-27-2018
03:33 PM
Matt, thanks a lot for all your help. I was able to refactor my dataflow, reducing the number of groups and keeping everything simple in a single dynamic flow. Just to elaborate a little bit better, here's what I did. Data coming in CSV format separated by pipes. e.g.: (transaction #, sequence #, table code) 123|456|35| 123|456|36| 123|456|100| First I split the flowfile into multiple ones using SplitText >> then I used the ExtractText processor to grab the 3rd field (table code) >> LookupAttribute setting the user-defined-field schema.name (to be used by AvroSchemaRegistry controller service) >> Push the data to Kafka and Hive using the appropriate processors. Thanks a lot!
... View more
07-27-2018
05:53 PM
What issues are you having? That flow description seems like it should work. Perhaps your regular expression or other config of ExtractText needs tweaking?
... View more
02-05-2018
02:03 PM
@Shu, thank you very much. It worked perfectly!
... View more