- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
extract attribute value from file dynamically in nifi
- Labels:
-
Apache NiFi
Created on 12-29-2020 02:25 AM - edited 12-29-2020 02:39 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I am trying to extract the single attribute(dynamic attribute passing) value from a JSON file or TEXT file .
The config file format looks like below
{
"product.csv" :"INSERT INTO Product",
"Wholesale.csv" :"INSERT INTO Wholesale"
}
The above JSON can evolve based new sources files and can added many file names with respective hive queries .
I have config file at shareserver and able to retrieve the into middle of the nifi flow as per the attached screen-print . I able to retrieve the attribute values using Evaluate JSON path with fixed number of attributes . But the attribute value can be changed and i can't configure the same attribute name in Evaluate JSON path processor . I understand that getting attribute value by dynamically passing attribute names not possible in nifi processor like JSON to Attributes .
I know that below JSON path is wrong for product.csv and it should be like $.product.csv . Assume that before Fetchconfig , i will update the "filename1" attribute to product.csv or wholesale.csv or any filename in Updateattribute processor . Using that attribute , i need to get the value from the config file dynamically .
I would need some work around to pass dynamic "filename" as attribute and extract the values for that attribute and store into next processor flowfile attribute or flowfile content . I am still new to Nifi expression language . Please assist . Currently i need to retrieve 1 attribute value passing dynamic attribute value like filename
Created 01-05-2021 01:40 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thoughts
maybe queryrecord
maybe partitionrecord
get one row at a time
perhaps JoltRecordprocessing
see these articles
https://www.datainmotion.dev/2020/12/smart-stocks-with-flank-nifi-kafka.html
https://www.datainmotion.dev/2020/12/simple-change-data-capture-cdc-with-sql.html
https://www.datainmotion.dev/2020/12/ingesting-websocket-data-for-live-stock.html
