Member since
09-27-2018
138
Posts
23
Kudos Received
10
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 12814 | 02-28-2021 10:23 PM | |
| 2897 | 02-08-2021 11:53 PM | |
| 38738 | 12-16-2020 11:31 PM | |
| 10503 | 12-14-2020 11:02 PM | |
| 7977 | 12-14-2020 12:18 AM |
11-08-2023
04:03 AM
1 Kudo
Hello Vidya, would you support me in this as well? regards, Mahrous Badr
... View more
03-12-2023
01:40 AM
{
"operation": "default",
"spec":{ "*" : {
"flow_file_id" : "${uuid}" }
}
} this spec is generating the same uuid for all the objects in the array of json.
... View more
12-17-2021
06:18 AM
Hello @alim, Can you please indicate where I can find the ./conf/providers.xml file ? Also regarding this "Flow snapshot histories are managed as Git commits, meaning only the latest version of Buckets and Flows exist in the Git directory.", is there any way to save all versions of a specific flow in GIT? Thank you!
... View more
02-28-2021
10:23 PM
1 Kudo
@murali2425 Here are two possible solutions. At solution 1 the value is set into an array. At solution 2 I took the flowfile-content (json) and set it into an attribute. Then you can work with expression language to get the value. For testing the syntax I recommend this site: http://jsonpath.herokuapp.com/
... View more
02-17-2021
05:35 AM
@justenji Thanks a lot! The suggestion you proposed works great! However, the source of the problem was not a lack of schema definition. In fact, I managed also to have a working flow with the option Schema Access Strategy = Use String Fields From Header in the CSVReader processor. The warning "None of the fields in the record map to the columns defined by the tablename table." is also obtained when the processor is not able to find the table and this can happen also when the table name is correctly configured in PutDatabaseRecord but there is some issue with user access rights (which ended up to be the actual cause of my error ...).
... View more
02-08-2021
11:53 PM
Hi @Jarinek I think you have to set -pv in front of the new value: nifi set-param -u http://localhost:8088 -pcid 85b8b5bf-0177-1000-3881-8ddd8d31fed8 -pn sample.URL -pv newvalue This one worked on my local NiFi. Remark: Haven't seen that you found it yourself. Great!
... View more
12-24-2020
07:35 AM
@adhishankarit There is nothing you can pull form the NiFi Rest-API that is going to tell you about successful outcomes form processor execution on a FlowFile. Depending on data volumes, this also sounds like a resource expensive endeavor. That being said, NiFi does have a Site-To-SIte (S2S) Bulletin reporting task. When a processor throws an error it will produce a bulletin and this reporting task can capture those bulletins and send them via NiFi's S2S protocol to another NiFi instance directly into a dataflow were you can handle via dataflow design however you like. Only way you can get INFO level logs in to bulletins is by setting the bulletin level to INFO on all you processors. This only works if you have also configured your NiFI logback.xml so that all NiFi components log at the INFO level as well. Downsides to this: 1. Every processor would display the red bulletin square in upper right corner of processor, This makes using this to find components that are having issues difficult 2. This results in a lot of INFO level logging to the nifi-app.log. You mention edge nodes. You could setup a tailFile processor that tails the nifi-app.log and then send via a dataflow that log data via FlowFiles to some monitor NiFi cluster where another dataflows parses those records via a partitionRecord processor by log_level and then routes based on that log_level for additional handling/notification processing. Downside here: 1. Since you what to track success, you still need INFO level logging enabled for all components. This means even this log collection flow is producing log output. So large logs and logs being written to even when actual data being processed in other flows is not happening. NiFi does have a master bulletin board which you could hit via the rest-api, but this does not get you past the massive logging you may be producing to monitor success. https://nifi.apache.org/docs/nifi-docs/rest-api/index.html Hope this gives you some ideas, Matt
... View more
12-18-2020
05:42 AM
1 Kudo
Excellent news. Can you accept the first 2 responses to close this solution?
... View more
12-15-2020
12:44 AM
@justenji Yes, exactly. All "main" elements are part of an array in the input.
... View more
12-14-2020
07:23 AM
Hello @justenji Thanks for the detailed explanation, this is also what I have tested on my side and in my opinion this does the job being "record oriented" 🙂 Have a nice day
... View more