Member since
07-30-2019
3406
Posts
1621
Kudos Received
1006
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 44 | 12-17-2025 05:55 AM | |
| 105 | 12-15-2025 01:29 PM | |
| 70 | 12-15-2025 06:50 AM | |
| 201 | 12-05-2025 08:25 AM | |
| 342 | 12-03-2025 10:21 AM |
04-06-2018
05:15 PM
2 Kudos
Thank you, but I am not @Matt Burgess LOL Forum TIP: Try to avoid responding to answers with new answers. Instead add comments to answers you are responding to.
... View more
04-06-2018
12:08 PM
1 Kudo
@Vivek Singh Try the following configuration in your splitJson processor: Thanks, Matt If you found this answer addressed your initial question, please take a moment to login and click "accept".
... View more
04-06-2018
12:02 PM
2 Kudos
@Tanmoy NiFi templates where never intended when they were designed for the purpose of version control. A template by design bundles everything that the selected components need to work. Also to make sure templaet could be deployed to any other environment, all sensitive properties are cleared. The NIFi team realized that design made it less then ideal for dataflow promotion between environments. That is where the NiFi Registry came int to the picture (Available as of Apache NiFi 1.5 and HDF 3.1.x) https://nifi.apache.org/registry.html The NIFi Registry runs as an external service but complimentary service to NiFi. Multiple NiFi instances/clusters can be configured to access the same registry. User can start version control on process groups within NiFi which stores a version controlled copy of the flow (similar to a template) in the NIFi Registry. User on another NiFi instance/cluster can connect to the same registry and load that flow on to their NiFi. Initially adding a flow form the registry works just like templates (Need to enter sensitive properties and controller services are copied over.) What is different is how that info is preserved through version upgrades from that point forward. Let say on system one (dev) and new version of that version controlled process group is pushed to the registry. On System two they can upgrade to version 2 without needing to reset sensitive properties or worry about getting another copy of a controller service. Thank you, Matt *** If you found this answer addressed your initial question, please take a moment to login and click on "Accept".
... View more
04-05-2018
04:58 PM
@Anishkumar Valsalam TIP: Do not respond to an answer with a new answer. If you are responding to an existing answer thread, use comments. The threads status total shown in the status bar can include threads active for things other then just processor components. For example: Controller Services and Reporting Tasks. Thank you, Matt
... View more
04-05-2018
01:39 PM
@Anishkumar Valsalam In HDF 3.0 the thread stats in the status bar also reflect the total number of threads allocated to the Event Driven Thread pool. Is it possible the event driven thread pool was ever increased and then later reduced back to 5? Reductions to the Event Driven thread pool allocation will not take affect until a restart of NIFi.
... View more
04-04-2018
08:08 PM
@Patrick There may issues external to a processors code execution that can result in a stoppage in a dataflow. The Monitor activity processor can be placed inline anywhere in your flow to monitor FlowFiles passing through it on a configurable interval. The processor then will generate alert type FlowFiles when a activity is outside of those thresholds. You could then send those generated alert messaged to a putEmail processor to send out a notification of an outage. Not all failures are unexpected, Often times a second attempt is successful. Setting up a count loop is a good way of having FlowFiles that are routed to failure to to try x number of times in a loop before taking some other action. https://cwiki.apache.org/confluence/download/attachments/57904847/Retry_Count_Loop.xml?version=1&modificationDate=1433271239000&api=v2 You may also want to consider looking at the SiteToSiteBulletinReportingTask. NiFi processor produce bulletins for ERROR and WARN level logs. With this reporting task, you can feed these produced bulletins back to same NiFi or another NiFi where they can be handled like any other FlowFile and pushed to what ever database you want. Thank you, Matt *** If you found this answer addressed your question, please take a moment to login and click "accept"
... View more
04-04-2018
07:40 PM
@Anishkumar Valsalam
How are you determining that only 12 threads are actually being used? The UI is merely a snapshot in time, it does not update in real time. What version of HDF are you running? What is "Max Timer Driven Thread Count" set to? What is "Max Event Driven Thread Count" set to? How many nodes in your NiFi cluster? Thanks, Matt
... View more
04-03-2018
01:43 PM
@ANKIT PATEL Username is a required field in all of the NiFi FTP processors (GetFTP, ListFTP, and FetchFTP). Connecting to a FTP endpoint that does not require a username may require you to write your own custom processor implementation. Another option might be to use one of the NiFi Scripting processors that calls a custom script you would need to write to return files from your FTP server. Thank you, Matt
... View more
04-02-2018
04:36 PM
@Sonny Chee Apache NiFi 1.6 has not been released yet. It also will not include with it the Ranger plugin stuff that is included with the HDF NiFi releases versions. If you have a support contract with Hortonworks for HDF, you could requests a hotfix build from them. The HDF NiFi releases are never identical to any Apache NiFi release. Some processors are modified to support HDP better and some additional libraries are added (such as Ranger). Just as a note, you can always think of the HDF releases as being build on Apache NiFi <version> plus more.
... View more
04-02-2018
01:14 PM
2 Kudos
@Sonny Chee What version of HDF are you running? Is it HDF 3.1? Is your NiFi secured and using Ranger for NiFi authorizations? If so, you may have run into this identified bug: https://issues.apache.org/jira/browse/NIFI-4925 There are currently no HDF releases with the fix included. Thank you, Matt
... View more