Member since
07-30-2019
3406
Posts
1622
Kudos Received
1008
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 151 | 12-17-2025 05:55 AM | |
| 212 | 12-15-2025 01:29 PM | |
| 147 | 12-15-2025 06:50 AM | |
| 264 | 12-05-2025 08:25 AM | |
| 440 | 12-03-2025 10:21 AM |
03-02-2020
06:53 AM
@Gubbi What that stat is telling you is that the processor executed a total of 297 threads in the past 5 minutes. You can see that the cumulative time for all 297 threads that executed was only 0.044 seconds. So we can see that these 297 threads consumed in total very little time on the CPU. What this also tells us is that the processor executed about every 1 sec. This is either because your configured run schedule is every 1 sec (0 sec is default) or there where no new files found when it did each execution. To prevent a processor from consuming excessive CPU when the run schedule is set to 0 sec, NiFi will yield a processor after a thread runs which produces no results. What you have shown looks to be as expected behavior. Thanks, Matt
... View more
02-28-2020
12:45 PM
@Gubbi Not sure what you mean by spikes up to 300. A screenshot(s) maybe helpful to understand what you are observing.
... View more
02-28-2020
12:44 PM
1 Kudo
@maryem Any action you can do through the NiFi UI, you can also do via interacting directly with the NiFi rest-api. This will not animate the action of actually dragging and dropping a processor on the canvas, but you can make a rest-api call that would add new processor of type ABC at coordinates x,y on the canvas. NiFi's rest-api documentation can be found here: https://nifi.apache.org/docs/nifi-docs/rest-api/index.html Some users find it easier to learn the rest-api call through examples. If you open the developer tools in your browser, you can perform the action via the UI and see the rest-api call that was made. Most browser developer tools even let you save the rest-api call as a curl command that you could then execute yourself via command line. Matt
... View more
02-26-2020
12:41 PM
@Gubbi It is impossible to say what is going on here without more detail. From where are you determining NiFi is using greater than 100% CPU? If you are you are looking at top, with 8 cpu you would have 800% cpu available (100% for each of the 8 cpu). So 100%+ my be normal expected for the NiFi Java process. You have the NiFi core process running, plus each processor can execute concurrently. For example, FetchFile may be fetching the content of FLowFile 2 while PutS3 is putting FlowFile 1 to S3 at the same time. Since NiFi is designed to multi threaded, it is possible that multiple threads being executed concurrently, with each of those threads being handled by a different CPU. By default, the configured "Max Timer Driven Thread Count" in NiFi is set to 10. This means that across all processors 10 threads can be requested concurrently. This is a soft limit, so there are scenarios where the number of active threads can extend beyond the max configured thread count. Plus core level thread are used which do not come from this thread pool which is used only by NiFi components added to the canvas. What other processes are consuming CPU on your system? How is your NiFi configured, what are all the processors on your canvas, how are they configured, how large are the 3 files you are processing, what do the tasks/time stats on your processors show when this dataflow is executing against your 3 FlowFiles, etc? Hope this helps give you some direction to investigate. Matt
... View more
02-25-2020
11:20 AM
@nishank_paras You can use the invokeHTTP processor to fetch your file. Here is an example: The above example fetches the Apache nifi-toolkit-1.11.3-bin.tar.gz file. You can then construct a dataflow using other processors to manipulate as you want or simply just connect to another invokeHTTP processor that instead of "GET" uses "PUT" to put your file "nn.csv" at the new http endpoint. Hope this helps you, Matt
... View more
02-25-2020
11:05 AM
@saivenkatg55 You need to literally use ./keystore.p12 in your command instead of just keystore.p12 curl --cert-type P12 --cert ./keystore.p12:password --cacert nifi-cert.pem -v https://w0lxqhdp04:9091/nifi-api/flow/search-results?q= Hope this helps, Matt
... View more
02-25-2020
11:00 AM
@vineet_harkut I recommend starting a new question since this one already has a resolution. You'll get better responses that way. But the clientId can be whatever you want to use. For example even below would work: "revision":{"clientId":"1","version":0} The JSON response will have the UUID for the component you created which you would then use to modify the configuration of that component in future rest api calls. Thank you, Matt
... View more
02-24-2020
06:59 AM
1 Kudo
@justjoe With Apache NiFi 1.0 - 1.9 versions, only input and output ports added at the root canvas level (very top level) will be "remote" input/output ports. All input/output ports added within sub-process groups are local ports only. NiFi Remote Process Groups (RPG) are only able to send to or pull from "remote" input and output ports. With Apache NiFi 1.10 and newer you can now have the option of designating an input or output port as either local or remote when it is being added to the canvas within a sub process group. Hope this helps, Matt
... View more
02-18-2020
05:29 AM
@nishank_paras First we need to asks questions to better understand the http cloud endpoint: How do you currently accomplish this task outside of NiFi? Is it possible to connect via a terminal/command prompt to get the file from the http cloud server? Does the http cloud service have a rest-api for getting data from it? My first thought would be the invokeHTTP processor to get your file from the http cloud server... Then use an UpdateAttribute processor to change the "filename" attribute value on the NiFi FlowFile created by the invokeHTTTP processor... You did not mention what you want to do with this file once you have retrieved it ad renamed it in NiFi. Are you trying to write it back to the same http cloud service? Are you trying to write the file to another service/endpoint? Are you writing it to a local directory? Then we can start to look at what processors NiFi may have available to accomplish the same methods. Hope this helps get you started, Matt
... View more
02-17-2020
04:58 PM
1 Kudo
@thuylevn The ConvertJsonToAvro processor was removed from the default NiFi distribution bundle because of space limitations as of the Apache NiFi 1.10 release. https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance The ConvertJSONToAvro processor was part of the nifi-kite-nar https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-kite-bundle Hope this helps, Matt
... View more