Member since
10-09-2019
12
Posts
0
Kudos Received
0
Solutions
03-08-2020
11:01 PM
Thanks @MattWho
... View more
11-06-2019
05:32 AM
2 Kudos
@girish6 NiFi processor components are configured to execute based on a run schedule. There are two schedule driven strategies available (Cron Driven and Timer Driven). The Cron Driven scheduling strategy uses a user configured Quartz Cron to set how often the processor will execute. The Timer Driven scheduling strategy (most common strategy used) uses a user configured run schedule (default run schedule is 0 secs, which means run as often as system will allow). When a processor executes based on the configured scheduling strategy, it will do one of two things: 1. If the processor has one or more inbound connections, it will check if any of them have any queued FlowFiles. If none of the connections contain any queued FlowFiles, the processor will yield. The yield is intended to keep the processors with run schedule of 0 secs from simply constantly requesting CPU threads to check empty inbound connection queues. No matter the run schedule, a yielded processor will not execute until the yield has expired reducing CPU usage by that processor. 2. Some processor have no inbound connections. These processors will not yield, but continuously execute on the configured run schedule. You would not have any such processors in your PG2 since they will have upstream connections to components in PG1. So for "source" type processors like listSFTP, ListFile, GenerateFlowFIle, or any other processor that does not support an inbound/upstream connection, if the feed of data is not continuous, it is best to use the Cron Driven scheduling strategy or set a Timer Driven run schedule that is not the default 0 secs to reduce CPU usage. On the face of every processor is a state for Tasks/Time. The stat tells you how many threads reported as completed in the past 5 minutes and how much cumulative CPU time was used by all those completed threads. This allows you to see the impact a given processor is having on your CPU. Hope this helps explain cpu usage for you, Matt
... View more
10-10-2019
10:15 AM
Hello @girish6 Your first EL statement would work as well once you correct the single quote marks around the Z. Many outside text editors end up messing up the single quotes, so the NiFi EL ends up being incorrect when you copy/paste. NiFi's EL editor built in to the processor property text blocks can help identify when the EL format is wrong. In above you can see your NiFi EL statement as copied and pasted on line 3. You'll notice the color coding indicates an issue with the function's input parameters. Line 1 has the properly formatted version of that same NiFi EL statement. Be careful when using copy paste from some external editors. Thank you, Matt
... View more
10-10-2019
09:58 AM
1 Kudo
Hello @girish6, NiFi Expression language does have a function that will increment a number, but it does not reset back to zero (unless you restart NiFi) and is incremented globally across call calls to that function. So while this function will give each FlowFile a unique filename per NiFi node, there is no guarantee that they will be sequential or will start at 0 for new source zip file you extract. If this works for your use case the NiFi Expression language statement would look like this in the UpdateAttribute processor: If you are just looking for filename uniqueness, a better solution may be to instead append the FlowFile's uniquely assigned UUID to the filename instead as follows: Tips: 1. There is an expression language guide in the embedded help of your NiFi installation found by clicking on the global menu icon in upper right corner. 2. Within a NiFi processor property that support NiFi EL you can display a list of functions with details by typing "${" (signifies start of an EL statement) and then hold "Control" while clicking space bar. This also works if you are mid statement and type ":"then "control" + click spacebar. Examples: file-${ (then cntrl+spacebar) ${uuid: (then cntrl+spacebar) Hope this helps, Matt
... View more