About MattWho

MattWho · ‎07-07-2022

@sayak17 If you are simply looking to GET from a REST API endpoint and take the response and write to a local file on the server where your NiFi service is running, you'll want to use the InvokeHTTP processor and feed that to your putFile processor. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt

MattWho · ‎07-07-2022

@Reddy3 There is not enough history or detail to understand what lead to the corruption of your NiFi HD DBs. However, the NiFi H2 DBs are used to hold flow configuration history (what user did what on the canvas) and authenticated user server side tokens. These DBs can be removed/renamed and on startup NiFi will created new DBs. Removing these will have no impact on your flow.xml, content, etc... You will simply lose your flow configuration history (will start over from this point forward tracking changes) and your users will be forced to login again. Hope this helps, Matt

MattWho · ‎07-05-2022

@pk87 Also consider that using timer driven may not always give you 60 minute scheduling. When using timer driven the component will get scheduled upon start and then again x configured amount of time later. A NiFi restart or stopping and starting the processor will reset this. If you need to make sure that a component is only scheduled every x amount of time consistently, you should be using cron driven scheduling strategy which will allow you to set specific time of schedule. Thanks, Matt

DianaTorres · ‎07-04-2022

@shuhaib3 Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks!

MattWho · ‎07-01-2022

@Brenigan Are you running your dataflow on a standalone NiFi install or a NiFi cluster install? If a multi node NiFi cluster, are all 200 FlowFiles on the same NiFi node? Does your partition_number start at 0? Do you see your FlowFiles getting routes to the overtook relationship after 10 minutes? Assuming all the following: 1. All FlowFiles are on same NiFi node 2. partition_number starts at "0" and "increments consistently by "1" 3. All FlowFiles have same filename 4. Wait relationship is route via a connection back to the EnforceOrder processor. You should be seeing: 1. All FlowFiles routed to the "wait" relationship until a FlowFile with attribute "partition_number" equal to "0" is processed which will result in that FlowFile routing to success. 2. Other FlowFiles meeting above 4 criteria will continue to loop through wait until "partition_number" attribute with value "1" is seen and routed to success. 3. If a FlowFile in incremental order is missing, all FlowFiles with a partition_number higher than the next expected integer will continue to route to wait relationship. 4. after the configured "wait timeout" any FlowFile that has been waiting this long will be routed to the "overtook" relationship. You can right click on a connection holding the FlowFiles and list the queue. From there you can select the "view details" icon to the far left to examine the FlowFiles current attributes. You should see a new attribute "EnforceOrder.expectedOrder" that contains the next expected integer value that the group this FlowFile belongs to is waiting for. You will also find your "partition_number" which will have the current integer for this FlowFile. If you have your FlowFiles distributed across multiple nodes in a NiFi cluster, you will need to get all FlowFiles with the same "group identifier" moved to the same NiFi node in order to enforce order (you can not enforce order across different nodes in a NiFi cluster). You can accomplish this by editing the connection feeding your enforceOrder processor and under settings select a "Load Balancing Strategy" of "Partition by Attribute" using the "filename" attribute that you are using as your group identifier in the Enforce Order processor. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt

rafy · ‎06-29-2022

Thank you so much Mr @MattWho for comprehensive explanation on what Mr @SAMSAL earlier proposed. I really learn alot & appreciate.

MattWho · ‎06-29-2022

@Tryfan You mention this file comes in daily. You also mention that this file arrives through a load-balancer so you don't know which node will receive it. This means you can't configure your source processor for "primary node" only execution as you have done in your shared sample flow with the ListFile. As Primary Node only, the elected primary node will be the only node that executes that processor. So if the source file lands on any other node, it would not get listed. You could handle this flow in the following manor: GetFile ---> (7 success relationships) PostHTTP or InvokeHTTP (7 of these with one configured for each node in your cluster cluster) ListenHTTP --> UpdateAttribute --> PutFile So in this flow, no matter which node receives your source file, the GetFile will consume it. It will then get cloned 6 times (7 copies then exist) with one copy of the FlowFile getting routed to 7 unique PostHttp processors. Each of these targets the ListenHTTP processor listening on each node in your cluster. That ListenHTTP processor will receive all 7 copies (one copy per node) of the original source file. Then use the UpdateAttribute to set your username and location info before the putFile which place each copy in the desired location on the source node. If you add or remove nodes from your cluster, you would need to modify this flow accordingly which is a major downside to such a design. Thus the best solution is still one where the source file is placed somewhere all nodes can retrieve it from so it scales automatically. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt

DianaTorres · ‎06-27-2022

@yagoaparecidoti Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks

araujo · ‎06-22-2022

@nada , Please check this solution: https://community.cloudera.com/t5/Community-Articles/Decompressing-nested-ZIP-files-in-NiFi/ta-p/346169 Cheers, André

arshahin · ‎06-22-2022

Dear @MattWho Thanks a lot for the response. I installed Java SDK 11 and that helps to run NiFi on my laptop. I also change the environmental variables. I add JAVA_HOME under user variables and defined it like %JAVA_HOME%\bin in the path and now it is working fine. For my desktop I did the same thing and I notice NI applications (LabView) installed on my desktop was also using port 8080, so I fixed that one as well. NiFi is working fine on my desktop as well. I also found the following link relevant and useful for any other community members getting the same error: https://stackoverflow.com/questions/69830860/nifi-will-run-but-wont-open-in-browser-possible-error-with-java All the best Alireza

Online	Offline
Last Visited	‎06-11-2026 12:54 PM

Member Since	‎07-30-2019 10:41 AM
Last Visited	‎06-11-2026 12:54 PM
Posts	3,471
Kudos received	1638

Cloudera Community

Re: Can we detect who did a particular operation i...

Re: How to invoke a url in nifi which is protected...

Re: Retry impacts scheduler

Re: 503 error while copying/versioning big process...

Re: FetchSMB not fetching all files

Re: How to use the RestLookupService to make REST ...

Re: Error creating bean with name 'org.springframe...

Re: Regarding NIFI Timer driven scheduling not wo...

Re: Nifi CLI PKIX path building failed

Re: EnforceOrder processor doesn't work.

Re: Can NIFI nodes access different records on a D...

Re: How to replicate a file in NiFi

Re: NIFI - Unknown user with identity 'user'. Cont...

Re: Decompress .json.gz inside folder in NIFI

Re: An unexpected error has occurred. Please check...