Member since
07-30-2019
3472
Posts
1642
Kudos Received
1020
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 223 | 06-03-2026 06:06 PM | |
| 512 | 05-06-2026 09:16 AM | |
| 967 | 05-04-2026 05:20 AM | |
| 568 | 05-01-2026 10:15 AM | |
| 676 | 03-23-2026 05:44 AM |
09-30-2022
02:35 PM
@Kushisabishii What are you seeing in the nifi-user.log when you make this import attempt? You may be getting the 403 because the user is not authorized properly to perform the import call. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
09-30-2022
02:31 PM
@KD9 How long the NiFi server will validate a clients token is configured within the login-identity-providers.xml file via the following property: Authentication Expiration When setting up an automated process, using client tokens is not the best method. A better option would be to authenticate your client via a client certificate. With a client certificate, there is not need to obtain a token. That Client certificate will continue to work for the life of the certificate (certificates do have a valid until date set when you generate the certificate). So instead of passing a bearer token in your curl command, you would use your client pem key. The owner DN from the client certificate would be used as the user identity that you would then need to authorize in NiFi for the rest-api endpoint(s) needed for your automation. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
09-30-2022
02:19 PM
@John-MaxQ The MergeContent processor utilizes the Apache Commons Compress library which has a hard limit in tar size. There is an existing Apache NiFi jira for this here: https://issues.apache.org/jira/browse/NIFI-10273 If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
09-30-2022
02:13 PM
@leandrolinof ListSFTP and FetchSFTP are commonly used to fetch files from a remote server. In order to use ListFile or FetchFile the location remote directory would need to be mounted as a local file directory. Matt
... View more
09-30-2022
02:09 PM
@PepeClaro Can you share your processor configuration and NiFi version being used? Thanks, Matt
... View more
09-30-2022
02:04 PM
@ImranAhmed If I am understanding you correctly, you are trying to verify the content has correctly been modified before it is written out via your invokeHTTP processor.Inspecting that your dataflow is working as expected. If you stop your InvokeHTTP processor after the ReplaceText processor, the FlowFile processed by the ReplaceText will queue in the connection. You can then right click on the connection and list the queue. From there you can view/download the content of that FlowFile post ReplaceText to verify it contains the expected modified content before the InvokeHTTP is started and writes it to your DB. Thanks, Matt
... View more
09-26-2022
03:36 PM
@ImranAhmed I am not clear ion what you are trying to do here. You are looking for some exact string and replacing with nothing? The search value is a java regex so what little i can see is looking for an exact string match which is being compared against the "entire text". So entire text means entire contents of your FlowFile is being loaded into NiFi's JVM heap also. Why "entire text" instead of "line-by-line"?
... View more
09-26-2022
03:29 PM
1 Kudo
@knighttime You should not be configuring any of your NiFi processors to us the Event Driven scheduling strategy. It was not moved from an experimental method to production ready. Advances in the Time rDriven scheduling strategy has made int more efficient. So Event Driven is pretty much deprecated at this point in time. If you are not using Event Driven scheduling on any processor component in your NiFi, You should not be setting a large "Maximum Event Drive Thread Count" pool (default is 5, but i recommend setting to 1). While you can increase the Maximum while NiFi is running, reducing will require you to restart your NiFi. Now when it comes to the "Maximum Timer Driven thread count" pool. We can create a large pool which is per node in your 4 node cluster (80 thread X 4). Then you configure concurrent tasks on your individual processors to scale concurrency on each processor component. Also keep in mind that many processors execute to check connection inbound connection queues and those thread may only be active for micro seconds before being released back to the thread pool. so actually seeing full thread utilization represented in the status bar of your NiFi's may be difficult to see. Tips about concurrent task setting on processors. Setting high concurrent tasks configuration across many processors can be worse than leaving everything set at 1 in terms of overall performance. Start in the basement (1 concurrent task) and slowly increment concurrent tasks on processors as needed. You mention data queueing up, but it is difficult to tell you why or provide guidance without seeing your dataflow and knowing which processors have data backed up to them in their inbound connections and the configuration of those processors. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
09-22-2022
11:04 AM
@wasabipeas I can think of no way to force delete if it is blocking on a revision mismatch between nodes. Nothing here has anything to do with version control. Is it always the same node reported in the pop-up message that fails to process the request? If so, have you verified the libs and version running on that one node match rest of cluster? If you go to the cluster UI and select "VERSIONS" tab, they all reflect same version? You could manually disconnect the one node that it keeps complaining about from the "NODES" tab. After it is disconnected, you could delete it from the cluster (Deleting the nodes does nothing flows or data on that node. It will require a restart of that one node to get it to rejoin cluster). Once the node is removed form your cluster (temporarily), your cluster should reflect 10/10 connected nodes now in the status bar of the canvas UI. Check to see if your are still having revision issues with the process after reloading the page. If all looks good, you could access the filesystem of the currently disconnected and deleted node, stop the NiFi service on that node, and delete/rename the flow.xml.gz and flow.json.gz files. Then start this node again. On startup, NiFi will inherit the flow from the cluster and in doing so get the cluster flows current revision for the problematic process group. If problem persists, restart node that was deleted so that it rejoins the cluster. Then disconnected the currently elected cluster coordinator. A new cluster coordinator will then be elected by zookeeper. Check to see if issue with process group is resolved. Reload your browser to force a page refresh. If issue is resolved, rejoin node to cluster via the cluster UI to see if issue returns. If so, we at least know which is our problematic node. You can of course, disconnect, delete, rename flow.xml.gz and flow.json.gz, and then restart node, just as we performed before so that flow is pulled from cluster on startup. If issue still persists, there is something unique about this node. Disk space ok?, any exceptions in logs?, while node may report same NiFi version, something different with contents of lib(s) folders (get a checksum and compare against other nodes). Hope this helps without needing to restart entire cluster, Matt
... View more
09-22-2022
06:33 AM
@Chhavi Your question is not very clear. is there any way I can integrate clustered nifi for storing and fetching the value Storing and fetching what value? NiFi includes a couple Redis controller services: RedisConnectionPoolService RedisDistributedMapCacheClientService For example, The RedisDistributedMapCacheClientService could be used by the PutDistributedMapCache NiFi processor to write values to Redis. The FetchDistributedMapCache NiFi processor could be used to fetch values from Redis. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more