Member since
04-08-2021
5
Posts
0
Kudos Received
0
Solutions
06-13-2021
10:58 PM
@AnkushKoul , did @MattWho's response resolve your issue? If so, can you please mark it as the solution? It will make it easier for others to find the answer in the future.
... View more
05-05-2021
06:10 AM
@AnkushKoul You may consider having two dataflows here because you would need to store this password somewhere that your putSFTP processor dataflow would need have access to. One dataflow that acquires the password from the rest endpoint and another that is actually performs your putSFTP. Flow to get password would consist of a GenerateFlowFile (configure to generate a FlowFIle on each NiFi node using a run schedule to control how often password will be refreshed)--> invoketHTTP --> putDistributedMapCache (Configured to use DistributedMapCacheClient that uses "localhost" as server hostname. Will need to also add a DistributedMapCacheServer Controller service that above client will talk to. This will make sure each node grabs password and stores it locally on that nodes's DMCServer) Then in your flow using putSFTP, you would replace the invokeHTTP processor a fetchDistributedMapCache processor just before the putSftp processor (suggest looping failure relationship from putSFTP back to fetchDistributedMapCache processor just in case password changes or there was no password stored when fetch was attempted first time). Keep in mind that flows like this are not idea. The retrieved password that you are placing in to a FlowFile attribute is stored in plaintext. If you found this solution helpful, please take a moment to login and click accept on this solution. Good luck, Matt
... View more
04-12-2021
05:49 AM
@AnkushKoul Since the 30 seconds since last execution has past, the processor is available to be immediately scheduled once a thread becomes available. So second thread would not wait till 60 seconds. This setting is minimum wait between executions. Other factors come int play that can affect component execution scheduling. NiFi hands out threads to processors from the Max Timer Driven Thread Count resource pool set via Controller Settings under the global menu in the upper right corner. Naturally you will have more components on your canvas then the size of this resource pool (which should be set initially to only 2-4 times the number fo cores you have on a single node since setting applies per node). NiFi will hand these available threads out to processors requesting CPU time to execute. Most component threads are in the range of milliseconds of execution, bit some can be more resource intensive and take longer to complete. Before increasing this resource pool, you should monitor the CPU impact/usage with all your dataflows running. Then make small increments if resource exist. Hope this answers your questions. If, so please take. moment to accept the answer(s) that helped. Matt
... View more