Member since
11-15-2019
14
Posts
0
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2786 | 03-04-2020 06:11 AM | |
2401 | 11-15-2019 05:35 AM |
03-04-2020
06:11 AM
Mattwho, Thanks for your comments. After reading your mail, I spent a lot of time thinking. I saw that 8 threads were created, and there was no performance improvement because all the threads were doing the same thing and executing the script on all files that were unpacked. Later your comment that the threads actually operate on the flowfiles, I changed the code so that it accepts the flow files as one of the input and processes using multiple threads. This improved the performance by 30%. The time taken dropped from 13 mins to 3-4 mins. So many thanks for your comments, I now understand how to use concurrent tasks.
... View more
01-14-2020
07:42 AM
@TVGanesh Great, it worked out for you. So if you think my answer helped resolve the issue then accept it to close the thread. Happ hadooping.
... View more