@Onkar_Gagre  The Max Timer Driven Thread pool setting is applied to each node individually.   NiFi nodes configured as a cluster ate expected to be running on same hardware configuration.  The guidance of 2 to 4 times the number of cores as starting point is based on the cores of a single node in the cluster and not based off the cumulative cores across all NiFi cluster nodes.  You can only reduce wait time as you reduce load on the CPU.  In most cases, threads given out to most NiFi processors execute for only milliseconds.  But some processors operating against the content can take several seconds or much longer depending on function of the processor and/or size of the content. When the CPU is saturated these threads will take even longer to complete as the CPU is giving time to each active thread.  Knowing the only 8 threads at a time per node can actually execute concurrently, a thread only gets a short duration of time before giving some to another.  The pauses in between are the CPU wait time as thread queued up wait for their turns to execute.  So reducing the max Timer Driven Thread count (requires restart to reduction to be applied) would reduce maximum threads sent to CPU concurrently which would reduce CPU wait time.  Of course the means less concurrency in NiFi.  Sometimes you can reduce CPU through different flow designs, which is a much bigger discussion than can be handle efficiently via the community forums.  Other times, your dataflow simply needs more CPU to handle the volumes and rates you are looking to achieve.   CPU and Disk I/O are the biggest causes of slowed data processing. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@Onkar_Gagre    I would suggest you to tweak the following settings for better MirrorMaker scaling:   1) Kafka > Config > consumer.request.timeout.ms to 500000 Kafka > Config > session.timeout.ms to 60000 Kafka > Config > buffer.memory to 66000000 Kafka > Config > replica.fetch.max.bytes to 56700160 Kafka > Config > message.max.bytes to 56700160   2) You can increase the heartbeat.interval.ms to 20 secs in line with session.timeout.ms (1/3rd of session time). This has to be set in CM > Kafka > Config > Kafka MirrorMaker Advanced Configuration Snippet (Safety Valve) for mirror_maker_consumers.properties > heartbeat.interval.ms = 20000 > Save changes > Restart MM.   3) Add   partition.assignment.strategy=org.apache.kafka.clients.consumer.RoundRobinAssignor   to Kafka safety value for mirror.maker.consumer.properties   If you still face the same issue, then provide us the following details for further investigation:   1) Are you getting any errors in the broker logs and MM logs? OR we need broker and MM logs  2) Run in each broker on the destination cluster a couple of times every 5 hours below to discard networking issues: netstat -s | egrep "retransmited|sent out|socket|buffer|overflow|total packets received" 3) How many brokers are there in the Kafka cluster?  4) Was it working fine before and have you made any recent changes?    Thank you. ... View more

@Onkar_Gagre    A] There is no any standard tool that Kafka or Cloudera provides to migrate only Kafka zookeeper metadata. But yes, there are some third-party tools which you can use to migrate zookeeper metadata.    You can use a third-party tool like zkcopy to migrate/copy metadata from one cluster to another.  But we would recommend testing it in your dev lab first to avoid further issues    Note: Just something to be aware of.. If you decide on copying over the znodes uses this third party tool then you need to do this when the Kafka service is shutdown. You would then copy it to the new Znode root location, change the chroot config in Kafka and start that up again.   B] You can use SRM for migration and once the migration is completed you can lower the retention settings 1-2 hours as you don't need data so that all data will be cleared automatically and then you can set retetion back to 7 days which is default or as per your requirement.    C] You can also create all topics on a new cluster with the required configuration    If you found this response assisted with your query, please take a moment to log in and click on  KUDOS 🙂 & ”Accept as Solution" below this post.   Thank you. ... View more

Thank you Araujo. This has helped a lot.  ... View more