Created on 08-23-201811:11 PM - edited 09-16-202201:44 AM
For HDFS analysis, an Activity Analyzer needs to be deployed to each NameNode in the cluster. These instances will automatically begin processing the fsimage on startup and will reprocess the latest fsimage data once every 24 hours. By default, when deployed on a NameNode, these Activity Analyzers do not process YARN, or MapReduce & Tez utilization data; This is to reduce the amount of processing done on servers hosting critical services like the NameNode.
Unless we install analyzer on namenode, it will not be able to find fsimage in directory and hence fail with below error
2018-08-23 14:28:04,401 ERROR [pool-3-thread-1] ActivityManager:78 - Failed to process activity id null of type HDFS
java.lang.NullPointerException
at java.io.File.<init>(File.java:277)
at com.hortonworks.smartsense.activity.hdfs.HDFSImageProcessor.processActivity(HDFSImageProcessor.java:71)
at com.hortonworks.smartsense.activity.ActivityManager$1.call(ActivityManager.java:69)
at com.hortonworks.smartsense.activity.ActivityManager$1.call(ActivityManager.java:62)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-08-23 14:28:04,403 INFO [Thread-3] ActivityManager:172 - Shutting down activity manager...
2018-08-23 14:28:04,425 WARN [Thread-6] HeapMemorySizeUtil:55 - hbase.regionserver.global.memstore.upperLimit is deprecated by hbase.regionserver.global.memstore.sizectivityManager:78 - Failed to process activity id null of type HDFSShutting down activity
Solution - Install activity analyzer on both the namenodes