Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
avatar
Contributor

For HDFS analysis, an Activity Analyzer needs to be deployed to each NameNode in the cluster. These instances will automatically begin processing the fsimage on startup and will reprocess the latest fsimage data once every 24 hours. By default, when deployed on a NameNode, these Activity Analyzers do not process YARN, or MapReduce & Tez utilization data; This is to reduce the amount of processing done on servers hosting critical services like the NameNode.

Unless we install analyzer on namenode, it will not be able to find fsimage in directory and hence fail with below error

2018-08-23 14:28:04,401 ERROR [pool-3-thread-1] ActivityManager:78 - Failed to process activity id null of type HDFS
java.lang.NullPointerException
        at java.io.File.<init>(File.java:277)
        at com.hortonworks.smartsense.activity.hdfs.HDFSImageProcessor.processActivity(HDFSImageProcessor.java:71)
        at com.hortonworks.smartsense.activity.ActivityManager$1.call(ActivityManager.java:69)
        at com.hortonworks.smartsense.activity.ActivityManager$1.call(ActivityManager.java:62)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-08-23 14:28:04,403  INFO [Thread-3] ActivityManager:172 - Shutting down activity manager...
2018-08-23 14:28:04,425  WARN [Thread-6] HeapMemorySizeUtil:55 - hbase.regionserver.global.memstore.upperLimit is deprecated by hbase.regionserver.global.memstore.sizectivityManager:78 - Failed to process activity id null of type HDFSShutting down activity

Solution - Install activity analyzer on both the namenodes

678 Views
0 Kudos