Community Articles
Find and share helpful community-sourced technical articles
Explorer

For HDFS analysis, an Activity Analyzer needs to be deployed to each NameNode in the cluster. These instances will automatically begin processing the fsimage on startup and will reprocess the latest fsimage data once every 24 hours. By default, when deployed on a NameNode, these Activity Analyzers do not process YARN, or MapReduce & Tez utilization data; This is to reduce the amount of processing done on servers hosting critical services like the NameNode.

Unless we install analyzer on namenode, it will not be able to find fsimage in directory and hence fail with below error

2018-08-23 14:28:04,401 ERROR [pool-3-thread-1] ActivityManager:78 - Failed to process activity id null of type HDFS
java.lang.NullPointerException
        at java.io.File.<init>(File.java:277)
        at com.hortonworks.smartsense.activity.hdfs.HDFSImageProcessor.processActivity(HDFSImageProcessor.java:71)
        at com.hortonworks.smartsense.activity.ActivityManager$1.call(ActivityManager.java:69)
        at com.hortonworks.smartsense.activity.ActivityManager$1.call(ActivityManager.java:62)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2018-08-23 14:28:04,403  INFO [Thread-3] ActivityManager:172 - Shutting down activity manager...
2018-08-23 14:28:04,425  WARN [Thread-6] HeapMemorySizeUtil:55 - hbase.regionserver.global.memstore.upperLimit is deprecated by hbase.regionserver.global.memstore.sizectivityManager:78 - Failed to process activity id null of type HDFSShutting down activity

Solution - Install activity analyzer on both the namenodes

363 Views
0 Kudos
Don't have an account?
Version history
Revision #:
1 of 1
Last update:
‎08-23-2018 11:11 PM
Updated by:
 
Contributors
Top Kudoed Authors