Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Cloudera Employee

Affected versions:

All SmartSense versions

Symptoms:

YARN and MR dashboards work fine but there is no data for HDFS dashboard. Activity Analyzer logs are available at /var/log/smartsense-activity/activity-analyzer.log. These logs will shows an error stack trace similar to something as provided below:

ERROR [pool-14-thread-1] ActivityManager:89 - Failed to process
activity id /hadoop/hdfs/namenode/current/fsimage_0009281 of type HDFScom.hortonworks.smartsense.activity.ActivityException: Failed to process activity. Activity Type: HDFS; ActivityId:
/hadoop/hdfs/namenode/current/fsimage_0009281; ActivityData: null  
at com.hortonworks.smartsense.activity.hdfs.HDFSImageProcessor.processActivity(HDFSImageProcessor.java:104)  
at com.hortonworks.smartsense.activity.ActivityManager$1.call(ActivityManager.java:80)  
at com.hortonworks.smartsense.activity.ActivityManager$1.call(ActivityManager.java:73)  
at java.util.concurrent.FutureTask.run(FutureTask.java:262)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:745)Caused by: com.hortonworks.smartsense.activity.ActivityException: Error while getting fsImage iterator from path /hadoop/hdfs/namenode/current/fsimage_0009281  
at org.apache.hadoop.hdfs.tools.offlineImageViewer.PBImageParser.getIterator(PBImageParser.java:118) 
at com.hortonworks.smartsense.activity.hdfs.HDFSImageProcessor.processActivity(HDFSImageProcessor.java:88)  ... 6 more

Caused by: java.io.IOException: Unsupported layout version -63  at org.apache.hadoop.hdfs.server.namenode.FSImageUtil.loadSummary(FSImageUtil.java:75)

Reason:

Activity Analyzer maintains all the dashboard in a new schema created in the same database as setup by Ambari Metrics Collector. It also utilizes the clients for HDFS available in the AMS libraries. This issue can happen if Activity Analyzer is used with older Hadoop/MapReduce/YARN libraries from AMS.

Solution:

Verify all the jars available under /usr/lib/ambari-metrics-collector and remove the older versions. Restart the Activity Analyzer after removing the older jars.

Examples of older version jars:

/usr/lib/ambari-metrics-collector/hadoop-annotations-2.6.0.2.2.0.0-2041.jar
/usr/lib/ambari-metrics-collector/hadoop-auth-2.6.0.2.2.0.0-2041.jar 
/usr/lib/ambari-metrics-collector/hadoop-client-2.6.0.2.2.0.0-2041.jar 
/usr/lib/ambari-metrics-collector/hadoop-common-2.6.0.2.2.0.0-2041.jar 
/usr/lib/ambari-metrics-collector/hadoop-hdfs-2.6.0.2.2.0.0-2041.jar 
/usr/lib/ambari-metrics-collector/hadoop-mapreduce-client-app-2.6.0.2.2.0.0-2041.jar
/usr/lib/ambari-metrics-collector/hadoop-mapreduce-client-common-2.6.0.2.2.0.0-2041.jar
/usr/lib/ambari-metrics-collector/hadoop-mapreduce-client-core-2.6.0.2.2.1.0-2340.jar
/usr/lib/ambari-metrics-collector/hadoop-mapreduce-client-jobclient-2.6.0.2.2.0.0-2041.jar
/usr/lib/ambari-metrics-collector/hadoop-mapreduce-client-shuffle-2.6.0.2.2.0.0-2041.jar
/usr/lib/ambari-metrics-collector/hadoop-yarn-api-2.6.0.2.2.0.0-2041.jar
/usr/lib/ambari-metrics-collector/hadoop-yarn-client-2.6.0.2.2.0.0-2041.jar
/usr/lib/ambari-metrics-collector/hadoop-yarn-common-2.6.0.2.2.0.0-2041.jar
/usr/lib/ambari-metrics-collector/hadoop-yarn-registry-2.6.0.2.2.0.0-2041.jar
1,001 Views
0 Kudos
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎04-27-2017 11:42 PM
Updated by:
 
Contributors
Top Kudoed Authors