Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Indexing Issue in HDFS

avatar
Rising Star

Hello,
I was trying to take report from HDFS Report and get the below message

1.JPG

Upon checking /var/log/cloudera-scm-headlamp and see the following error 

 

021-04-15 18:09:44,934 ERROR com.cloudera.headlamp.HeadlampIndexManager: Index build failed for service hdfs
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: Unsupported layout version -64
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at com.cloudera.headlamp.AbstractIndexBuilder.buildIndex(AbstractIndexBuilder.java:80)
        at com.cloudera.headlamp.HeadlampIndex.buildIndex(HeadlampIndex.java:257)
        at com.cloudera.headlamp.HeadlampIndex.reindex(HeadlampIndex.java:325)
        at com.cloudera.headlamp.HeadlampIndexManager.reindexIndexes(HeadlampIndexManager.java:240)
        at com.cloudera.headlamp.HeadlampIndexManager.access$100(HeadlampIndexManager.java:57)
        at com.cloudera.headlamp.HeadlampIndexManager$1.run(HeadlampIndexManager.java:494)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: Unsupported layout version -64
        at com.cloudera.headlamp.AbstractIndexBuilder$1.run(AbstractIndexBuilder.java:74)
        at com.cloudera.cmf.cdhclient.CdhExecutor$RunnableWrapper.call(CdhExecutor.java:221)
        at com.cloudera.cmf.cdhclient.CdhExecutor$RunnableWrapper.call(CdhExecutor.java:211)
        at com.cloudera.cmf.cdhclient.CdhExecutor$CallableWrapper.doWork(CdhExecutor.java:236)
        at com.cloudera.cmf.cdhclient.CdhExecutor$1.call(CdhExecutor.java:125)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.io.IOException: Unsupported layout version -64
        at com.cloudera.headlamp.IndexBuilderCDH.buildIndexImpl(IndexBuilderCDH.java:77)
        at com.cloudera.headlamp.AbstractIndexBuilder$1.run(AbstractIndexBuilder.java:72)
        ... 8 more
Caused by: java.io.IOException: Unsupported layout version -64
        at org.apache.hadoop.hdfs.server.namenode.FSImageUtil.loadSummary(FSImageUtil.java:75)
        at org.apache.hadoop.hdfs.tools.offlineImageViewer.CdhClientPBImageViewer.go(CdhClientPBImageViewer.java:113)
        at com.cloudera.headlamp.IndexBuilderCDH.buildIndexImpl(IndexBuilderCDH.java:70)
        ... 9 more

 

 

CM / CDH - 6.3.3

Appreciate any help / guidance in fixing this issue .

 

Thanks 

Amn

1 ACCEPTED SOLUTION

avatar
Contributor

Yes, right. Please move all the contents and start RMAN service once again. 

View solution in original post

5 REPLIES 5

avatar
Rising Star

 

Any assistance on this please ?

avatar
Contributor

Hello Amn,

 

Thanks for using Cloudera Community. I believe this issue is happening due to index failure. Can you please follow the below steps?


1. Stop the RMAN service from CM.

2. Navigate to CM > Cloudera Management Service > Configuration and search for Reports Manager Working Directory


By default it would be /var/lib/cloudera-scm-headlamp


Once identified we can move contents of /var/lib/cloudera-scm-headlamp/HDFS-1/ns1/* to some other tmp directory. Post that start RMAN service. This should help us in rebuilding the index once again.


Please let me know if it helps.

avatar
Rising Star

Hi @amk 

 

Hi,

When I go to /var/lib/cloudera-scm-headlamp/hdfs/ I see the below listed :

[nm1@myserver hdfs]# ls -ltr
-rw-r----- 1 cloudera-scm cloudera-scm 710967007 Jan 29 23:35 fsimage
-rw-r----- 1 cloudera-scm cloudera-scm 25680 Jan 29 23:44 aggregates
drwxr-x--- 2 cloudera-scm cloudera-scm 8192 Jan 29 23:44 index
-rw-r----- 1 cloudera-scm cloudera-scm 468901714 May 3 14:26 fsimage.tmp

 

Are you suggesting that I move everything that is inside hdfs folder to tmp location that then start Rman server? Kindly confirm.

 

Thanks 

Amn

avatar
Contributor

Yes, right. Please move all the contents and start RMAN service once again. 

avatar
Explorer

You can find the error message 'Index build failed for service hdfs' in the RM log. This issue is caused by exceptional ' /var/lib/cloudera-scm-headlamp/' where the Reports Manager role is configured to run.

step 1: stop CMS Reports Manager role

 

step 2: RUN THIS ON THE HOST WHERE RM ROLE RUNS
sudo rm -rf /var/lib/cloudera-scm-headlamp/*

 

step 3: start CMS Reports Manager role (will rebuild the headlamp dir)

Done.
You can now use CM> HDFS> File Browser.