HDFS File Browser not working

[#1]

Hello,

We recently upgraded from CM 5.14.x to 5.16. I have noticed that post upgraded HDFS file browser options is not working, When I go to HDFS → File Browser option there is no change , nor there is any error, I have also tried it in IE and see the same behavior.

Any help / guidance in fixing this issue.

CM & CDH Ver -  5.16.2

Regards

Wert

[#2]

@wert_1311 ，

When you say "there is no change", did you mean that after you add a new file/directory, it is not reflected in the UI?

This might be indexing issue, have you checked the Reports Manager service log to see if there is any errors?

Cheers
Eric

[#3]

@EricL 

When I click on File Browser from HDFS Status page, ideally it should take me to File Browser page, but here it stays at HDFS Status page.

I have not come across any error in Reports Manager, have attached a snippet from RM logs.

Regards

Wert

Spoiler

2019-09-11 16:40:53,660 INFO org.apache.hadoop.hdfs.tools.offlineImageViewer.CdhClientPBImageViewer: The ratio of extra space for indexing is set with value 0.2
2019-09-11 16:40:53,660 INFO com.cloudera.headlamp.LuceneImageVisitor: Starting index build
2019-09-11 16:43:08,426 INFO com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
2019-09-11 16:43:09,426 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2019-09-11 16:43:09,597 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2019-09-11 16:43:09,631 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM.
2019-09-11 16:46:49,091 INFO com.cloudera.headlamp.LuceneImageVisitor: Wrote '/var/lib/cloudera-scm-headlamp/hdfs/aggregates.tmp' (25243 bytes)
2019-09-11 16:46:49,091 INFO com.cloudera.headlamp.LuceneImageVisitor: Starting publishing HBase metrics
2019-09-11 16:46:49,091 INFO com.cloudera.headlamp.LuceneImageVisitor: Finished publishing HBase metrics
2019-09-11 16:46:49,091 INFO com.cloudera.headlamp.HeadlampIndex: Temporarily shutting down search service
2019-09-11 16:46:49,091 INFO com.cloudera.headlamp.HeadlampServiceImpl: Saving existing quotas for later merge
2019-09-11 16:46:49,091 INFO com.cloudera.headlamp.HeadlampServiceImpl: Getting all files with quotas of type ALL
2019-09-11 16:46:49,092 INFO com.cloudera.headlamp.HeadlampServiceImpl: Performing query: +(DSQUOTA:[0 TO *} NSQUOTA:[0 TO *}) sortBy: null sortRerverse: false limit: 10000
2019-09-11 16:46:49,093 INFO com.cloudera.headlamp.HeadlampServiceImpl: Got 1 results for +(DSQUOTA:[0 TO *} NSQUOTA:[0 TO *})
2019-09-11 16:46:49,093 INFO com.cloudera.headlamp.HeadlampServiceImpl: Search for '+(DSQUOTA:[0 TO *} NSQUOTA:[0 TO *})' took 1ms
2019-09-11 16:46:49,094 INFO com.cloudera.headlamp.HeadlampIndex: Swapping index and FS image files
2019-09-11 16:46:49,095 INFO com.cloudera.headlamp.HeadlampIndex: Deleting old FS Image (/var/lib/cloudera-scm-headlamp/hdfs/fsimage)
2019-09-11 16:46:49,205 INFO com.cloudera.headlamp.HeadlampIndex: Moving new FS Image in place.
2019-09-11 16:46:49,205 INFO com.cloudera.headlamp.HeadlampIndex: Deleting old Index (/var/lib/cloudera-scm-headlamp/hdfs/index)
2019-09-11 16:46:50,018 INFO com.cloudera.headlamp.HeadlampIndex: Moving new Index in place.
2019-09-11 16:46:50,018 INFO com.cloudera.headlamp.HeadlampIndex: Deleting old Aggregate Summary (/var/lib/cloudera-scm-headlamp/hdfs/aggregates)
2019-09-11 16:46:50,018 INFO com.cloudera.headlamp.HeadlampIndex: Moving new Aggregate Summary in place.
2019-09-11 16:46:50,018 INFO com.cloudera.headlamp.HeadlampIndex: Starting up search service
2019-09-11 16:46:50,046 INFO com.cloudera.headlamp.HeadlampServiceImpl: Performing query: PATH:/ sortBy: null sortRerverse: false limit: 10000
2019-09-11 16:46:50,046 INFO com.cloudera.headlamp.HeadlampServiceImpl: Got 1 results for PATH:/
2019-09-11 16:46:50,046 INFO com.cloudera.headlamp.HeadlampServiceImpl: Search for 'PATH:/' took 0ms
2019-09-11 16:46:50,046 INFO com.cloudera.headlamp.HeadlampServiceImpl: Updating DSQUOTA in quota cache for '/' from -1 to -1
2019-09-11 16:46:53,165 INFO com.cloudera.headlamp.HeadlampServiceImpl: Performing query: PATH:/ sortBy: null sortRerverse: false limit: 10000
2019-09-11 16:46:53,167 INFO com.cloudera.headlamp.HeadlampServiceImpl: Got 1 results for PATH:/
2019-09-11 16:46:53,167 INFO com.cloudera.headlamp.HeadlampServiceImpl: Search for 'PATH:/' took 2ms
2019-09-11 16:46:53,167 INFO com.cloudera.headlamp.HeadlampServiceImpl: Updating NSQUOTA in quota cache for '/' from 9223372036854775807 to 9223372036854775807
2019-09-11 16:46:53,383 INFO com.cloudera.headlamp.HeadlampIndexManager: Finished indexing HDFS services

[#4]

You can find the error message 'Index build failed for service hdfs' in RM log. This issue is caused by exceptional ' /var/lib/cloudera-scm-headlamp/' where the Reports Manager role is configured to run.

step 1: stop CMS Reports Manager role

step 2: RUN THIS ON THE HOST WHERE RM ROLE RUNS
sudo rm -rf /var/lib/cloudera-scm-headlamp/*

step 3: start CMS Reports Manager role (will rebuild the headlamp dir)

Done.

You can now use CM> HDFS> File Browser.