Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

On CDH5 I cannot acces files with hftp

avatar
Contributor

[root@bdcxlocal1 hadoop-httpfs]# hadoop fs -ls hftp://bdcx1:50070/user/weather/defaultBlockSize
Found 1 items
-rw-r--r-- 3 weather hadoop 1102205 2014-05-15 08:59 hftp://bdcx1:50070/user/weather/defaultBlockSize
[root@bdcxlocal1 hadoop-httpfs]# hadoop fs -cat hftp://bdcx1:50070/user/weather/defaultBlockSize
cat: HTTP_OK expected, received 500

 

On my CDH4 Cluster this works. Where is the error?

1 ACCEPTED SOLUTION

avatar
Mentor
Given that the 'ls' works and 'cat' is the only problem, the error log
and its exception stack trace behind the 500 code would presently be
on one of the DataNodes on which the file
"/user/weather/defaultBlockSize"'s first block's replicas reside upon.
Check the identified DataNode's logs to find the error.

Do you by any chance use Cloudera Manager, and have wildcards enabled
on the HDFS service?

Also, since you are now long on a CDH version that has WebHDFS on it,
you can also alternatively try the more modern HTTP interface
alternative of webhdfs://, instead of hftp://:

hadoop fs -cat webhdfs://bdcx1:50070/user/weather/defaultBlockSize

Does that work?

View solution in original post

2 REPLIES 2

avatar
Mentor
Given that the 'ls' works and 'cat' is the only problem, the error log
and its exception stack trace behind the 500 code would presently be
on one of the DataNodes on which the file
"/user/weather/defaultBlockSize"'s first block's replicas reside upon.
Check the identified DataNode's logs to find the error.

Do you by any chance use Cloudera Manager, and have wildcards enabled
on the HDFS service?

Also, since you are now long on a CDH version that has WebHDFS on it,
you can also alternatively try the more modern HTTP interface
alternative of webhdfs://, instead of hftp://:

hadoop fs -cat webhdfs://bdcx1:50070/user/weather/defaultBlockSize

Does that work?

avatar
Contributor

We upgraded ou cluster to CDH 5.1.1 and the problem disappeared.So currently I cannot reproduce the problem. Thanks for the tip with webhdfs.