Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

where can I find fs.hdfs.impl property

avatar
Explorer

In the past I used to set 'fs.hdfs.impl', however after moving to 2.3 I can't set it (in the past it was in core-site).

was it moved? removed?

Thanks

Alon

4 REPLIES 4

avatar
Rising Star

HDP 2.3 uses Apache Hadoop 2.7.1

The available configuration(in core-site.xml) for this version of Hadoop can be found here:

https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/core-default.xml

It looks like it may have been changed to: fs.AbstractFileSystem.hdfs.impl

avatar
Explorer

Hi Nasheb

thank you for the answer, however I am a bit confused.

fs.AbstractFileSystem.hdfs.impl expects a uri will it be the same format as fs.hdfs.impl?

also, on my local hdp 2.3 setup I can't find core-defualt.xml file - what is it's location? what happens if it is not there? can I place it in core-site?

avatar
Rising Star

Alon, what are you attempting to do? Is fs.hdfs.impl the correct property you are looking to change?

avatar
Rising Star

If we look at the configuration for an older version of Hadoop:

name|value|description

fs.hdfs.impl

org.apache.hadoop.dfs.DistributedFileSystemThe FileSystem for hdfs: uris.

If we look at the configuration of Hadoop 2.7.1

name|value|description

fs.AbstractFileSystem.hdfs.implorg.apache.hadoop.fs.HdfsThe FileSystem for hdfs: uris.

For the second part of your question:

core-default.xml just gives you the default implementation for core-site.xml

You can add the properties to core-site.xml to overwrite the default values