where can I find fs.hdfs.impl property

[#1]

In the past I used to set 'fs.hdfs.impl', however after moving to 2.3 I can't set it (in the past it was in core-site).

was it moved? removed?

Thanks

Alon

[#2]

HDP 2.3 uses Apache Hadoop 2.7.1

The available configuration(in core-site.xml) for this version of Hadoop can be found here:

https://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/core-default.xml

It looks like it may have been changed to: fs.AbstractFileSystem.hdfs.impl

[#3]

Hi Nasheb

thank you for the answer, however I am a bit confused.

fs.AbstractFileSystem.hdfs.impl expects a uri will it be the same format as fs.hdfs.impl?

also, on my local hdp 2.3 setup I can't find core-defualt.xml file - what is it's location? what happens if it is not there? can I place it in core-site?

[#4]

Alon, what are you attempting to do? Is fs.hdfs.impl the correct property you are looking to change?

[#5]

If we look at the configuration for an older version of Hadoop:

name|value|description

fs.hdfs.impl

org.apache.hadoop.dfs.DistributedFileSystem

The FileSystem for hdfs: uris.

If we look at the configuration of Hadoop 2.7.1

name|value|description

fs.AbstractFileSystem.hdfs.impl

org.apache.hadoop.fs.Hdfs

The FileSystem for hdfs: uris.

For the second part of your question:

core-default.xml just gives you the default implementation for core-site.xml

You can add the properties to core-site.xml to overwrite the default values