In the past I used to set 'fs.hdfs.impl', however after moving to 2.3 I can't set it (in the past it was in core-site).
was it moved? removed?
HDP 2.3 uses Apache Hadoop 2.7.1
The available configuration(in core-site.xml) for this version of Hadoop can be found here:
It looks like it may have been changed to: fs.AbstractFileSystem.hdfs.impl
thank you for the answer, however I am a bit confused.
fs.AbstractFileSystem.hdfs.impl expects a uri will it be the same format as fs.hdfs.impl?
also, on my local hdp 2.3 setup I can't find core-defualt.xml file - what is it's location? what happens if it is not there? can I place it in core-site?
If we look at the configuration for an older version of Hadoop:
|org.apache.hadoop.dfs.DistributedFileSystem||The FileSystem for hdfs: uris.|
If we look at the configuration of Hadoop 2.7.1
|fs.AbstractFileSystem.hdfs.impl||org.apache.hadoop.fs.Hdfs||The FileSystem for hdfs: uris.|
For the second part of your question:
core-default.xml just gives you the default implementation for core-site.xml
You can add the properties to core-site.xml to overwrite the default values