Setting HDFS Default FS to Ozone FS

Question

Hi Team,

Looking for inputs on defaulting HDFS storage to Ozone FS. As part of HDFS configuration from Cloudera Manager, in core-site.xml tried setting the following properties mentioned in :

https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/ozone-storing-data/topics/ozone-setting-up-ozonefs.html

https://ozone.apache.org/docs/1.0.0/interface/ofs.html

Post restarting the service, SecondaryNameNode fails with below exception :

Failed to start secondary namenode
java.io.IOException: This is not a DFS
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.getInfoServer(SecondaryNameNode.java:456)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:239)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:194)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:690)

However other role types i.e DataNode and NamedNode start up fine.

Listing the files using the command :

sudo -u hdfs hdfs dfs -ls /

Still list the content from HDFS file system and not from Ozone storage. The same config settings work fine for Hive in defaulting to Ozone from HDFS.

Any suggestions on right way of configuring HDFS to Ozone storage is appreciated ? Also is this a supported config scenario ? Thanks.

amk · Accepted Answer

Hi  @skommineniWe don't support setting Ozone as the default fs yet (Not fully integrated). This will be supported in future release. As a workaround you can try configuring HA (Active + Standby NN) instead of secondary NN and then set ozone as default fs.

cjervis · Answer

Welcome to the community @skommineni. Allow me to tag in @amk @rblough and @ChrisGe in case they have any ideas here.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.