Created 08-11-2023 04:09 AM
Hi Team,
Looking for inputs on defaulting HDFS storage to Ozone FS. As part of HDFS configuration from Cloudera Manager, in core-site.xml tried setting the following properties mentioned in :
https://ozone.apache.org/docs/1.0.0/interface/ofs.html
Post restarting the service, SecondaryNameNode fails with below exception :
Failed to start secondary namenode
java.io.IOException: This is not a DFS
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.getInfoServer(SecondaryNameNode.java:456)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:239)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:194)
at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:690)
However other role types i.e DataNode and NamedNode start up fine.
Listing the files using the command :
Still list the content from HDFS file system and not from Ozone storage. The same config settings work fine for Hive in defaulting to Ozone from HDFS.
Any suggestions on right way of configuring HDFS to Ozone storage is appreciated ? Also is this a supported config scenario ? Thanks.
Created 08-16-2023 09:33 AM
Hi @skommineni
We don't support setting Ozone as the default fs yet (Not fully integrated). This will be supported in future release. As a workaround you can try configuring HA (Active + Standby NN) instead of secondary NN and then set ozone as default fs.
Created 08-14-2023 06:11 AM
Welcome to the community @skommineni. Allow me to tag in @amk @rblough and @ChrisGe in case they have any ideas here.
Created 08-16-2023 09:33 AM
Hi @skommineni
We don't support setting Ozone as the default fs yet (Not fully integrated). This will be supported in future release. As a workaround you can try configuring HA (Active + Standby NN) instead of secondary NN and then set ozone as default fs.
Created 08-16-2023 08:57 PM
Got it..thanks @amk for confirming the same. Will check on the above recommendation.
Created 08-16-2023 10:18 PM
@skommineni, if the recommendation from @amk helps you resolve your issue, can you please mark the appropriate reply as the solution? This will make it easier for others to find the answer in the future.
Regards,
Vidya Sargur,