Can we still configure in cdh 6.1 hbase to work with adls ?

Question

We are trying to use hbase with adls as a storage layer. Is this still possible in cdh 6.1?

This is described in the docs for cdh 5.12 :

https://docs.cloudera.com/documentation/enterprise/5-12-x/topics/admin_using_adls_storage_with_hbase.html#hbase_adls_configuration_steps

but we are using cdh 6.1 and there I don't see a similar page anymore.

Off course I ask since I have an issue following the 5.12 docs.
When I want to change in Cloudera Manager in the hbase configuration,

hbase.root.dir = adl://ourproject.azuredatalakestore.net/hbase

I can not save this method as I get an error saying the variable is not in the allowed format:

HDFS Root Directory: Path adls://ourproject.azuredatalakestore.net/hbase does not conform to the pattern "(/[-+=_.a-zA-Z0-9]+)+(/)*".

JB0000012345 · Accepted Answer

Thank you for verifying!

PabitraDas · Answer

Hello @JB0000000000001 , Your assessment of ADLS support in CDH6.x is correct. The doc page is no more available in CDH 6.x because ADLS Gen1 or ADLS Gen2 storage can't be used as an HBase Root directory in CDH 6.x. This is a limitation in CDH 6.x

You can use ADLS with Hive, Impala, Oozie, Spark, YARN, MapReduce, Sqoop and DistCp but not with HBase [1].

Per the Cloudera 6.x document [2], ADLS is not supported as the default filesystem. So do not set the default file system property (fs.defaultFS) to an adl:// URI in HDFS or as a root directory (hbase.rootdir) in HBase (adl://<adls_account_name>.azuredatalakestore.net/<hbase_directory>) . You can still use ADLS as a secondary filesystem while HDFS remains the primary filesystem [2].

[1] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_adls_docs_ref.html

[2] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_adls2_config.html#abfs_gen2_limitations

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.