Created on 09-25-2020 04:07 AM - edited 09-16-2022 07:38 AM
We are trying to use hbase with adls as a storage layer. Is this still possible in cdh 6.1?
This is described in the docs for cdh 5.12 :
but we are using cdh 6.1 and there I don't see a similar page anymore.
Off course I ask since I have an issue following the 5.12 docs.
When I want to change in Cloudera Manager in the hbase configuration,
hbase.root.dir = adl://ourproject.azuredatalakestore.net/hbase
I can not save this method as I get an error saying the variable is not in the allowed format:
HDFS Root Directory: Path adls://ourproject.azuredatalakestore.net/hbase does not conform to the pattern "(/[-+=_.a-zA-Z0-9]+)+(/)*".
Created 09-30-2020 09:02 AM
Thank you for verifying!
Created 09-29-2020 01:54 PM
Hello @JB0000000000001 , Your assessment of ADLS support in CDH6.x is correct. The doc page is no more available in CDH 6.x because ADLS Gen1 or ADLS Gen2 storage can't be used as an HBase Root directory in CDH 6.x. This is a limitation in CDH 6.x
You can use ADLS with Hive, Impala, Oozie, Spark, YARN, MapReduce, Sqoop and DistCp but not with HBase [1].
Per the Cloudera 6.x document [2], ADLS is not supported as the default filesystem. So do not set the default file system property (fs.defaultFS) to an adl:// URI in HDFS or as a root directory (hbase.rootdir) in HBase (adl://<adls_account_name>.azuredatalakestore.net/<hbase_directory>) . You can still use ADLS as a secondary filesystem while HDFS remains the primary filesystem [2].
[1] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_adls_docs_ref.html
Created 09-30-2020 09:02 AM
Thank you for verifying!