Support Questions

Find answers, ask questions, and share your expertise

Can we still configure in cdh 6.1 hbase to work with adls ?

avatar
Contributor

We are trying to use hbase with adls as a storage layer. Is this still possible in cdh 6.1?

This is described in the docs for cdh 5.12 :

https://docs.cloudera.com/documentation/enterprise/5-12-x/topics/admin_using_adls_storage_with_hbase...

but we are using cdh 6.1 and there I don't see a similar page anymore. 

 

Off course I ask since I have an issue following the 5.12 docs.
When I want to change in Cloudera Manager in the hbase configuration, 

 

hbase.root.dir = adl://ourproject.azuredatalakestore.net/hbase

 

I can not save this method as I get an error saying the variable is not in the allowed format:

 

HDFS Root Directory: Path adls://ourproject.azuredatalakestore.net/hbase does not conform to the pattern "(/[-+=_.a-zA-Z0-9]+)+(/)*". 

 

 

1 ACCEPTED SOLUTION

avatar

Thank you for verifying!

View solution in original post

2 REPLIES 2

avatar
Expert Contributor

Hello @JB0000000000001 , Your assessment of ADLS support in CDH6.x is correct. The doc page is no more available in CDH 6.x  because ADLS Gen1 or ADLS Gen2 storage can't be used as an HBase Root directory in CDH 6.x. This is a limitation in CDH 6.x

 

You can use ADLS with Hive, Impala, Oozie, Spark, YARN, MapReduce, Sqoop and DistCp but not with HBase [1].

 

Per the Cloudera 6.x document [2], ADLS is not supported as the default filesystem. So do not set the default file system property (fs.defaultFS) to an adl:// URI in HDFS or as a root directory (hbase.rootdir) in HBase (adl://<adls_account_name>.azuredatalakestore.net/<hbase_directory>) . You can still use ADLS as a secondary filesystem while HDFS remains the primary filesystem [2]. 

 

[1] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_adls_docs_ref.html 

[2] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_adls2_config.html#abfs_gen2_li... 

avatar

Thank you for verifying!