Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error reading Hive table backed by Azure ADLS

Error reading Hive table backed by Azure ADLS

Expert Contributor

I have a new HDP 2.6.5 cluster running in Azure VM's. Hive is pointed to an existing metastore DB that already contains Hive tables that are backed with data in Azure ADLS storage. This metastore DB is used by a number of HDInsight clusters and the tables were created through them.

I have configured this HDP cluster to access ADLS using instructions in

Based on this I have set the following in core-site.xml

# Have the following set in core-site.xml

fs.adl.oauth2.access.token.provider.type=ClientCredential 
fs.adl.oauth2.client.id=<app id> 
fs.adl.oauth2.credential=<key> 
fs.adl.oauth2.refresh.url=https://login.microsoftonline.com/<...>/oauth2/token 
dfs.adls.home.hostname=mystore.azuredatalakestore.net 
dfs.adls.home.mountpoint=/clusters/mycluster/

I can successfully access HDFS

hadoop fs -ls adl://home/hive/warehouse/hivesampletable
Found 1 items
-rwxrwxrwx+  1 66DD8aa9-958c-4B2c-119b-233332425b3c df3e05f4-0300-45a0-afb9-210eaba7dd19    4955715 2018-06-29 21:42 adl://home/hive/warehouse/hivesampletable/HiveSampleData.txt

However if I try to access any of these tables (or databases) backed by ADLS, I get the following message.

Error: Error while compiling statement: FAILED: SemanticException java.lang.IllegalArgumentException: No value for dfs.adls.home.hostname found in conf file. (state=42000,code=40000)

I have verified that the dfs.adls.home.hostname property does exist in core-site.xml. Am I missing some obvious configuration to use ADLS?

Don't have an account?
Coming from Hortonworks? Activate your account here