Trouble creating databases in Hive using Spark 2.0

[#1]

Hi Team,

I am trying to create a database in Hive using spark sql in spark 2.0 version & encounter the below permission issue,

command - spark.sql("create database test") ************************************************************************************************************************

Caused by: org.apache.hadoop.hive.metastore.api.MetaException: java.security.AccessControlException: Permission denied: ************************************************************************************************************************

However when I try to do the same in Spark 1.6 version, I am able to do it without any issues. command - sqlContext.sql("create database test")

Do I need to set some configuration to execute from Spark 2.0?

Thanks in Advance!

[#2]

Hi @Rohit Ravishankar,

spark 1.6 and spark 2 have a different hive warehouse.

For spark 1.6 , warehouse location is identified by hive.metastore.warehouse.dir (default = /apps/hive/warehouse)

For spark 2, warehouse location is identified by spark.sql.warehouse.dir (default = <user.dir>/spark-warehouse)

I guess the user has right permissions to spark 1.6's warehouse location but not spark 2's location. Can you please check it and try giving proper permissions and run the query.

Thanks,

Aditya