Support Questions
Find answers, ask questions, and share your expertise

Unable to connect hive warehouse from PySpark..

Highlighted

Unable to connect hive warehouse from PySpark..

Explorer

I have enabled LLAP and HSI as instructed in HDP document and trying to connect hive warehouse from PySpark. PFB error..


>>> from pyspark_llap import HiveWarehouseSession

>>> hive=HiveWarehouseSession.session(spark).build()

>>> hive.showDatabases().show(100)

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

File "/tmp/spark-ac3ac8d8-5b24-4339-9ef6-e6eed46932cf/userFiles-29c83c49-13b6-4e28-b902-5a6cfbdf7ada/pyspark_hwc-1.0.0.3.0.0.0-1634.zip/pyspark_llap/sql/session.py", line 127, in showDatabases

File "/usr/local/lib/python3.7/site-packages/pyspark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py", line 1257, in __call__

File "/usr/local/lib/python3.7/site-packages/pyspark/sql/utils.py", line 63, in deco

return f(*a, **kw)

File "/usr/local/lib/python3.7/site-packages/pyspark/python/lib/py4j-0.10.7-src.zip/py4j/protocol.py", line 328, in get_return_value

py4j.protocol.Py4JJavaError: An error occurred while calling o36.showDatabases.

: java.lang.RuntimeException: shadehive.org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [anonymous] does not have [USE] privilege on [default]


Don't have an account?