Created 11-19-2018 01:33 PM
We are struggling to get SparkR to work together with Hive LLAP in HDP 3.0. In the https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/integrating-hive/content/hive_hivewarehouse... documentation, there is only talk about python and Java. Nothing about R. Is there not support for R anymore? And if so, what other options do we have as we have a lot of SparkR code accessing Hive today.
The problem I get is that the SparkR code tries to access the HDFS files for the database and tables in Hive, as if LLAP connector is not there at all. So basically, we get the following
Caused by: org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table addresstype. java.security.AccessControlException: Permission denied: user=<username>, access=EXECUTE, inode="/apps/hive":hdfs:hdfs:drwx------
And that is correct. The user I’m running with don’t have, and shouldn’t have permissions to that folder. Is anybody using sparkr together with Hive LLAP in HDP3.0?
Created 01-09-2019 01:14 AM
@Berry Österlund , SparkR is indeed no longer supported to access Hive, R is not considered in the HWC connector.