Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Guru

Goal

  • Access Hive through R shell with Rhive
  • Authentication mechanism: Kerberos

Assumptions

  • Have a HDP cluster up and running
  • Have configured and installed R
  • Cluster is configured on CentOS7
  • Need to have openssl-devel installed on the server where R is installed
  • If using openjdk, ensure that the JAVA_HOME is set to the path where javac is available

Steps

  • Download Rserve
wget https://rforge.net/Rserve/snapshot/Rserve_1.8-5.tar.gz
  • Use R to install Rserve
R CMD INSTALL Rserve_1.8-5.tar.gz
  • Download rhive library
git clone https://github.com/nexr/RHive.gita
  • Set the appropriate path for HIVE_HOME and HADOOP_HOME
export HADOOP_HOME=/usr/hdp/2.5.0.0-1245/hadoop; export HIVE_HOME=/usr/hdp/2.5.0.0-1245/hivea
  • Change directory to RHive
cd RHive
  • Use ant to build
ant build
  • Use R to build RHive, this will build a file within the same directory
R CMD build RHive
  • Use the newly created file name to install RHive library
R CMD INSTALL RHive_2.0-0.10.tar.gz

Demo

  • Launch R using "R" at the unix prompt
  • Load RHive library
> library(RHive)
Loading required package: rJava
Loading required package: Rserve
  • Use the connection string in this format
> rhive.connect(host="node1.hortonworks.com:10000/default;principal=hive/node1.hortonworks.com@HDP.COM;AuthMech=1;KrbHostFQDN=service.hortonworks.com;KrbServiceName=hive;KrbRealm=HDP.COM", defaultFS="hdfs://node1.hortonworks.com/rhive", hiveServer2=TRUE,updateJar=FALSE)

2016-11-09 22:30:00,425 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-11-09 22:30:01,361 WARN  [main] shortcircuit.DomainSocketFactory (DomainSocketFactory.java:<init>(117)) - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2016-11-09 22:30:01,853 INFO  [Thread-7] jdbc.Utils (Utils.java:parseURL(316)) - Supplied authorities: node1.hortonworks.com:10000
2016-11-09 22:30:01,853 INFO  [Thread-7] jdbc.Utils (Utils.java:parseURL(432)) - Resolved authority: node1.hortonworks.com:10000


  • Querying a table
> rhive.query("SELECT * FROM default.test")
  test.col1 test.col2
1         1         1
2         2         2
3         3         3
4         4         4
> 

Enjoy !!

1,445 Views
Comments
Expert Contributor

Where are steps to configure R-server for Kerberos? Does it not need any kerberos related settings? No kinit etc..?

I noticed that the connect string has kerberos related variables though. Is it really good enough? Please confirm.

rhive.connect(host="node1.hortonworks.com:10000/default;principal=hive/node1.hortonworks.com@HDP.COM;AuthMech=1;KrbHostFQDN=service.hortonworks.com;KrbServiceName=hive;KrbRealm=HDP.COM", defaultFS="hdfs://node1.hortonworks.com/rhive", hiveServer2=TRUE,updateJar=FALSE))
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎11-10-2016 02:35 AM
Updated by:
 
Contributors
Top Kudoed Authors