Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to access hive database/tables through spark and pyspark shell?

Highlighted

how to access hive database/tables through spark and pyspark shell?

Explorer
 
2 REPLIES 2

Re: how to access hive database/tables through spark and pyspark shell?

Rising Star
Hi Harish,

You can create a hive context and can access the hive table.

Example Program:

from pyspark.sql import HiveContext
hive_context = HiveContext(sc)
sample = hive_context.table("default.<tablename>")
sample.show()

Reference Link: https://stackoverflow.com/questions/36051091/query-hive-table-in-pyspark

Re: how to access hive database/tables through spark and pyspark shell?

Explorer

from spark or pyspark shell use the below commands to access hive database objects.

 

spark.sql("show databases;")

spark.sql("select * from databasename.tablename;")

or

spark.read.table("databasename.tablename")

 

You can give any query inside spark.sql which will give you results.

Don't have an account?
Coming from Hortonworks? Activate your account here