Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

how to access hive database/tables through spark and pyspark shell?

Explorer
 
2 REPLIES 2

Rising Star
Hi Harish,

You can create a hive context and can access the hive table.

Example Program:

from pyspark.sql import HiveContext
hive_context = HiveContext(sc)
sample = hive_context.table("default.<tablename>")
sample.show()

Reference Link: https://stackoverflow.com/questions/36051091/query-hive-table-in-pyspark

Explorer

from spark or pyspark shell use the below commands to access hive database objects.

 

spark.sql("show databases;")

spark.sql("select * from databasename.tablename;")

or

spark.read.table("databasename.tablename")

 

You can give any query inside spark.sql which will give you results.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.