Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
how to access hive database/tables through spark and pyspark shell?
Labels:
- Labels:
-
Apache Hive
-
Apache Spark
Contributor
Created on ‎04-26-2019 02:13 AM - edited ‎09-16-2022 07:20 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2 REPLIES 2
Expert Contributor
Created ‎05-28-2019 03:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Harish,
You can create a hive context and can access the hive table.
Example Program:
from pyspark.sql import HiveContext
hive_context = HiveContext(sc)
sample = hive_context.table("default.<tablename>")
sample.show()
Reference Link: https://stackoverflow.com/questions/36051091/query-hive-table-in-pyspark
You can create a hive context and can access the hive table.
Example Program:
from pyspark.sql import HiveContext
hive_context = HiveContext(sc)
sample = hive_context.table("default.<tablename>")
sample.show()
Reference Link: https://stackoverflow.com/questions/36051091/query-hive-table-in-pyspark
Explorer
Created ‎06-03-2019 01:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
from spark or pyspark shell use the below commands to access hive database objects.
spark.sql("show databases;")
spark.sql("select * from databasename.tablename;")
or
spark.read.table("databasename.tablename")
You can give any query inside spark.sql which will give you results.
