question Re: Dataframe/Dataset from Query in Archives of Support Questions (Read Only)

Dataframe/Dataset from Query

thangarajanp — Wed, 29 Mar 2017 13:21:35 GMT

Hi ,

I have one query which contain many join. Now I want to create a Dataframe or Dataset from the query (not from a single table) in scala

Re: Dataframe/Dataset from Query

yvora — Thu, 30 Mar 2017 03:12:27 GMT

@Thangarajan Pannerselvam,

1) Python:

You can actually use panda to get a data frame from a query result. Read below blog for details on how to run a query on a database and get data frame from a result.

https://www.dataquest.io/blog/python-pandas-databases/

2) Scala:

You can also use jdbc driver to run a query and save the result of query in data frame. ( use spark.read.jdbc api).

Details can be found in below link.

https://sparkour.urizone.net/recipes/using-jdbc/

Re: Dataframe/Dataset from Query

thangarajanp — Thu, 30 Mar 2017 11:58:53 GMT

Great Working Fine now