Reply
Highlighted
New Contributor
Posts: 7
Registered: ‎06-29-2015

Using Spark 1.5 to read data from SQL server and directly writing to Impala

Using Spark 1.5 to read data from SQL server and directly writing to Impala is it possiable

Cloudera Employee
Posts: 322
Registered: ‎01-16-2014

Re: Using Spark 1.5 to read data from SQL server and directly writing to Impala

We currently do not have Spark 1.5 as a released version and on top of that Spark SQL is not supported for any of the releases that we have done.

 

Could you explain a bit more what you are trying to do exactly then we might be able to help. Normally pulling data from a SQL database into a Hadoop cluster is something we use sqoop for and not Spark.

 

Wilfred

Cloudera Employee
Posts: 366
Registered: ‎07-29-2013

Re: Using Spark 1.5 to read data from SQL server and directly writing to Impala

I don't think anything about Spark helps you read from SQLServer, but
in principle you can use the JDBC-related support code to do so.
Writing to Impala means writing Parquet files, probably, which is
doable as well. It's not clear what you mean to do here, so not even
clear if Spark is the right tool.