Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Write Dataframe to teradata

avatar
New Contributor

Hi, How to write a dataframe to existing table in teradata, I am using Spark-1.6.

1 ACCEPTED SOLUTION

avatar
Guru

@Anchika Agarwal

Assuming that reading and writing data from Teradata is like MySQL or Postgresql.... You will need to include the JDBC driver for Teradata on the spark classpath.

$ SPARK_CLASSPATH=teradata-jdbc.jar bin/spark-shell

Use the following code in Spark shell. Modify and pass all necessary parameters

scala> val jdbcUsername = "USER_NAME" 
scala> val jdbcPassword = "PASSWORD" 
scala> val jdbcHostname = "HOSTNAME" 
scala> val jdbcPort = port_num 
scala> val jdbcDatabase ="DATABASE" 
scala> val jdbcUrl = s"jdbc:teradata://${jdbcHostname}:${jdbcPort}/${jdbcDatabase}?user=${jdbcUsername}&password=${jdbcPassword}"
scala> val connectionProperties = new java.util.Properties()
scala> Class.forName("com.teradata.jdbc.Driver")
scala> import java.sql.DriverManager 
scala> val connection = DriverManager.getConnection(jdbcUrl, jdbcUsername, jdbcPassword) connection.isClosed()
scala> sqlContext.table("jdbcDF").withColumnRenamed("table", "table_number") .write .jdbc(jdbcUrl, "tablename", connectionProperties)

View solution in original post

1 REPLY 1

avatar
Guru

@Anchika Agarwal

Assuming that reading and writing data from Teradata is like MySQL or Postgresql.... You will need to include the JDBC driver for Teradata on the spark classpath.

$ SPARK_CLASSPATH=teradata-jdbc.jar bin/spark-shell

Use the following code in Spark shell. Modify and pass all necessary parameters

scala> val jdbcUsername = "USER_NAME" 
scala> val jdbcPassword = "PASSWORD" 
scala> val jdbcHostname = "HOSTNAME" 
scala> val jdbcPort = port_num 
scala> val jdbcDatabase ="DATABASE" 
scala> val jdbcUrl = s"jdbc:teradata://${jdbcHostname}:${jdbcPort}/${jdbcDatabase}?user=${jdbcUsername}&password=${jdbcPassword}"
scala> val connectionProperties = new java.util.Properties()
scala> Class.forName("com.teradata.jdbc.Driver")
scala> import java.sql.DriverManager 
scala> val connection = DriverManager.getConnection(jdbcUrl, jdbcUsername, jdbcPassword) connection.isClosed()
scala> sqlContext.table("jdbcDF").withColumnRenamed("table", "table_number") .write .jdbc(jdbcUrl, "tablename", connectionProperties)