Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Write Dataframe to teradata

Solved Go to solution
Highlighted

Write Dataframe to teradata

New Contributor

Hi, How to write a dataframe to existing table in teradata, I am using Spark-1.6.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Write Dataframe to teradata

Guru

@Anchika Agarwal

Assuming that reading and writing data from Teradata is like MySQL or Postgresql.... You will need to include the JDBC driver for Teradata on the spark classpath.

$ SPARK_CLASSPATH=teradata-jdbc.jar bin/spark-shell

Use the following code in Spark shell. Modify and pass all necessary parameters

scala> val jdbcUsername = "USER_NAME" 
scala> val jdbcPassword = "PASSWORD" 
scala> val jdbcHostname = "HOSTNAME" 
scala> val jdbcPort = port_num 
scala> val jdbcDatabase ="DATABASE" 
scala> val jdbcUrl = s"jdbc:teradata://${jdbcHostname}:${jdbcPort}/${jdbcDatabase}?user=${jdbcUsername}&password=${jdbcPassword}"
scala> val connectionProperties = new java.util.Properties()
scala> Class.forName("com.teradata.jdbc.Driver")
scala> import java.sql.DriverManager 
scala> val connection = DriverManager.getConnection(jdbcUrl, jdbcUsername, jdbcPassword) connection.isClosed()
scala> sqlContext.table("jdbcDF").withColumnRenamed("table", "table_number") .write .jdbc(jdbcUrl, "tablename", connectionProperties)
1 REPLY 1

Re: Write Dataframe to teradata

Guru

@Anchika Agarwal

Assuming that reading and writing data from Teradata is like MySQL or Postgresql.... You will need to include the JDBC driver for Teradata on the spark classpath.

$ SPARK_CLASSPATH=teradata-jdbc.jar bin/spark-shell

Use the following code in Spark shell. Modify and pass all necessary parameters

scala> val jdbcUsername = "USER_NAME" 
scala> val jdbcPassword = "PASSWORD" 
scala> val jdbcHostname = "HOSTNAME" 
scala> val jdbcPort = port_num 
scala> val jdbcDatabase ="DATABASE" 
scala> val jdbcUrl = s"jdbc:teradata://${jdbcHostname}:${jdbcPort}/${jdbcDatabase}?user=${jdbcUsername}&password=${jdbcPassword}"
scala> val connectionProperties = new java.util.Properties()
scala> Class.forName("com.teradata.jdbc.Driver")
scala> import java.sql.DriverManager 
scala> val connection = DriverManager.getConnection(jdbcUrl, jdbcUsername, jdbcPassword) connection.isClosed()
scala> sqlContext.table("jdbcDF").withColumnRenamed("table", "table_number") .write .jdbc(jdbcUrl, "tablename", connectionProperties)
Don't have an account?
Coming from Hortonworks? Activate your account here