Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Cloudera Employee

I have been playing quite a bit with CDSW lately. Here is a quick article on how to setup a CDSW project in scala connecting to an external RDBMS

Step 1: Create a new CDSW Project

Using the CDSW UI, create a new Scala Project:

103482-screen-shot-2019-02-13-at-41728-pm.png

Step 2: Reference the external Jar in your spark-defaults.conf

Open your project, and edit your spark-defaults.conf to add an external jar:

spark.jars=http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.6/mysql-connector-java-5.1.6.jar

103483-screen-shot-2019-02-13-at-41812-pm.png

Step 3: Create a simple Scala file to connect to the DB

Create a new file and add this code in it:

val sqlContext = new org.apache.spark.sql.SQLContext(sc)

val df = sqlContext.read.format("jdbc").option("url", "jdbc:mysql://[YOUR_SERVER_IP]:3306/[YOUR_DB]").option("driver", "com.mysql.jdbc.Driver").option("dbtable", "[YOUR_TABLE]").option("user", "[YOUR_USER]").option("password", "[YOUR_PWD]").load()

df.show()

Step 4: Run your application

Launch a session and run your code:

103484-screen-shot-2019-02-13-at-42817-pm.png

332 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 04:49 AM
Updated by:
 
Contributors
Top Kudoed Authors