I am trying to load records into MS SQL SERVER through Spark 2 using Spark SQL and JDBC connectivity. The program compiled successfully. Also ran without any errors. But records are not inserted into SQL Server. Below I mentioned the code
Spark Code Block:
LeadsDF.write
.format("jdbc")
.option("url", "jdbc:sqlserver://localhost:1433;databaseName=LeadTester")
.option("dbtable", TestTable)
.option("user", "XXX")
.option("password", "XXXXX") .mode(SaveMode.Append)
SBT Dependencies:
name := "EDWETL"
version := "1.0"
scalaVersion := "2.11.8"
libraryDependencies += "com.microsoft.sqlserver" % "mssql-jdbc" % "6.1.0.jre8" %"provided"
libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.0.2"
libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.0.0"
libraryDependencies += "com.thetransactioncompany" % "java-property-utils" % "1.8"
mainClass in (Compile,run) := Some("ssss.HBaseToSQLLoader.Main")
Spark-Submit:
spark-submit \
--class ssss.HBaseToSQLLoader \
--verbose \
--master local \
--driver-class-path mssql-jdbc-6.2.1.jre8.jar \
--num-executors 2 \
--driver-memory 512m \
--executor-memory 512m \
--executor-cores 1 \
/home/maria_dev/Mahe/edwetl_Main.jar