with newer versions of spark, the sqlContext is not load by default, you have to specify it explicitly :   scala> val sqlContext = new org.apache.spark.sql.SQLContext(sc) warning: there was one deprecation warning; re-run with -deprecation for details sqlContext: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@6179af64 scala> import sqlContext.implicits._ import sqlContext.implicits._ scala> sqlContext.sql("describe mytable") res2: org.apache.spark.sql.DataFrame = [col_name: string, data_type: string ... 1 more field]  I'm working with spark 2.3.2 ... View more

I've placed more ram, added jar path missing and worked perfect. Thanks for your help ... View more

I'm interested in the telecom use case, too.... We're dealing with hourly ingests that result in a number of small files we'd like regularly compacted... ... View more

That was it. Thanks ! Used http://tweeterid.com/ to convert from username to user id. ... View more

@Ancil McBarnett check out the repo i've been maintianing that uses maven to pull the required jars. https://community.hortonworks.com/repos/33592/hive-jdbc-uber-jar.html ... View more