Member since
04-26-2017
7
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2340 | 05-15-2017 11:41 PM |
05-15-2017
11:41 PM
A simple Spark app demonstrating how to read and write data in the Parquet and Avro
formats. I hope this can help. https://github.com/sryza/simplesparkavroapp
... View more
05-10-2017
03:31 AM
Using HDP 2.5 with Spark 2 If you defind the code as follows: val spark = SparkSession
.builder
.appName("my app")
.getOrCreate() import spark.implicits._ val test = spark.sqlContext.sql("select max (test_dt) as test_dt from abc").as[String] val test1 = spark.sqlContext.table("testing") The following two statements will compile val output2 = test1.filter(test1("audit_date").gt(test).toString()) val output2 = test1.filter(test1("audit_date").gt(test)) of course you can always convert test to String and use the variable in the filter clause.
... View more