Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to define date format in spark data frames?

how to define date format in spark data frames?

New Contributor

Explanation:

Mycode:

#############

import org.joda.time.{DateTimeZone}

import org.joda.time.format.DateTimeFormat

case class datecheck(Date:String,ID:Int)

val daterdd =sc.textFile("/FileStore/tables/m67ki3931500986208915/Datetest.csv").map(_.split(",")).map(x=>datecheck(x(0),x(1).toInt)).toDF()

daterdd.registerTempTable("datecheck")

val datedf = sqlContext.sql("select ID,(from_unixtime(Date, 'dd-MM-yyyy')) as dateformat from datecheck order by ID desc") datedf.show()

#################

Problem statement:

After performing date format operation in spark sql am getting output dataframe date column as null how to get output as date

please kindly solve this problem

Thanks in advance

swathi.T


sparkdate.jpg
Don't have an account?
Coming from Hortonworks? Activate your account here