Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Casting date string to datetime in Pyspark returning sometimes null

Highlighted

Casting date string to datetime in Pyspark returning sometimes null

New Contributor

I'm using Pyspark in order to load some data, then convert the string date column to datetime. But, I'im getting some strange results. Here's the file...(txt) But, when I'm trying to convert the dates to datetime, I get a good result for just one of them.. My code is here...

93736-capture-decran-2018-11-27-a-164324.png

93735-capture-decran-2018-11-27-a-164050.png

93737-capture-decran-2018-11-27-a-164505.png

I cannot understand the problem.....Thank you guys.....

2 REPLIES 2
Highlighted

Re: Casting date string to datetime in Pyspark returning sometimes null

@joo Antoine,

I guess the date format should be "dd/MM/yy HH:mm" instead of "d/MM/yy HH:mm". Looks like a 'd' is missing

Highlighted

Re: Casting date string to datetime in Pyspark returning sometimes null

New Contributor

Thanks for the reply....

I tried with two d's but the result is the same.....;; very strange

Don't have an account?
Coming from Hortonworks? Activate your account here