Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Who agreed with this topic

Kudu to HDFS data load timestamp issue.

Labels:
avatar
author-rank GopiG
New Contributor

Created on ‎08-01-2019 01:33 PM - last edited on ‎08-02-2019 09:43 AM by Community Manager Community Manager

Hello,

I am trying to load tables from Kudu to HDFS using spark2 and i have noticed that timestamp is off by 8 hours between Kudu and HDFS. 

 

df=spark_session.read.format('org.apache.kudu.spark.kudu')
.option('kudu.master','dcaldd163:7051,dcaldd162:7051,dcaldd161:7051')
.option('kudu.table',"impala::DB.kudu_table_name").load()

 

df.write.format("parquet").mode('overwrite').saveAsTable("db_name.kudu_table_name")

 

I have tried to set the timezone locally for the session in Spark2 and still it does not solve the issue. 

 

Can someone give a hint on how to solve this issue? 

5,772
0 Kudos
1
Who agreed with this topic