Skip to main content
Bartlomiej
Gold ⭐️
July 2, 2024
Solved

How to set default time zone/local time for SparkSql on Hue Sessions.

Hello

I am struggling to set the time zone for the sessions on Spark Sql while submitting the query over the HUE.Maybe somebody already has this problem and know how to solve it - I would be grateful for the help.

 
So far I checked that:

on pyspak session running on linux the time zone is the one i want :

spark.conf.get("spark.sql.session.timeZone")
'Europe/Warsaw'

However then i submit the TIMESTAMP on HUE running query against SparkSql I got the UTC Time Stamp.

SELECT TIMESTAMP 'now';

 

 
  TIMESTAMP '2024-07-02 08:01:06.157849' My current time
 
 
12024-07-02T06:01:06Z
 the UTC TIME
 
I have also added the variable to the spark-defaults.conf file like spark.sql.session.timeZone=Europe/Warsaw
But after restarting Spark3 and Hue nothing has changed.
 
Thank you for your help in advance.

Regards,

Bart

 

Bartlomiej
Gold ⭐️
October 8, 2024

Yes , upgrading spark to newest SPARK version SPARK3-3.3.2.3.3.7190.5-2-1.p0.54391297 - fixed the issue

6 replies

tj2007
Gold ⭐️⭐️
Gold ⭐️⭐️
July 3, 2024

Hello @Bartlomiej,

You can change the timezone for Hue using the steps below:

  1. Log in to Clouder Manager as an Administrator.
  2. Go to Clusters > Hue > Configuration > Time Zone and set a desired value.
    The default value is America/Los_Angeles. You can refer to https://timezonedb.com/time-zones for information on the timezones.
  3. Click Save Changes.
  4. Restart the Hue service.

Let me know if this helps.

Cheers!

Translate
Bartlomiej
Gold ⭐️
July 3, 2024

Hello 

Thank you for this hint. However I have already check that configuration and it was already  Europe/Warsaw so the one diesired. 

There must be  somewhere some other place to specify this i think.

Translate
RangaReddy
Gold ⭐️
Gold ⭐️
July 10, 2024

Issue looks like you are hitting CDPD-66940 jira and try to upgrade your CDS3 parcel to latest and it will resolve your issue.

Translate
DianaTorres
Community Manager
Community Manager
July 16, 2024

@Bartlomiej Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

Thanks.

Translate
Regards, Diana Torres, Senior Community Moderator
Bartlomiej
BartlomiejAuthorAnswer
Gold ⭐️
October 8, 2024

Yes , upgrading spark to newest SPARK version SPARK3-3.3.2.3.3.7190.5-2-1.p0.54391297 - fixed the issue

Translate