Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to set default time zone/local time for SparkSql on Hue Sessions.

avatar
Explorer

Hello

I am struggling to set the time zone for the sessions on Spark Sql while submitting the query over the HUE.Maybe somebody already has this problem and know how to solve it - I would be grateful for the help.

 
So far I checked that:

on pyspak session running on linux the time zone is the one i want :

spark.conf.get("spark.sql.session.timeZone")
'Europe/Warsaw'

However then i submit the TIMESTAMP on HUE running query against SparkSql I got the UTC Time Stamp.

SELECT TIMESTAMP 'now';

 

 
  TIMESTAMP '2024-07-02 08:01:06.157849' My current time
 
 
12024-07-02T06:01:06Z
 the UTC TIME
 
I have also added the variable to the spark-defaults.conf file like spark.sql.session.timeZone=Europe/Warsaw
But after restarting Spark3 and Hue nothing has changed.
 
Thank you for your help in advance.

Regards,

Bart

 

4 REPLIES 4

avatar
Super Collaborator

Hello @Bartlomiej,

You can change the timezone for Hue using the steps below:

  1. Log in to Clouder Manager as an Administrator.
  2. Go to Clusters > Hue > Configuration > Time Zone and set a desired value.
    The default value is America/Los_Angeles. You can refer to https://timezonedb.com/time-zones for information on the timezones.
  3. Click Save Changes.
  4. Restart the Hue service.

Let me know if this helps.

Cheers!

avatar
Explorer

Hello 

Thank you for this hint. However I have already check that configuration and it was already  Europe/Warsaw so the one diesired. 

There must be  somewhere some other place to specify this i think.

avatar
Explorer

I think teh time zone has nothing to do with CURRENT_TIMESTAMP () finally. As teh actually need is to use the CURRENT_TIMESTAMP () or TIMESTAMP 'now' and have the local time as default time of that function in Spark SQL  while running the query over the HUE.

Now these function are giving the time 2 h late with means in UTC format and I would like to have by default 2h later or EUROPE/Berlin time for example

avatar
Master Collaborator

Issue looks like you are hitting CDPD-66940 jira and try to upgrade your CDS3 parcel to latest and it will resolve your issue.