Can we change location of staging data dir in hive ?

SK1 — Fri, 16 Sep 2022 10:13:46 GMT

Team:

Can we change location of temp staging file .hive-staging_hive_2016-04-13_08-1* which is getting create inside /apps/hive/warehouse whenever we are running some create table as select statement.

CREATE TABLE test_survey_answers_2016_01_04_oldrecords_temp AS SELECT AM.question_id AS question_id_old, AM.answer_id AS answer_id_old, AM.value AS value_old, AM.text AS text_old, AM.start_date AS start_date_old, AM.end_date as end_date_old, AM.load_date_time as load_date_time_old, AMD.question_id AS question_id_new, AMD.answer_id AS answer_id_new, AMD.value AS value_new, AMD.text AS text_new FROM test_survey_answers AM LEFT OUTER JOIN test_survey_answers_2016_01_04_daily_temp AMD ON (AM.question_id= AMD.question_id AND AM.answer_id=AMD.answer_id;

I tried to change hive.metastore.warehouse.dir to some other dir but problem is new database also getting create in that location.

So I want to have to diffrent dirs, one is for databases(i.e /apps/hive/warehouse) and another one is for temp staging data(like /tmp/ or something else).

Can someone please help on this issue.

Re: Can we change location of staging data dir in hive ?

jyadav — Wed, 13 Apr 2016 20:01:14 GMT

Can you try to set below parameter on hive shell and run again?

set hive.exec.stagingdir=/tmp/

Re: Can we change location of staging data dir in hive ?

jyadav — Fri, 15 Apr 2016 18:42:51 GMT

@Saurabh Kumar

Hi, did that property worked for you?

Re: Can we change location of staging data dir in hive ?

PARTOMIA — Thu, 03 Nov 2016 11:48:51 GMT

@Saurabh

Try doing :

set hive.exec.scratchdir=/new_dir

question Can we change location of staging data dir in hive ? in Archives of Support Questions (Read Only)

Can we change location of staging data dir in hive ?

Re: Can we change location of staging data dir in hive ?

Re: Can we change location of staging data dir in hive ?

Re: Can we change location of staging data dir in hive ?