Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Can we change location of staging data dir in hive ?

avatar
Guru

Team:

Can we change location of temp staging file .hive-staging_hive_2016-04-13_08-1* which is getting create inside /apps/hive/warehouse whenever we are running some create table as select statement.

CREATE TABLE test_survey_answers_2016_01_04_oldrecords_temp AS SELECT AM.question_id AS question_id_old, AM.answer_id AS answer_id_old, AM.value AS value_old, AM.text AS text_old, AM.start_date AS start_date_old, AM.end_date as end_date_old, AM.load_date_time as load_date_time_old, AMD.question_id AS question_id_new, AMD.answer_id AS answer_id_new, AMD.value AS value_new, AMD.text AS text_new FROM test_survey_answers AM LEFT OUTER JOIN test_survey_answers_2016_01_04_daily_temp AMD ON (AM.question_id= AMD.question_id AND AM.answer_id=AMD.answer_id;

I tried to change hive.metastore.warehouse.dir to some other dir but problem is new database also getting create in that location.

So I want to have to diffrent dirs, one is for databases(i.e /apps/hive/warehouse) and another one is for temp staging data(like /tmp/ or something else).

Can someone please help on this issue.

1 ACCEPTED SOLUTION

avatar
Super Guru

Can you try to set below parameter on hive shell and run again?

set hive.exec.stagingdir=/tmp/

View solution in original post

3 REPLIES 3

avatar
Super Guru

Can you try to set below parameter on hive shell and run again?

set hive.exec.stagingdir=/tmp/

avatar
Super Guru

@Saurabh Kumar

Hi, did that property worked for you?

avatar
Expert Contributor

@Saurabh

Try doing :

set hive.exec.scratchdir=/new_dir