Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can we change location of staging data dir in hive ?

Solved Go to solution
Highlighted

Can we change location of staging data dir in hive ?

Guru

Team:

Can we change location of temp staging file .hive-staging_hive_2016-04-13_08-1* which is getting create inside /apps/hive/warehouse whenever we are running some create table as select statement.

CREATE TABLE test_survey_answers_2016_01_04_oldrecords_temp AS SELECT AM.question_id AS question_id_old, AM.answer_id AS answer_id_old, AM.value AS value_old, AM.text AS text_old, AM.start_date AS start_date_old, AM.end_date as end_date_old, AM.load_date_time as load_date_time_old, AMD.question_id AS question_id_new, AMD.answer_id AS answer_id_new, AMD.value AS value_new, AMD.text AS text_new FROM test_survey_answers AM LEFT OUTER JOIN test_survey_answers_2016_01_04_daily_temp AMD ON (AM.question_id= AMD.question_id AND AM.answer_id=AMD.answer_id;

I tried to change hive.metastore.warehouse.dir to some other dir but problem is new database also getting create in that location.

So I want to have to diffrent dirs, one is for databases(i.e /apps/hive/warehouse) and another one is for temp staging data(like /tmp/ or something else).

Can someone please help on this issue.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Can we change location of staging data dir in hive ?

Can you try to set below parameter on hive shell and run again?

set hive.exec.stagingdir=/tmp/

3 REPLIES 3

Re: Can we change location of staging data dir in hive ?

Can you try to set below parameter on hive shell and run again?

set hive.exec.stagingdir=/tmp/

Re: Can we change location of staging data dir in hive ?

@Saurabh Kumar

Hi, did that property worked for you?

Re: Can we change location of staging data dir in hive ?

Rising Star

@Saurabh

Try doing :

set hive.exec.scratchdir=/new_dir
Don't have an account?
Coming from Hortonworks? Activate your account here