Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Create Hive External Table With Location Pointing To Local Storage

avatar
Rising Star

Hello,

Earlier we used to point the Hive's external table's location to S3.

We now have a requirement to point it to a local filesystem like /tmp etc but not HDFS.

Can this be achieved in Hive?

1 ACCEPTED SOLUTION

avatar

@Rinku Singh

Unfortunately, it is not possible. The definition of External table itself explains the location for the file:

"An EXTERNAL table points to any HDFS location for its storage, rather than being stored in a folder specified by the configuration property hive.metastore.warehouse.dir."

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-ExternalTables

View solution in original post

3 REPLIES 3

avatar

@Rinku Singh

Unfortunately, it is not possible. The definition of External table itself explains the location for the file:

"An EXTERNAL table points to any HDFS location for its storage, rather than being stored in a folder specified by the configuration property hive.metastore.warehouse.dir."

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-ExternalTables

avatar

Hi @Sindhu,

Can you help me understand can I have my external table created in hive on top of the file location marked as one in the Google storage cloud (GS). I already have one created. I am able to add partitions in hive, which successfully creates a directory in Hive, however on adding file to the partitioned columns (directories in google storage), however when I try to update the meta-store with the : MSCK REPAIR TABLE <table_name>

However this runs unsuccessfully, as -

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

,

@Sindhu, can you help me understand if the location of my external table can be Google Cloud storage or is it always going to be HDFS. I have my external table created on Hive (on top of HDFS) with location as that of the Google drive, however MSCK REPAIR TABLE is not working even though that google storage location is manually updated, but not being successfully loaded into Hive.

avatar

Please reply asap. Thanks for your time.