Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to Load data from hdfs Multi level directory into Hive table?

Solved Go to solution

How to Load data from hdfs Multi level directory into Hive table?

Contributor

An error occur when i try to load data from hdfs directory with multy level directories?

Should change some properties in the hive-site.xml?

Thanks

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to Load data from hdfs Multi level directory into Hive table?

Contributor

is that the only properties I need to add into the hive-site.xml?

<property>

<name>mapred.input.dir.recursive</name>

<value>true</value>

</property> <property>

<name>hive.mapred.supports.subdirectories</name>

<value>true</value>

</property>

5 REPLIES 5

Re: How to Load data from hdfs Multi level directory into Hive table?

Rising Star

Can you give some details?

Saying multi-level, do you try to import files from multiple folders? Can you give sample directory or directories?

Re: How to Load data from hdfs Multi level directory into Hive table?

Contributor

it is a directory with many folders

/log/20170721

/log/20170720

/log/20170719

Re: How to Load data from hdfs Multi level directory into Hive table?

Contributor

is that the only properties I need to add into the hive-site.xml?

<property>

<name>mapred.input.dir.recursive</name>

<value>true</value>

</property> <property>

<name>hive.mapred.supports.subdirectories</name>

<value>true</value>

</property>

Highlighted

Re: How to Load data from hdfs Multi level directory into Hive table?

Expert Contributor

@Helmi Khalifa

Please use below snytax to load data from hdfs to hive tables

LOAD DATA INPATH '/hdfs/path' OVERWRITE INTO TABLE TABLE_NAME;

In case if you are trying to load to a specific partition of the table

LOAD DATA INPATH '/hdfs/path' OVERWRITE INTO TABLE TABLE_NAME PARTITION (ds='2008-08-15');

Re: How to Load data from hdfs Multi level directory into Hive table?

Contributor

I am using the same syntax as yours but it does'nt work.

there are some missing properties in the hive-site.xml file.

I added these properties in my comment below and it works now

mapred.input.dir.recursive

hive.mapred.supports.subdirectories

Thanks

Don't have an account?
Coming from Hortonworks? Activate your account here