question Re: How to Load data from hdfs Multi level directory into Hive table? in Archives of Support Questions (Read Only)

How to Load data from hdfs Multi level directory into Hive table?

helmi_khalifa — Fri, 21 Jul 2017 15:49:51 GMT

An error occur when i try to load data from hdfs directory with multy level directories?

Should change some properties in the hive-site.xml?

Thanks

sedatkestepe — Fri, 21 Jul 2017 16:02:10 GMT

Can you give some details?

Saying multi-level, do you try to import files from multiple folders? Can you give sample directory or directories?

helmi_khalifa — Fri, 21 Jul 2017 18:48:04 GMT

it is a directory with many folders

/log/20170721

/log/20170720

/log/20170719

helmi_khalifa — Fri, 21 Jul 2017 19:22:41 GMT

is that the only properties I need to add into the hive-site.xml?

<name>mapred.input.dir.recursive</name>

</property> <property>

<name>hive.mapred.supports.subdirectories</name>

</property>

sreeviswa_athic — Sat, 22 Jul 2017 01:14:57 GMT

@Helmi Khalifa

Please use below snytax to load data from hdfs to hive tables

LOAD DATA INPATH '/hdfs/path' OVERWRITE INTO TABLE TABLE_NAME;

In case if you are trying to load to a specific partition of the table

LOAD DATA INPATH '/hdfs/path' OVERWRITE INTO TABLE TABLE_NAME PARTITION (ds='2008-08-15');

helmi_khalifa — Sat, 22 Jul 2017 16:35:19 GMT

I am using the same syntax as yours but it does'nt work.

there are some missing properties in the hive-site.xml file.

I added these properties in my comment below and it works now

mapred.input.dir.recursive

hive.mapred.supports.subdirectories

Thanks