Created 07-21-2017 08:49 AM
An error occur when i try to load data from hdfs directory with multy level directories?
Should change some properties in the hive-site.xml?
Thanks
Created 07-21-2017 12:22 PM
is that the only properties I need to add into the hive-site.xml?
<property>
<name>mapred.input.dir.recursive</name>
<value>true</value>
</property> <property>
<name>hive.mapred.supports.subdirectories</name>
<value>true</value>
</property>
Created 07-21-2017 09:02 AM
Can you give some details?
Saying multi-level, do you try to import files from multiple folders? Can you give sample directory or directories?
Created 07-21-2017 11:48 AM
it is a directory with many folders
/log/20170721
/log/20170720
/log/20170719
Created 07-21-2017 12:22 PM
is that the only properties I need to add into the hive-site.xml?
<property>
<name>mapred.input.dir.recursive</name>
<value>true</value>
</property> <property>
<name>hive.mapred.supports.subdirectories</name>
<value>true</value>
</property>
Created 07-21-2017 06:14 PM
@Helmi Khalifa
Please use below snytax to load data from hdfs to hive tables
LOAD DATA INPATH '/hdfs/path' OVERWRITE INTO TABLE TABLE_NAME;
In case if you are trying to load to a specific partition of the table
LOAD DATA INPATH '/hdfs/path' OVERWRITE INTO TABLE TABLE_NAME PARTITION (ds='2008-08-15');
Created 07-22-2017 09:35 AM
I am using the same syntax as yours but it does'nt work.
there are some missing properties in the hive-site.xml file.
I added these properties in my comment below and it works now
mapred.input.dir.recursive
hive.mapred.supports.subdirectories
Thanks