Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 07-21-2017 08:49 AM
An error occur when i try to load data from hdfs directory with multy level directories?
Should change some properties in the hive-site.xml?
Thanks
Created 07-21-2017 12:22 PM
is that the only properties I need to add into the hive-site.xml?
<property>
<name>mapred.input.dir.recursive</name>
<value>true</value>
</property> <property>
<name>hive.mapred.supports.subdirectories</name>
<value>true</value>
</property>
Created 07-21-2017 09:02 AM
Can you give some details?
Saying multi-level, do you try to import files from multiple folders? Can you give sample directory or directories?
Created 07-21-2017 11:48 AM
it is a directory with many folders
/log/20170721
/log/20170720
/log/20170719
Created 07-21-2017 12:22 PM
is that the only properties I need to add into the hive-site.xml?
<property>
<name>mapred.input.dir.recursive</name>
<value>true</value>
</property> <property>
<name>hive.mapred.supports.subdirectories</name>
<value>true</value>
</property>
Created 07-21-2017 06:14 PM
@Helmi Khalifa
Please use below snytax to load data from hdfs to hive tables
LOAD DATA INPATH '/hdfs/path' OVERWRITE INTO TABLE TABLE_NAME;
In case if you are trying to load to a specific partition of the table
LOAD DATA INPATH '/hdfs/path' OVERWRITE INTO TABLE TABLE_NAME PARTITION (ds='2008-08-15');
Created 07-22-2017 09:35 AM
I am using the same syntax as yours but it does'nt work.
there are some missing properties in the hive-site.xml file.
I added these properties in my comment below and it works now
mapred.input.dir.recursive
hive.mapred.supports.subdirectories
Thanks