I am currently using HiveQL to move a folder from one storage account to my own hadoop cluster storage account in HDInsight. the format of the folder that I am trying to relocate is: "year=2016" the folder contains many sub-folders formatted as: "month=1","month=2", etc, and eventually each of these sub-folders contain ORC files. Based on my understanding, I am supposed to use table partitioning in Hive in order to be able to relocate these folders. However, Hive does not recognize the path that I am providing (i.e. ....../testdata/year=2016). The qurey is the following:
SET hive.exec.dynamic.partition = true;
SET hive.exec.dynamic.partition.mode = nonstrict;
SET hive.mapred.mode = nonstrict;
DROP TABLE bihdfsarchive_test.bihdfsarchive;
create database if not exists bihdfsarchive_test;
CREATE TABLE IF NOT EXISTS bihdfsarchive_test.bihdfsarchive
PARTITIONED BY (year string)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\n' LINES TERMINATED BY '\n'
STORED AS ORC LOCATION 'wasb:///COPY_FROM_testdata_STORAGE_ACCOUNT';
LOAD DATA INPATH 'wasb://email@example.com/' INTO TABLE bihdfsarchive_test.bihdfsarchive;
Is there anything that I need to add or remove to get the query up and running?