Load several files into HIVE table

AndresUrrego — Sat, 23 Sep 2017 08:02:22 GMT

Look I'm trying to analyze too many files into just one HIVE table. Key insights, I'm working with json files and the tables structure is :

CREATE EXTERNAL TABLE test1 (

STATIONS ARRAY<STRING>,

SCHEMESUSPENDED STRING,

TIMELOAD TIMESTAMP )

ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'

LOCATION '/user/andres/hive/bixihistorical/';

I need to load around 50 files with the same structure all of them. I have tried things like:

LOAD DATA INPATH '/user/andres/datasets/bixi2017/*.json' OVERWRITE INTO TABLE test1;

LOAD DATA INPATH '/user/andres/datasets/bixi2017/*' OVERWRITE INTO TABLE test1;

LOAD DATA INPATH '/user/andres/datasets/bixi2017/' OVERWRITE INTO TABLE test1;

Any of those above have worked, any idea guys about how should I go thru? thanks so much

Re: Load several files into HIVE table

AndresUrrego — Sat, 23 Sep 2017 10:05:09 GMT

Hi Guys, I'm so so .... Well, I just remember that you can create just an external table stored in the same folder all files with the same structure are located. So , in that way I will load whole records in one shoot.

> CREATE EXTERNAL TABLE bixi_his

> ( > STATIONS ARRAY<STRUCT<id: INT,s:STRING,n:string,st:string,b:string,su:string,m:string,lu:string,lc:string,bk:string,bl:string,la:float,lo:float,da:int,dx:int,ba:int,bx:int>>, > SCHEMESUSPENDED STRING,

> TIMELOAD BIGINT > )

> ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'

> LOCATION '/user/ingenieroandresangel/datasets/bixi2017/';

thanks

question Re: Load several files into HIVE table in Archives of Support Questions (Read Only)

Load several files into HIVE table

Re: Load several files into HIVE table