Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Load several files into HIVE table

Solved Go to solution

Load several files into HIVE table

Rising Star

Look I'm trying to analyze too many files into just one HIVE table. Key insights, I'm working with json files and the tables structure is :

CREATE EXTERNAL TABLE test1 (

STATIONS ARRAY<STRING>,

SCHEMESUSPENDED STRING,

TIMELOAD TIMESTAMP )

ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'

LOCATION '/user/andres/hive/bixihistorical/';

I need to load around 50 files with the same structure all of them. I have tried things like:

LOAD DATA INPATH '/user/andres/datasets/bixi2017/*.json' OVERWRITE INTO TABLE test1;

LOAD DATA INPATH '/user/andres/datasets/bixi2017/*' OVERWRITE INTO TABLE test1;

LOAD DATA INPATH '/user/andres/datasets/bixi2017/' OVERWRITE INTO TABLE test1;

Any of those above have worked, any idea guys about how should I go thru? thanks so much

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Load several files into HIVE table

Rising Star

Hi Guys, I'm so so .... Well, I just remember that you can create just an external table stored in the same folder all files with the same structure are located. So , in that way I will load whole records in one shoot.

> CREATE EXTERNAL TABLE bixi_his

> ( > STATIONS ARRAY<STRUCT<id: INT,s:STRING,n:string,st:string,b:string,su:string,m:string,lu:string,lc:string,bk:string,bl:string,la:float,lo:float,da:int,dx:int,ba:int,bx:int>>, > SCHEMESUSPENDED STRING,

> TIMELOAD BIGINT > )

> ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'

> LOCATION '/user/ingenieroandresangel/datasets/bixi2017/';

thanks

View solution in original post

1 REPLY 1
Highlighted

Re: Load several files into HIVE table

Rising Star

Hi Guys, I'm so so .... Well, I just remember that you can create just an external table stored in the same folder all files with the same structure are located. So , in that way I will load whole records in one shoot.

> CREATE EXTERNAL TABLE bixi_his

> ( > STATIONS ARRAY<STRUCT<id: INT,s:STRING,n:string,st:string,b:string,su:string,m:string,lu:string,lc:string,bk:string,bl:string,la:float,lo:float,da:int,dx:int,ba:int,bx:int>>, > SCHEMESUSPENDED STRING,

> TIMELOAD BIGINT > )

> ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'

> LOCATION '/user/ingenieroandresangel/datasets/bixi2017/';

thanks

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here