Member since
09-02-2016
12
Posts
1
Kudos Received
0
Solutions
10-06-2016
09:07 AM
But Just want to make a note that it defeates the purpose of using AVRO schema as for any schema changes we will have to make changes to the AVRO schema file and also drop the table and recreate them using the new schema to work.
... View more
09-19-2016
08:09 AM
Voila its works fine they way you asked me to define the tables to get to view the data on Impala. Thanks much Alex.
... View more
09-16-2016
10:12 AM
I am trying out the options you have suggested alex, i should have my results mostly by today.
... View more
09-12-2016
02:31 PM
Sorry for replying late. Alex, I dint alter the table or columns, this is how i created the table. CREATE TABLE MI_FULL
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
WITH SERDEPROPERTIES
('avro.schema.url'='hdfs://path/filename.avsc')
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES
('avro.schema.url'=hdfs://path/filename.avsc)
; And then did an insert into this table, please let me know if there is something wrong with creating like this.
... View more
09-06-2016
01:42 PM
I did kwho, i dont see any entries on the files you have listed.
... View more
09-02-2016
08:18 AM
I have an AVRO schema which works and querries fine on HIVE, but when we query the same on Impala it thorws an error saying "Your query has the following error(s): Could not connect to <HOST NAME>:21050" Which is weird and i dont see any log information on Hue as i am able to browse other tables that are defined on TEXT format within same database. we are on impala-2.1.3+cdh5.3.3+0 One thing which looks wired on my schema when i decsribe it on Impala is it has the first row as NULL like below and i am not sure why! My AVRO schema file is defined like below: Its just sample and all the fields are defined as STRING { "namespace": "HDFS", "name": "sample", "type": "record", "fields":[{"name":"recipient_identification_number", "type":"string"}, {"name":"validation_digit", "type":"string"}, {"name":"eligibility_status_code", "type":"string"}, {"name":"recipient_last_name", "type":"string"}, {"name":"recipient_first_name", "type":"string"}, {"name":"recipient_middle_name", "type":"string"}, {"name":"recipient_name_appel", "type":"string"}, {"name":"recipient_dob", "type":"string"}, {"name":"recipient_ssn", "type":"string"}, {"name":"recipient_ssn_prefix", "type":"string"}, {"name":"recipient_race_code", "type":"string"}, PS: Not sure why my pics have these wired color.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Impala
-
Cloudera Hue
-
HDFS