convert text file into avro file..

[#1]

Hi All,

i have to convert the text file into avro file

following are the contents of my files

emp_avro.avsc

{
  "type" : "record",
  "name" : "emp_avro",
  "namespace" : "default",
  "fields" : [ {
    "name" : "id",
    "type" : [ "null", "int" ],
    "doc" : "\u0000\u0000\u0000",
    "default" : null
  }, {
    "name" : "name",
    "type" : [ "null", "string" ],
    "default" : null
  }, {
    "name" : "sex",
    "type" : [ "null", "string" ],
    "default" : null
  }, {
    "name" : "state",
    "type" : [ "null", "string" ],
    "default" : null
  } ]
}

emp_info.txt

1,amit,male,TX
2,manisg,male,CA
3,hsan,male,AZ
4,shnk,male,FL
5,manisha,female,TX
6,shana,female,AZ
7,sahana,female,CA
8,seema,female,FL

i am using the following command to convert the text file into avro file

java -jar avro-tools.jar fromtext /home/cloudera/Downloads/emp_avro.avsc /home/cloudera/Downloads/emp_info.txt > /home/cloudera/Downloads/emp_avro.avro

and i got this file

emp_avro.avro

Writing line = 1
Writing line = 20
Writing line = 22
Writing line = 26
Writing line = 16
Writing line = 18
Writing line = 31
Writing line = 33
Writing line = 20
Writing line = 6
Writing line = 20
Writing line = 34
Writing line = 20
Writing line = 6
Writing line = 19
Writing line = 34
Writing line = 20
Writing line = 6
Writing line = 21
Writing line = 34
Writing line = 20
Writing line = 5
Writing line = 1


so this file can't be accesible through hive tables;
any assistance will be helpful

[#2]

Are you having issues creating a table backed as AVRO type? What is your CREATE TABLE statement, and how are you loading the file into the table?

For more on creating and using Avro tables in Hive, see http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_hive.html

[#3]

Hi Harsh,

Thanks for respond,

I have no problem in creating the table when backed table file is already in avro type.

But once I have created the table and loaded the data, it is done. After that if I have another text file, which I want to load in the same table, how will I do that?

I was trying to explicitly convert that text file into avro file (using avro tools)  the files contents in my previous question is that one.

so here it was not converting the text file into proper avro file which could be loaded by the hive.

[#4]

Why not creating an Hive table on top of the text file and then simply use an hive query to load the data into the avro table ?