Reply
Highlighted
New Contributor
Posts: 3
Registered: ‎10-09-2017

Tutorial 1 - having real problems with avro

[ Edited ]

Guys Im really stuck. Very new to all of this and i started with exercise 1. The sqoop import into parquetfiles worked fine as set out in exercise 1 but then at the end it says

 

Now you have gone through the first basic steps to Sqoop structured data into HDFS, transform it into Avro file format (you can read about the benefits of Avro as a common format in Hadoop here), and import the schema files for use when we query this data.

 

...and thats when I started getting problems. I did the command again but this time specified used

sqoop import-all-tables \
 -m 1 \
 --connect jdbc:mysql://quickstart:3306/retail_db \
 --username=retail_dba \ --password=cloudera \ 
--compression-codec=snappy \
--as-sequencefile \
--warehouse-dir=/user/hive/warehouse \
 --hive-overwrite

 

tried this:

select * from products where product_id = 1004;

refresh default.products

 

but now in Hue I get the following error when trying to see the tables

 

File 'hdfs://quickstart.cloudera:8020/user/hive/warehouse/products/part-m-00000.avro' has an invalid version number: {l This could be due to stale metadata. Try running "refresh default.products".

 

Any idea how I can get myself out of this problem?

 

Announcements