Reply
Explorer
Posts: 7
Registered: ‎11-15-2016

Flume Twitter not able to fetch data from Hive table.

Hi,

     We are trying to fetch data from Twitter using Flume.

 

      We sucessfully pulled data as well. 

 

      Followed two ways to put data into hive:

 

      Approach 1: We used steps given in below link:

                https://github.com/cloudera/cdh-twitter-example

 

              But here we are able to load but while doing query its returning 0 records, even though files are available in external table path.!!!

 

       Approach 2: We followed steps from below link:

 

                  http://hadooptutorial.info/twitter-data-analysis-using-hadoop-flume/

 

             Here after loading to table, while running select query its giving following errror:

              > select * from tweets;
                  OK
                   Failed with exception java.io.IOException:org.apache.avro.AvroRuntimeException:                    java.io.IOException: Block size invalid or too large for this implementation: -40
     

           I have checked for this issue online, in below jira bug they have mensioned this issue is fixed:

           

                            https://issues.apache.org/jira/browse/AVRO-1597

      

           But still i am getting same error.

 

Can someone help to resolve the above errors???

Thanks & Regards,

Akki

Champion
Posts: 761
Registered: ‎05-16-2016

Re: Flume Twitter not able to fetch data from Hive table.

For your approach 1 

Did you try Refresh statement and the select statement. 

 

Highlighted
Explorer
Posts: 7
Registered: ‎11-15-2016

Re: Flume Twitter not able to fetch data from Hive table.

Hi,

     Ya we have tried with refresh as well. But no luck.

 

     If we can get any other ways then it would be really helpfull.


Thanks & Regards,

Akki

Announcements
New solutions