Posts: 43
Registered: ‎08-20-2013

Flume HBase integration..

I has been using flume to ingest data into HBase for almost a year.  Everything works out fine. 


I wrote my own implemetation of AsyncHbaseEventSerializer to generate Puts. 


However, I found there is a real limitation here.  I can't serialize something into DeleteRequest.  AsyncHbaseEventSerializer does not have such method. 


So how do you suppose to use flume for cleanup purpose?  


For example: 


Day 1:  I put two columns for a row c1=a; c2=b

Day 2:  I got a new record for the row c1=a_chg 


My Serializer can easily generate put request to put c1=a_chg; But c2 is removed and I need to delete column c2 for this rowKey. I can't figure out how to generate a DeleteRequest.  


What is your experience? What should I do?

Posts: 43
Registered: ‎08-20-2013

Re: Flume HBase integration..

Am I the only one updating HBase from flume? Someone out there has experience on this and care to share?

New solutions