Support Questions

Find answers, ask questions, and share your expertise

flume with hbase with avro


We have data coming into hdfs through flume. This data is serially encoded. We use the avro schema to decode it.

Now we want to put this data from flume into a hbase table instead of hdfs.

We use below parameter to do serial decoding based on avro schema :

ragent.sinks.sink1.serializer.schemaURL = hdfs://

Will this same argument work when the target is hbase instead of hdfs? along with hbase table, hbase table column family etc?

Actually for some reason the data is not going in properly into hbase, and I am not sure if this avro schema is the issue.

Appreciate the insights.



this is the flume configuration hbase sink part : = channel1

r_hbase.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink

r_hbase.sinks.sink1.table = mev:rtable

r_hbase.sinks.sink1.columnFamily = me_data

r_hbase.sinks.sink1.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer r_hbase.sinks.sink1.serializer.schemaURL = hdfs:// r_hbase.sinks.sink1.serializer.columns = col1,col2,col3,col4,col5,col6,col7,col8,col9,col10