Created 12-20-2018 07:15 PM
We have data coming into hdfs through flume. This data is serially encoded. We use the avro schema to decode it.
Now we want to put this data from flume into a hbase table instead of hdfs.
We use below parameter to do serial decoding based on avro schema :
ragent.sinks.sink1.serializer.schemaURL = hdfs://host.com:8020/tmp/avroschemas-new/rtable.json
Will this same argument work when the target is hbase instead of hdfs? along with hbase table, hbase table column family etc?
Actually for some reason the data is not going in properly into hbase, and I am not sure if this avro schema is the issue.
Appreciate the insights.
Created 12-20-2018 07:24 PM
this is the flume configuration hbase sink part :
r_hbase.sinks.sink1.channel = channel1
r_hbase.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
r_hbase.sinks.sink1.table = mev:rtable
r_hbase.sinks.sink1.columnFamily = me_data
r_hbase.sinks.sink1.serializer = org.apache.flume.sink.hbase.SimpleHbaseEventSerializer r_hbase.sinks.sink1.serializer.schemaURL = hdfs://host.com:8020/tmp/avroschemas-new/rtable.json r_hbase.sinks.sink1.serializer.columns = col1,col2,col3,col4,col5,col6,col7,col8,col9,col10