Reply
Explorer
Posts: 20
Registered: ‎12-02-2013

Logging Avro object into Hbase (separate columns) via Flume

I'm using Flume to collect logs coming in avro format into Hbase. When I use the SimpleAsyncHbaseSerializer, the entire avro object gets logged into one hbase column. I would like to each field in the avro object to be logged into a separate column in hbase. Would this require a custom event serializer or can it be accomplished through other means (like using a combination of morphline commands)?

Highlighted
New Contributor
Posts: 5
Registered: ‎09-20-2017

Re: Logging Avro object into Hbase (separate columns) via Flume

I am trying to:

1) create Avro objects in variables that can

2) be stored in different Hbase columns

 

I can find examples of how to combine data and a json schema and write the whole thing to a file, but I need the Python code that will just put the data and definition into a variable. 

Announcements
The Kite SDK is a collection of docs, sample code, APIs, and tools to make Hadoop application development faster. Learn more at http://kitesdk.org.