Support Questions

Find answers, ask questions, and share your expertise

How can I save my Hbase data to sql server using Nifi ?

Bottleneck is I am getting the data from hbase in below format :

"D:Field1":"@'\uFFFD\uFFFDQ\uFFFD\u001F"

"D:Feild2":"@\uFFFDs\uFFFD\uFFFD\u001E\uFFFD"

"D:Feild3":"@O\u0006fffff"

"D:Feild4":"\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000"

"D:Feild5":"\u0000\u0000\u0000\u0000\u0000\u0000\u0000H"

This data is in this format because I am putting data in hbase using spark job and converting it in Bytes

import org.apache.hadoop.hbase.util.Bytes

input._2.foreach( record => {

put.addColumn(Bytes.toBytes(builder.columnFamilyName),Bytes.toBytes(record._1),record._2.get.toBytes)

})

3 REPLIES 3

Super Guru

You can use GetHBase or (as of NiFi 1.6.0) ScanHBase to get the rows out of HBase and into JSON format. You can then use various processors for converting that data, such as JoltTransformJSON if necessary to change the JSON into an array of objects containing key/value pairs, then PutDatabaseRecord to send them to SQL Server. If other transformations are necessary before PutDatabaseRecord, you should be able to accomplish this with the record-aware processors such as UpdateRecord, ConvertRecord, QueryRecord, etc. If for some reason you must split the rows into individual flow files, you can use SplitJSON, then possible ConvertJSONToSQL and PutSQL, although this approach is not recommended as it is not as efficient as the record-based solution.

Thanks @Matt Burgess
I am already having the flow as GetHBase -> EvaluateJsonPath -> ReplaceText -> PutDatabaseRecord

Where ,
GetHbase is getting data from hbase tablesEvaluateJsonPath is saving value in form of attributes

ReplaceText is forming data in a line ${Name}, ${age}, ${city}
PutDatabaseRecord is saving data in SQL data base.

Issue is where my data contains value like "@\uFFFDs\uFFFD\uFFFD\u001E\uFFFD". (it should be some integer/double value)

i am not able to figure it out which processor to use.. or anyother way to proceed.

Still struggling in converting Bytes from Hbase to proper data type, when data is fetched using GetHBase processor in Nifi

@Matt Burgess @Bryan Bende @Matt Clarke

Thanks in advance

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.