question Re: Kafka connect in Archives of Support Questions (Read Only)

Kafka connect

arunpoy — Wed, 05 Apr 2017 13:19:09 GMT

I have a use case where i need to move data from a mysql database to Hbase on a realtime basis using kafka as the intemediate data store. Will Kafka connect help to achieve this usecase.

Re: Kafka connect

jsensharma — Wed, 05 Apr 2017 13:28:11 GMT

@ARUN

You can take the following references to achieve the same using MySQL -->Kafka --> HBase:

1). https://github.com/wushujames/kafka-mysql-connector

2) https://github.com/mravi/kafka-connect-hbase

If you are open for other options like using "sqoop" then you can refer to the following as well:

1). https://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_importing_data_into_hbase

2). http://www.dummies.com/programming/big-data/hadoop/importing-data-into-hbase-with-sqoop/

3). https://acadgild.com/blog/how-to-import-table-from-mysql-to-hbase/

Re: Kafka connect

arunpoy — Wed, 05 Apr 2017 13:51:22 GMT

Thanks @Jay SenSharma, It helps, our case is more of realtime in nature. so the usage of sqoop is not possible