Reply
New Contributor
Posts: 3
Registered: ‎12-26-2018

kafka real time with cloudera and IBM MQ

Is it possible to capture IBM MQ data with Kafka-Cloudera?

 

The confluent company offers a "IBM MQ Connector" to capture IBM MQ data, but I'm not sure if I can do the same with Kafka-Cloudera.

 

Cloudera Employee
Posts: 277
Registered: ‎01-09-2014

Re: kafka real time with cloudera and IBM MQ

You can use the flume jms source (http://flume.apache.org/FlumeUserGuide.html#jms-source) to consume messages of the IBM MQ queue and either use a kafka channel or kafka sink to send those messages to kafka.

-pd
Highlighted
New Contributor
Posts: 3
Registered: ‎12-26-2018

Re: kafka real time with cloudera and IBM MQ

I understand, but what about this connectorIs it possible to integrate it into Cloudera?

Cloudera Employee
Posts: 277
Registered: ‎01-09-2014

Re: kafka real time with cloudera and IBM MQ

It is possible to use it, although kafka connect isn't officially supported by cloudera:

https://www.cloudera.com/documentation/kafka/latest/topics/kafka_known_issues.html

-pd
New Contributor
Posts: 3
Registered: ‎12-26-2018

Re: kafka real time with cloudera and IBM MQ

[ Edited ]

I understand.

 

and if instead of using Kafka I bet for spark streaming? I see that there is also a connector for MQ, is it compatible with Cloudera?