Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Trying to integrate ibm mq as a flume source

avatar
New Member

Hi guys!

I am looking for a guide/ Howto use flume with a jms source of ibm mq.. so far i have used the official documentation but with no luck.. 

Anybody has any expereience with this? 

 

1 ACCEPTED SOLUTION

avatar
Explorer

Okay, I finally got this working by setting up a File System Initial Context for the IBM MQ queue.

 

tier1.sources.source2.type = jms
tier1.sources.source2.providerURL = file:///home/cloudera/JNDI-Directory
tier1.sources.source2.initialContextFactory = com.sun.jndi.fscontext.RefFSContextFactory
tier1.sources.source2.destinationType = QUEUE
tier1.sources.source2.destinationName = theQueueName

View solution in original post

11 REPLIES 11

avatar

We didn't have the rights to create the JNDI file and the required tools. 

Initially we wrote a thin JMS client for IBM MQ + SSL using IBM libs and 

convert it to a  flume pollable source for IBM MQ + SSL, 

 

 

avatar
Reader

I am able to connect to IBM MQ using the steps mentioned here. But when Flume is trying to consume any messages from the Q, its throwing following exception.

 

com.ibm.msg.client.jms.DetailedMessageFormatExcept​ion: JMSCC0053: An exception occurred deserializing a message, exception: 'java.lang.ClassNotFoundException: null class'.
It was not possible to deserialize the message because of the exception shown.

 

1) I am using all the ibm mq client jars. Flume is starting with out any exception. But exception is coming when trying to consume

the messages .

2) I am putting a custom message [Serializable object] into Q which Flume need to consume.

3) Flume 1.5.0-cdh5.4.1

4) MQ Version 8.x

 

a1.sources=fe_s1
a1.channels=c1
a1.sinks=k1

a1.sources.s1.type=jms
a1.sources.s1.channels=c1
a1.sources.s1.initialContextFactory=com.sun.jndi.f​scontext.RefFSContextFactory
a1.sources.s1.connectionFactory=FLUME_CF
a1.sources.s1.destinationName=MY.Q
a1.sources.s1.providerURL=file:///home/JNDI-Direct​ory
a1.sources.s1.destinationType=QUEUE
a1.sources.s1.transportType=1
a1.sources.s1.userName=mqm
a1.sources.s1.batchSize=1

a1.channels.c1.type=memory
a1.channels.c1.capacity=10000
a1.channels.c1.transactionCapacity=100

a1.sinks.k1.type=logger
a1.sinks.k1.channel=c1