Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Solved Go to solution

ConsumeJMS in a bad state when sending batch of messages via MQ

avatar
author-rank dougspadotto_h
Explorer

Created ‎07-24-2018 12:11 PM

The ConsumeJMS processor which is the origin of data on a processor group is failing whenever I cat a file and send it over to amqsput client application.

When I run:

	cat <file> | /opt/mqm/samp/bin/amqsput <queue> <queue manager>

It fails. I then tried a script which echoes the file line by line but it also returns an error. I also made sure the file I cat is encoded as UTF-8 (ran iconv on it).

I found a JIRA treating a similar error, NIFI-4976, from March 2018, but I'm not sure if it made it into HDF 3.1.0.0, which is the version I'm using.

When I echo a single message, it works fine.

Here's what I see on nifi-app.log:

2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] Processor Administratively Yielded for 1 sec due to processing failure
2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] due to uncaught Exception: org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
        at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
        at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
        at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
        at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@653dd9ad'
        at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
        at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
        at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
        at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
        at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
        ... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
        ... 24 common frames omitted
2018-07-24 09:04:42,148 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] failed to process due to org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]; rolling back session: {}
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
        at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
        at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
        at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
        at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
        at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
        at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
        at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
        at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
        at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
        ... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
        ... 24 common frames omitted
2018-07-24 09:04:42,148 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] failed to process session due to org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]: {}
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
        at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
        at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
        at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
        at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
        at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
        at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
        at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
        at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
        at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
        ... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
        ... 24 common frames omitted
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] Processor Administratively Yielded for 1 sec due to processing failure
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] due to uncaught Exception: org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
        at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
        at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
        at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
        at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
        at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
        at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
        at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
        at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
        at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
        ... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
        ... 24 common frames omitted
3,300 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank dougspadotto_h
Explorer

Created ‎07-24-2018 03:37 PM

Update: this was due to special characters on the <file>, which amqsput can't handle correctly. I replaced them to regular chars and it went well.

For example, the string "São" was corrupted. "Sao" went fine.

View solution in original post

3,038 Views
0 Kudos
1 REPLY 1

avatar
author-rank dougspadotto_h
Explorer

Created ‎07-24-2018 03:37 PM

Update: this was due to special characters on the <file>, which amqsput can't handle correctly. I replaced them to regular chars and it went well.

For example, the string "São" was corrupted. "Sao" went fine.

3,039 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera Data Engineering 1.23: Access Spark from Your Favor...
What's New @ Cloudera
HBase REST server scaling support is Generally Available
What's New @ Cloudera
New CLI option in the update-database command
What's New @ Cloudera
New Action menu item in the Cloudera Operational Database UI
What's New @ Cloudera
Get the list of supported instance types in the Cloudera Ope...
View More Announcements