Cloudera Community

Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.
Solved Go to solution

ConsumeJMS in a bad state when sending batch of messages via MQ

avatar
author-rank dougspadotto_h
New Member

Created on ‎07-24-2018 12:11 PM - last edited on ‎04-21-2026 05:15 AM by Community Manager Community Manager

The ConsumeJMS processor which is the origin of data on a processor group is failing whenever I cat a file and send it over to amqsput client application.

When I run:

	cat <file> | /opt/mqm/samp/bin/amqsput <queue> <queue manager>

It fails. I then tried a script which echoes the file line by line but it also returns an error. I also made sure the file I cat is encoded as UTF-8 (ran iconv on it).

I found a JIRA treating a similar error, NIFI-4976, from March 2018, but I'm not sure if it made it into HDF 3.1.0.0, which is the version I'm using.

When I echo a single message, it works fine.

Here's what I see on nifi-app.log:

2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] Processor Administratively Yielded for 1 sec due to processing failure
2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] due to uncaught Exception: org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
        at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
        at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
        at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
        at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@653dd9ad'
        at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
        at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
        at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
        at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
        at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
        ... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
        ... 24 common frames omitted
2018-07-24 09:04:42,148 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] failed to process due to org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]; rolling back session: {}
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
        at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
        at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
        at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
        at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
        at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
        at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
        at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
        at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
        at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
        ... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
        ... 24 common frames omitted
2018-07-24 09:04:42,148 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] failed to process session due to org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]: {}
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
        at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
        at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
        at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
        at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
        at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
        at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
        at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
        at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
        at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
        ... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
        ... 24 common frames omitted
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] Processor Administratively Yielded for 1 sec due to processing failure
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] due to uncaught Exception: org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
        at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
        at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
        at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
        at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
        at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
        at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
        at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
        at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
        at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
        at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
        at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
        at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
        ... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
        at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
        at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
        at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
        at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
        ... 24 common frames omitted
4,245 Views
0 Kudos
1 ACCEPTED SOLUTION

avatar
author-rank dougspadotto_h
New Member

Created ‎07-24-2018 03:37 PM

Update: this was due to special characters on the <file>, which amqsput can't handle correctly. I replaced them to regular chars and it went well.

For example, the string "São" was corrupted. "Sao" went fine.

View solution in original post

3,983 Views
0 Kudos
1 REPLY 1

avatar
author-rank dougspadotto_h
New Member

Created ‎07-24-2018 03:37 PM

Update: this was due to special characters on the <file>, which amqsput can't handle correctly. I replaced them to regular chars and it went well.

For example, the string "São" was corrupted. "Sao" went fine.

3,984 Views
0 Kudos
Announcements
What's New @ Cloudera
Cloudera Real time Monitoring for 7.1.9+ and 7.2.18+ with Cl...
What's New @ Cloudera
Announcing Cloudera Streaming Analytics 1.17: Python UDFs, S...
Community Announcements
Content Update: Evolving Our Community Content for Better, A...
Community Announcements
Product Name Updates — Community Label Changes & Notificatio...
What's New @ Cloudera
Announcing Cloudera Data Lineage for Trino
View More Announcements