Member since
11-22-2016
16
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3431 | 07-24-2018 03:37 PM |
06-18-2019
12:24 PM
Hi @Pierre Villard , The GET request for /nifi-api/flow/process-groups/{id}/controller-services is returning all Controller Services, not only the ones for the Process Group {id}. Do you mean I need to parse the returned JSON and look only for the ones with parentGroupId = {id} ? Thanks in advance.
... View more
07-24-2018
05:28 PM
I found it. In my case it was /user/nifi on HDFS, which is the user that the flow connects with. It moves a .jar file under the .hiveJars subdirectory on that directory in order to run commands.
... View more
07-24-2018
05:15 PM
@Saikrishna Tarapareddy, can you please specify what folders and files needed the permissions? Are they on the HiveServer2 server or on HDFS?
... View more
07-24-2018
03:37 PM
Update: this was due to special characters on the <file>, which amqsput can't handle correctly. I replaced them to regular chars and it went well. For example, the string "São" was corrupted. "Sao" went fine.
... View more
07-24-2018
12:11 PM
The ConsumeJMS processor which is the origin of data on a processor group is failing whenever I cat a file and send it over to amqsput client application.
When I run:
cat <file> | /opt/mqm/samp/bin/amqsput <queue> <queue manager>
It fails. I then tried a script which echoes the file line by line but it also returns an error. I also made sure the file I cat is encoded as UTF-8 (ran iconv on it). I found a JIRA treating a similar error, NIFI-4976, from March 2018, but I'm not sure if it made it into HDF 3.1.0.0, which is the version I'm using.
When I echo a single message, it works fine.
Here's what I see on nifi-app.log:
2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] Processor Administratively Yielded for 1 sec due to processing failure
2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] due to uncaught Exception: org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
2018-07-24 09:04:41,144 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@653dd9ad'
at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
... 24 common frames omitted
2018-07-24 09:04:42,148 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] failed to process due to org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]; rolling back session: {}
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
... 24 common frames omitted
2018-07-24 09:04:42,148 ERROR [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] failed to process session due to org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]: {}
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
... 24 common frames omitted
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.apache.nifi.jms.processors.ConsumeJMS ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] Processor Administratively Yielded for 1 sec due to processing failure
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding ConsumeJMS - JMSConsumer[destination:null; pub-sub:false;] due to uncaught Exception: org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
2018-07-24 09:04:42,149 WARN [Timer-Driven Process Thread-8] o.a.n.c.t.ContinuallyRunProcessorTask
org.apache.nifi.jms.processors.MessageBodyToBytesConverter$MessageConversionException: Failed to convert BytesMessage to byte[]
at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:42)
at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:106)
at org.apache.nifi.jms.processors.JMSConsumer$1.doInJms(JMSConsumer.java:66)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:494)
at org.apache.nifi.jms.processors.JMSConsumer.consume(JMSConsumer.java:66)
at org.apache.nifi.jms.processors.ConsumeJMS.rendezvousWithJms(ConsumeJMS.java:156)
at org.apache.nifi.jms.processors.AbstractJMSProcessor.onTrigger(AbstractJMSProcessor.java:147)
at org.apache.nifi.jms.processors.ConsumeJMS.onTrigger(ConsumeJMS.java:58)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.ibm.msg.client.jms.DetailedJMSException: JMSCMQ1049: The character set '1208(UTF-8) Unmappable Action: REPORT Unmappable Replacement: 63' cannot convert some or all of the string '[B@65f756f8'
at sun.reflect.GeneratedConstructorAccessor608.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319)
at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:226)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:382)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromByteBuffer(WMQUtils.java:421)
at com.ibm.msg.client.wmq.common.internal.messages.WMQTextMessage.getText(WMQTextMessage.java:240)
at com.ibm.msg.client.jms.internal.JmsTextMessageImpl.getText(JmsTextMessageImpl.java:205)
at com.ibm.jms.JMSTextMessage.getText(JMSTextMessage.java:124)
at org.apache.nifi.jms.processors.MessageBodyToBytesConverter.toBytes(MessageBodyToBytesConverter.java:40)
... 19 common frames omitted
Caused by: java.nio.charset.MalformedInputException: Input length = 1
at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
at com.ibm.mq.jmqi.system.JmqiCodepage.bytesToString(JmqiCodepage.java:690)
at com.ibm.msg.client.wmq.common.internal.WMQUtils.computeTextFromBytes(WMQUtils.java:375)
... 24 common frames omitted
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)
07-24-2018
11:59 AM
Thanks Matt. I installed the "aligned" HDP and HDF versions and it worked.
... View more
06-28-2018
03:49 PM
In my experience, the connection error goes away if you remove "thrift://" from the URI.
... View more
06-28-2018
03:45 PM
I've a Nifi datalow where I get data from a source, build an Avro file and am ready to stream to Hive. I setup Hive as requested on https://community.hortonworks.com/questions/59411/how-to-use-puthivestreaming.html. I also moved the entire hive-client and hadoop-client directories to the Nifi node, and reference this hive-site.xml on the PutHiveStreaming processor. But I'm getting the following error on this processor: 2018-06-28 11:29:21,053 ERROR [Timer-Driven Process Thread-7] o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=46762ec7-0164-1000-0000-00001008870a] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: Error writing [org.apache.nifi.processors.hive.PutHiveStreaming$HiveStreamingRecord@733df43a] to Hive Streaming transaction due to com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient: {}
org.apache.nifi.processor.exception.ProcessException: Error writing [org.apache.nifi.processors.hive.PutHiveStreaming$HiveStreamingRecord@733df43a] to Hive Streaming transaction due to com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient
at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onHiveRecordsError$1(PutHiveStreaming.java:555)
at org.apache.nifi.processor.util.pattern.ExceptionHandler$OnError.lambda$andThen$0(ExceptionHandler.java:54)
at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onHiveRecordError$2(PutHiveStreaming.java:562)
at org.apache.nifi.processor.util.pattern.ExceptionHandler.execute(ExceptionHandler.java:148)
at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onTrigger$12(PutHiveStreaming.java:740)
at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2175)
at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2145)
at org.apache.nifi.processors.hive.PutHiveStreaming.onTrigger(PutHiveStreaming.java:694)
at org.apache.nifi.processors.hive.PutHiveStreaming.lambda$onTrigger$4(PutHiveStreaming.java:572)
at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
at org.apache.nifi.processor.util.pattern.RollbackOnFailure.onTrigger(RollbackOnFailure.java:184)
at org.apache.nifi.processors.hive.PutHiveStreaming.onTrigger(PutHiveStreaming.java:572)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2203)
at com.google.common.cache.LocalCache.get(LocalCache.java:3937)
at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)
at org.apache.hive.hcatalog.common.HiveClientCache.getOrCreate(HiveClientCache.java:291)
at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:266)
at org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.getMetaStoreClient(HiveEndPoint.java:544)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:312)
at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:278)
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnectionImpl(HiveEndPoint.java:215)
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:192)
at org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:122)
at org.apache.nifi.util.hive.HiveWriter.lambda$newConnection$1(HiveWriter.java:246)
at org.apache.nifi.util.hive.HiveWriter.lambda$callWithTimeout$4(HiveWriter.java:365)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
... 3 common frames omitted
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1566)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:124)
at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:295)
at org.apache.hive.hcatalog.common.HiveClientCache$5.call(HiveClientCache.java:291)
at com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
... 17 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564)
... 27 common frames omitted
Caused by: java.lang.NullPointerException: null
at org.apache.thrift.transport.TSocket.open(TSocket.java:209)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:487)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:282)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:188)
at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.<init>(HiveClientCache.java:406)
... 32 common frames omitted I thought that by copying and referencing the hive-client dir I'd have all the jars needed. Is this a problem with the CLASSPATH, with the processor configuration or something else?
... View more
Labels:
- Labels:
-
Apache Hive
-
Cloudera DataFlow (CDF)
01-18-2018
03:44 PM
Hi Dinesh. These are default values for recent versions of Hive (0.13.0 and later). Sources: https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.server2.async.exec.threads and http://atlas.apache.org/Bridge-Hive.html
... View more
05-09-2017
06:12 PM
I could at least start Zookeeper by doing this (change for your environment's exact paths): cd /usr/hdp/2.5.3.0-37/zookeeper; mkdir bin; cd bin; cp /var/lib/ambari-agent/cache/common-services/ZOOKEEPER/3.4.5/package/files/* .; ln -s /usr/hdp/2.3.0-37/zookeeper /usr/hdp/current/zookeeper-server
Source for the last command in the string of commands was: https://github.com/KaveIO/AmbariKave/wiki/Detailed-Guides-Troubleshooting
... View more