Member since
09-29-2016
54
Posts
9
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2933 | 07-03-2017 09:12 PM | |
2249 | 05-22-2017 08:19 PM |
10-30-2017
06:36 PM
@Xu Zhe, yes the cluster is fully stable now.
... View more
10-27-2017
11:47 AM
Hi @Xu Zhe As mentioned in the comments by Wynner, the solution I used was to clean up the NiFi repositories and restart the cluster nodes at the same time.
... View more
09-06-2017
03:38 PM
Hi @Matt Burgess, I was able to connect successfully following your guidelines. I had to call the elasticsearch folder from the X-Pack Zip for the ES version 5.0.1 along with the 5.0.1 Transport Jar in the X-Pack Transport Location property as a comma-delimited list. Thanks for all the help, greatly appreciate it!
... View more
08-28-2017
07:16 PM
1 Kudo
Hello, I am trying to send feed from nifi version nifi-1.2.0.3.0.0.0-453 to elasticsearch version 5.5.1 however I keep getting the following error: PutElasticsearch5[id=f4da516d-2b44-1955-0000-00000a063fa3]
PutElasticsearch5[id=f4da516d-2b44-1955-0000-00000a063fa3] failed to process due
to org.apache.nifi.processor.exception.ProcessException:
org.apache.nifi.processor.exception.ProcessException: X-Pack plugin could not be
loaded and/or configured; rolling back session:
org.apache.nifi.processor.exception.ProcessException:
org.apache.nifi.processor.exception.ProcessException: X-Pack plugin could not be
loaded and/or configured
The full error log is the following: ERROR [Timer-Driven Process Thread-16] o.a.n.p.elasticsearch.PutElasticsearch5 PutElasticsearch5[id=f4da516d-2b44-1955-0000-00000a063fa3] Failed to create Elasticsearch client due to org.apache.nifi.processor.exception.ProcessException: X-Pack plugin could not be loaded and/or configured: org.apache.nifi.processor.exception.ProcessException: X-Pack plugin could not be loaded and/or configured
org.apache.nifi.processor.exception.ProcessException: X-Pack plugin could not be loaded and/or configured
at org.apache.nifi.processors.elasticsearch.AbstractElasticsearch5TransportClientProcessor.getTransportClient(AbstractElasticsearch5TransportClientProcessor.java:222)
at org.apache.nifi.processors.elasticsearch.AbstractElasticsearch5TransportClientProcessor.createElasticsearchClient(AbstractElasticsearch5TransportClientProcessor.java:170)
at org.apache.nifi.processors.elasticsearch.AbstractElasticsearch5Processor.setup(AbstractElasticsearch5Processor.java:94)
at org.apache.nifi.processors.elasticsearch.PutElasticsearch5.onTrigger(PutElasticsearch5.java:175)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.nifi.processors.elasticsearch.AbstractElasticsearch5TransportClientProcessor.getTransportClient(AbstractElasticsearch5TransportClientProcessor.java:215)
... 15 common frames omitted
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.elasticsearch.xpack.XPackPlugin
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:451)
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:116)
at org.elasticsearch.client.transport.TransportClient.newPluginService(TransportClient.java:81)
at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:106)
at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:228)
at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:69)
at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.<init>(PreBuiltXPackTransportClient.java:50)
at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.<init>(PreBuiltXPackTransportClient.java:46)
... 20 common frames omitted
I tried following the suggestions in the link: https://community.hortonworks.com/questions/81606/where-should-i-store-files-used-by-processors-in-a.html , but I am unable to fix the issue. I tried putting the x-pack jar and the transport jar for the elasticsearch version 5.5.1 on the X-Pack Transport Location configuration of the PutElasticsearch5 processor and I also tried using the x-pack jar and the transport jar of the elasticsearch version 5.0.1, however nothing seems to be working. Any suggestions on the issue would be greatly appreciated.
... View more
Labels:
- Labels:
-
Apache NiFi
07-06-2017
08:12 PM
@Matt Clarke that seemed to fix the issue, thanks a lot!
... View more
07-05-2017
10:22 PM
Full error trace: 2017-07-05 17:04:12,336 WARN [Replicate Request Thread-8] o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET /nifi-api/flow/process-groups/04a08abd-d360-3539-ad01-835bc54126b0 due to {}
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:611)
at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:822)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
... 12 common frames omitted
... View more
07-05-2017
09:42 PM
Hello, I am on the process of changing the nodes of our current nifi cluster for completely nifi dedicated nodes. I usually ran a 4 node nifi cluster and today I added 4 new nodes to that cluster. The main goal is to take out the old 4 nodes and to keep the new 4 nifi dedicated nodes in the cluster. All the nodes in the cluster are currently running on the nifi version nifi-1.2.0.3.0.0.0-453. I added the 4 new nodes successfully and gave them the proxy permissions and everything. I am able to log in to the new nodes and access the UI from them and they connected to the cluster without any issues. However whenever I try to access a specific process group from one of the 4 new nodes I keep getting the following error: This is only happening whenever I try to access that specific process group from any of the new nodes. If I access it from any of the old nodes no error happens and I can access the process group just fine. I looked at the logs from the new nodes and this is what I see: 2017-07-05 17:10:56,745 WARN [Replicate Request Thread-10] o.a.n.c.c.h.r.ThreadPoolRequestReplicator
com.sun.jersey.api.client.ClientHandlerException: java.net.SocketTimeoutException: Read timed out
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:611)
at org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:822)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
... 12 common frames omitted
Does anyone have any ideas of what could be causing this? Is there a configuration that I have to change to avoid this error? or a permission that I have to give these nodes? Any insights would be extremely helpful.
... View more
Labels:
- Labels:
-
Apache NiFi
07-03-2017
09:12 PM
I was able to resolve the problem. The issue was that my kbr5.conf file that I called in the nifi.kerberos.krb5.file line in the nifi.properties file, was not configured properly. After properly configuring it and calling it in the nifi.properties file I was able to successfully send to kafka. I also had to use the PublishKafka_0_10 processor instead of the PutKafka processor since the version of kafka I was trying to connect to was 0.10.
... View more
07-03-2017
06:14 PM
@Shashank Chandhok
I did some changes to the jass.conf file but I am still getting the same error. This is the new jass.conf file:
... View more
07-03-2017
04:01 PM
Hello @Shashank Chandhok, I named the jass.conf file zookeeper-jaas.conf and the configuration is the following: The PutKafka configuration is the following one: The boostrap.conf line I added is the following one: The kafka version that we are trying to connect to is: 0.10.0
... View more