Nifi error with fetchelasticsearch processor

[#1]

I'm getting strange error while running the FetchElasticSearch processor, below is the error.

FetchElasticsearch[id=f2b2fee3-b940-4a73-8a28-0436e765c9a2] Failed to read into Elasticsearch due to None of the configured nodes are available: [{#transport#-1}{127.0.0.1}{localhost/127.0.0.1:9500}], this may indicate an error in configuration (hosts, username/password, etc.). Routing to retry: NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{127.0.0.1}{localhost/127.0.0.1:9500}]] 2016-08-30 11:58:17,930 ERROR [Timer-Driven Process Thread-3] o.a.n.p.elasticsearch.FetchElasticsearch org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [{#transport#-1}{127.0.0.1}{localhost/127.0.0.1:9500}] at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:290) ~[elasticsearch-2.1.0.jar:2.1.0] at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:207) ~[elasticsearch-2.1.0.jar:2.1.0] at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:55) ~[elasticsearch-2.1.0.jar:2.1.0] at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:283) ~[elasticsearch-2.1.0.jar:2.1.0] at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:347) ~[elasticsearch-2.1.0.jar:2.1.0] at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:85) ~[elasticsearch-2.1.0.jar:2.1.0] at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:59) ~[elasticsearch-2.1.0.jar:2.1.0] at org.apache.nifi.processors.elasticsearch.FetchElasticsearch.onTrigger(FetchElasticsearch.java:164) ~[nifi-elasticsearch-processors-0.7.0.jar:0.7.0] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-0.7.0.jar:0.7.0] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054) [nifi-framework-core-0.7.0.jar:0.7.0] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-0.7.0.jar:0.7.0] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-0.7.0.jar:0.7.0] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127) [nifi-framework-core-0.7.0.jar:0.7.0] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_91] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]

ElasticSearch configurations are :

Host : localhost

port : 9500

not sure about the root cause of the problem, do the needful.

[#2]

@Matt Burgess, @Pierre Villard

- could you please help me with this

[#3]

@Pravin Battle your cluster name, host name/IP, port or authentication credentials are probably not correct, or maybe you don't have Elasticsearch configured to listen on the right IP/port. I'd double check that cluster.name, network.host and transport.tcp.port in elasticsearch.yml are as expected.

If you run netstat or telnet/nc to port 9500 on localhost what do you get? Can you curl the REST API?

[#4 · Accepted solution]

I found the problem, the fetchelasticsearch gives error if you give the default port(9200) , it doesn't matter with the port on which the elasticsearch is running. I started my elasticsearch in default port and gave the port as 9300 in fetchelasticsearch processor, worked 🙂

[#5]

The FetchElasticsearch processor uses the native transport, whose default port is 9300. In Nifi 0.7.0 (and in the upcoming HDF 2.0), there is a FetchElasticsearchHttp (and PutElasticsearchHttp) which uses the REST API (whose default port is 9200).