insert into Elasticsearch failing

[#1]

I have created a external table but insert is failing :

CREATE EXTERNAL TABLE pa_lane_txn_es (
    txn_id      BIGINT,
    ext_plaza_id    STRING,
    toll_amt_collected BIGINT,
    toll_amt_full BIGINT,
    ext_lane_id   STRING)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'lane_txn/txn_id','es.mapping.id'='txn_id','es.nodes' = '127.0.0.1:9200');

curl can get to elastic search port 9200 fine

[root@hadoop1 ~]# curl 127.0.0.1:9200
{
  "name" : "node1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "OXNZn6ImQpK8zlF61FXWiw",
  "version" : {
    "number" : "5.5.0",
    "build_hash" : "260387d",
    "build_date" : "2017-06-30T23:16:05.735Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

ERROR

ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1499980645886_0006_1_00, diagnostics=[Task failed, taskId=task_1499980645886_0006_1_00_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"txn_id":2.0891216818E10,"ext_plaza_id":"500092","toll_amt_collected":109.0,"toll_amt_full":109.0,"ext_lane_id":"09"}
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
        at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"txn_id":2.0891216818E10,"ext_plaza_id":"500092","toll_amt_collected":109.0,"toll_amt_full":109.0,"ext_lane_id":"09"}
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
        ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"txn_id":2.0891216818E10,"ext_plaza_id":"500092","toll_amt_collected":109.0,"toll_amt_full":109.0,"ext_lane_id":"09"}
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:565)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
        ... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[127.0.0.1:9200]]

[#2 · Accepted solution]

Are you trying the curl command from the node where the task failed or is this just a single node cluster?

Regards,

Mastan

[#3]

its a 5 node cluster , I am using the curl command where elasticsearch is installed.

[#4]

this issue is fixed by installing elastic search on all the cluster nodes.