Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

insert into Elasticsearch failing

Master Collaborator

I have created a external table but insert is failing :

CREATE EXTERNAL TABLE pa_lane_txn_es (
    txn_id      BIGINT,
    ext_plaza_id    STRING,
    toll_amt_collected BIGINT,
    toll_amt_full BIGINT,
    ext_lane_id   STRING)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'lane_txn/txn_id','es.mapping.id'='txn_id','es.nodes' = '127.0.0.1:9200');

curl can get to elastic search port 9200 fine

[root@hadoop1 ~]# curl 127.0.0.1:9200
{
  "name" : "node1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "OXNZn6ImQpK8zlF61FXWiw",
  "version" : {
    "number" : "5.5.0",
    "build_hash" : "260387d",
    "build_date" : "2017-06-30T23:16:05.735Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

ERROR

ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1499980645886_0006_1_00, diagnostics=[Task failed, taskId=task_1499980645886_0006_1_00_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"txn_id":2.0891216818E10,"ext_plaza_id":"500092","toll_amt_collected":109.0,"toll_amt_full":109.0,"ext_lane_id":"09"}
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:173)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
        at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"txn_id":2.0891216818E10,"ext_plaza_id":"500092","toll_amt_collected":109.0,"toll_amt_full":109.0,"ext_lane_id":"09"}
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:91)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
        ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"txn_id":2.0891216818E10,"ext_plaza_id":"500092","toll_amt_collected":109.0,"toll_amt_full":109.0,"ext_lane_id":"09"}
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:565)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:83)
        ... 17 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[127.0.0.1:9200]]
1 ACCEPTED SOLUTION

New Contributor

Are you trying the curl command from the node where the task failed or is this just a single node cluster?

Regards,

Mastan

View solution in original post

3 REPLIES 3

New Contributor

Are you trying the curl command from the node where the task failed or is this just a single node cluster?

Regards,

Mastan

Master Collaborator

its a 5 node cluster , I am using the curl command where elasticsearch is installed.

Master Collaborator

this issue is fixed by installing elastic search on all the cluster nodes.