Support Questions
Find answers, ask questions, and share your expertise

org.apache.storm.hive.common.HiveWriter$ConnectFailure: Failed connecting to EndPoint

org.apache.storm.hive.common.HiveWriter$ConnectFailure: Failed connecting to EndPoint

I am facing issue in storm and hive streaming with HDP 2.5. I am getting error as,

Org.apache.storm.hive.common.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://base1.rolta.com:9083', database='default', table='table_mqtt', partitionVals=[2017/08/242] }
    at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:80) ~[stormjar.jar:?]
    at org.apache.storm.hive.common.HiveUtils.makeHiveWriter(HiveUtils.java:50) ~[stormjar.jar:?]
    at org.apache.storm.hive.bolt.HiveBolt.getOrCreateWriter(HiveBolt.java:271) ~[stormjar.jar:?]
    at org.apache.storm.hive.bolt.HiveBolt.execute(HiveBolt.java:114) [stormjar.jar:?]
    at org.apache.storm.daemon.executor$fn__9364$tuple_action_fn__9366.invoke(executor.clj:734) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
    at org.apache.storm.daemon.executor$mk_task_receiver$fn__9285.invoke(executor.clj:466) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
    at org.apache.storm.disruptor$clojure_handler$reify__8798.onEvent(disruptor.clj:40) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
    at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
    at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
    at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
    at org.apache.storm.daemon.executor$fn__9364$fn__9377$fn__9430.invoke(executor.clj:853) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
    at org.apache.storm.util$async_loop$fn__656.invoke(util.clj:484) [storm-core-1.0.1.2.5.3.0-37.jar:1.0.1.2.5.3.0-37]
    at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
Caused by: org.apache.storm.hive.common.HiveWriter$TxnBatchFailure: Failed acquiring Transaction Batch from EndPoint: {metaStoreUri='thrift://base1.rolta.com:9083', database='default', table='table_mqtt', partitionVals=[2017/08/242] }
    at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:264) ~[stormjar.jar:?]
    at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72) ~[stormjar.jar:?]
    ... 13 more
Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to acquire lock on {metaStoreUri='thrift://base1.rolta.com:9083', database='default', table='table_mqtt', partitionVals=[2017/08/242] }
    at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:575) ~[stormjar.jar:?]
    at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:544) ~[stormjar.jar:?]
    at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:259) ~[stormjar.jar:?]
    at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72) ~[stormjar.jar:?]
    ... 13 more
Caused by: org.apache.thrift.transport.TTransportException
    at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) ~[stormjar.jar:?]
    at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) ~[stormjar.jar:?]
    at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378) ~[stormjar.jar:?]
    at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297) ~[stormjar.jar:?]
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204) ~[stormjar.jar:?]
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) ~[stormjar.jar:?]
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781) ~[stormjar.jar:?]
    at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768) ~[stormjar.jar:?]
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736) ~[stormjar.jar:?]
    at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:570) ~[stormjar.jar:?]
    at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:544) ~[stormjar.jar:?]
    at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:259) ~[stormjar.jar:?]
    at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72) ~[stormjar.jar:?]
    ... 13 more

I tried below,

1. Done coding as link http://storm.apache.org/releases/2.0.0-SNAPSHOT/storm-hive.html

2. Then changed hive-conf.xml on all nodes as link http://www.openkb.info/2015/06/hive-transaction-feature-in-hive-10.html

But then also getting error. Below is my pom.

<dependencies>
<dependency> 
   <groupId>joda-time</groupId> 
   <artifactId>joda-time</artifactId> 
   <version>2.9.9</version> 
</dependency> 
<dependency> 
   <groupId>org.apache.storm</groupId> 
   <artifactId>storm-core</artifactId> 
   <version>1.0.1</version> 
   <scope>provided</scope> 
</dependency>
 <dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-hive</artifactId>
<version>1.0.1</version>
</dependency>
<dependency>
  <groupId>org.eclipse.paho</groupId>
  <artifactId>org.eclipse.paho.client.mqttv3</artifactId>
  <version>1.1.0</version>
</dependency>
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.3.3</version>
</dependency>
 <dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-client</artifactId>
    <version>2.2.0</version>
    <exclusions>
        <exclusion>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-hdfs</artifactId>
    <version>2.2.0</version>
    <exclusions>
        <exclusion>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
        </exclusion>
    </exclusions>
</dependency>

Can anybody please help me with suggestions.