Hi Matt, I've upgraded my environment. I have HIve on HDP 2.5 (environment 1) and storm on HDF 2.1 (environment 2) I have the same roor: On storm: Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to acquire lock on {metaStoreUri='thrift://hdp1.local:9083', database='default', table='stock_prices', partitionVals=[Marcin] } at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:575) ~[stormjar.jar:?] On hive metastore: 2017-03-29 11:56:29,926 ERROR [pool-5-thread-17]: server.TThreadPoolServer (TThreadPoolServer.java:run(297)) - Error occurred during processing of message. java.lang.IllegalStateException: Unexpected DataOperationType: UNSET agentInfo=Unknown txnid:54 at org.apache.hadoop.hive.metastore.txn.TxnHandler.enqueueLockWithRetry(TxnHandler.java:938) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:814) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaSt ... View more

Hi Matt, Thank you very much for you reply. Now I know that I can fix it and I'm very happy with that. I'm begginer. I've installed ambari on my 4 virtual machines just to know Big Data. I'm using repositories (I can see it in my ambari web console) : http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.5.0.0 http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14 During ambari installation I used ambari list http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.4 .0.1/ambari.list I know that this ambari list points to HDP 2.5 Now my question is : How can I upgrade HDF ? Should I use other repositories ? Sorry if my question is stupid.... ... View more

Hi Matt, I have the same error with storm hive bolt and hive (hdp 2.5) It works fine with hdp 2.4. I have problem only with hdp 2.5. I don't want to rollback to older HDP versio. Can you help me to solve it . Any workaround or fix ... or something.... Storm logs below: org.apache.storm.hive.common.HiveWriter$ConnectFailure: Failed connecting to EndPoint {metaStoreUri='thrift://ambari.local:9083', database='default', table='stock_prices', partitionVals=[Marcin] } at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:80) ~[stormjar.jar:?] at org.apache.storm.hive.common.HiveUtils.makeHiveWriter(HiveUtils.java:50) ~[stormjar.jar:?] at org.apache.storm.hive.bolt.HiveBolt.getOrCreateWriter(HiveBolt.java:259) ~[stormjar.jar:?] at org.apache.storm.hive.bolt.HiveBolt.execute(HiveBolt.java:112) [stormjar.jar:?] at org.apache.storm.daemon.executor$fn__9362$tuple_action_fn__9364.invoke(executor.clj:734) [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245] at org.apache.storm.daemon.executor$mk_task_receiver$fn__9283.invoke(executor.clj:466) [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245] at org.apache.storm.disruptor$clojure_handler$reify__8796.onEvent(disruptor.clj:40) [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245] at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:451) [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245] at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245] at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245] at org.apache.storm.daemon.executor$fn__9362$fn__9375$fn__9428.invoke(executor.clj:853) [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245] at org.apache.storm.util$async_loop$fn__656.invoke(util.clj:484) [storm-core-1.0.1.2.5.0.0-1245.jar:1.0.1.2.5.0.0-1245] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77] Caused by: org.apache.storm.hive.common.HiveWriter$TxnBatchFailure: Failed acquiring Transaction Batch from EndPoint: {metaStoreUri='thrift://ambari.local:9083', database='default', table='stock_prices', partitionVals=[Marcin] } at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:264) ~[stormjar.jar:?] at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72) ~[stormjar.jar:?] ... 13 more Caused by: org.apache.hive.hcatalog.streaming.TransactionError: Unable to acquire lock on {metaStoreUri='thrift://ambari.local:9083', database='default', table='stock_prices', partitionVals=[Marcin] } at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:575) ~[stormjar.jar:?] at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:544) ~[stormjar.jar:?] at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:259) ~[stormjar.jar:?] at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72) ~[stormjar.jar:?] ... 13 more Caused by: org.apache.thrift.transport.TTransportException at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132) ~[stormjar.jar:?] at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) ~[stormjar.jar:?] at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378) ~[stormjar.jar:?] at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297) ~[stormjar.jar:?] at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204) ~[stormjar.jar:?] at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) ~[stormjar.jar:?] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:3781) ~[stormjar.jar:?] at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:3768) ~[stormjar.jar:?] at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1736) ~[stormjar.jar:?] at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransactionImpl(HiveEndPoint.java:570) ~[stormjar.jar:?] at org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.beginNextTransaction(HiveEndPoint.java:544) ~[stormjar.jar:?] at org.apache.storm.hive.common.HiveWriter.nextTxnBatch(HiveWriter.java:259) ~[stormjar.jar:?] at org.apache.storm.hive.common.HiveWriter.<init>(HiveWriter.java:72) ~[stormjar.jar:?] ... 13 more 2017-03-26 21:43:05.112 h.metastore [INFO] Trying to connect to metastore with URI thrift://ambari.local:9083 ... View more