Running on HDP2.6. This query seemed to be working fine (a simple create table operation; just doing a select works without probems), but now Hive LLAP returns the following error when trying to create a table: Error running query: java.lang.AssertionError: Unexpected type UNEXPECTED from: 2019-08-06T17:29:20,701 WARN  [HiveServer2-Handler-Pool: Thread-1858]: thrift.ThriftCLIService (ThriftCLIService.java:ExecuteStatement(516)) - Error executing statement: org.apache.hive.service.cli.HiveSQLException: Error running query: java.lang.AssertionError: Unexpected type UNEXPECTED         at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:225) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.cli.operation.Operation.run(Operation.java:312) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:508) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:495) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:309) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:506) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:599) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112]         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] Caused by: java.lang.AssertionError: Unexpected type UNEXPECTED         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.fixUpCtasAndInsertAfterCbo(CalcitePlanner.java:952) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:378) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11167) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:290) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:257) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:455) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1197) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1184) ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:191) ~[hive-service-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]         ... 15 more This is the first time I'm seeing such a error. What does it even link to? Any known solutions? EDIT: Important comment: this is 100% related to masking through Apache Ranger of some columns in the used tables. However, both the imposed masking AND the ability to create new tables using the initial masked one is important. Any known workaround? ... View more

Hi all, After setting up a fresh kerberized HDP 3.1 cluster with Hive LLAP, Spark2 and Livy, we're having trouble connecting to Hive's database through Livy. Pyspark from shell works without the problem, but something breaks when using Livy. 1. Livy settings are Ambari default, with additionally specified jars and pyfiles for the HWC connector, spark.sql.hive.hiveserver2.jdbc.url and spark.security.credentials.hiveserver2.enabled true. These are enough for pyspark shell to work without problems. 2. Connection is made through the latest HWC connector described here, since apparantly this is the only one that works for Hive 3 and Spark2. problem: 1. When spark.master is set to yarn client mode (See for example the comment here), the connector appends a principal "hive/_HOST@DOMAIN" and the connection returns GSS error - failing to find any Kerberos tgt (although, the ticket is there and livy has access to the hiveserver2). 2. When spark.master is set to yarn cluster mode, ";auth=delegationToken" is appended to the connection, where the error follows that "PLAIN" connection is made, where a kerberized one is expected. Notes: tried various settings -- zookeeper jdbc links vs direct through port 10500, hive.doAs = true vs false, various principals, but nothing works. Note2: everything works fine when connecting both through beeline (to hive at 10500 port) and through pyspark shell. Note3: HWC Connection snippet (from examples): from pyspark_llap import HiveWarehouseSession hive = HiveWarehouseSession.session(spark).build() hive.showDatabases().show(100) Any ideas? Feel like some setting on Livy is missing, especially weird seeing that "failed to find any Kerberos tgt" - where is it looking for it and why doesn't it see the ticker from "kinit"? @Geoffrey Shelton Okot @Hyukjin Kwon @Eric Wohlstadter ... View more

Hello all, I have set up a HDP 3 cluster with Hive running on LLAP-Interactive. However, when connecting through Hiveserver2 port (10500 default) (or if through zookeeper, then through Hiveserver2-Interactve), any query that is being run is going through YARN/TEZ containers. That is, if I check at the LLAP Dashboard (http://datanode:10502), I can see that there are 0 active/passive sessions. With my experience from HDP2.6, there should be every query listed. What can be the problem? LLAP is enabled with the default settings from Ambari 2.7, with the following settings: hive.execution.mode = llap #at hive-interactive-site hive.llap.execution.mode = all #same with the default "only" hive.execution.mode = container #at hive-site Is this the default behaviour? Currently I'm worried that there are two different "hive.execution.mode" settings specified, so will try playing with them. But other than that, I'm not sure what further settings to check. @Geoffrey Shelton Okot ... View more

Hello all, after fresh kerberization of Ambari 2.7.3 / HDP 3 cluster, the HDFS namenode isn't able to start because the hdfs user can't talk to the webhdfs. The following error is returned: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed) It is not only from ambari: I can recreate this error from a simple curl call from hdfs user: su - hdfs curl --negotiate -u : http://datanode:50070/webhdfs/v1/tmp?op=GETFILESTATUS Which returns </head> <body><h2>HTTP ERROR 403</h2> <p>Problem accessing /webhdfs/v1/tmp. Reason: <pre> GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)</pre></p> </body> </html> Overall permission for this user should be in tact, since I'm able to run hdfs operations from shell and kinit without problems. What could be the problem? I've tried recreating keytabs several times, and fiddling with ACL settings on the config, but nothing works. What principal is WEBHDFS expecting? The same results are when I'm trying accessing it with HTTP/host@EXAMPLE.COM principal. NB: I'll add that there's nothing fancy in the HDFS settings, mainly stock/default config. NB2: I will add, that I've added all possible encryption types to krb5.conf as I could find, but none if these helped: default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal permitted_enctypes = aes256-cts-hmac-sha1-96 aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal @Geoffrey Shelton Okot ... View more

We will be setting up a HDP 3 cluster, and the idea is to split 1xRAID1 and 2xHDD through 2 virtual machines. 1xRAID1 would handle the OS and local user files, while each of the 2xHDD's would be used for hdfs /grid/. My question: is 2xHDD enough to ensure file redundancy? I've read that Hadoop 3 does not require 3 replicas as it did in Hadoop 2, however, in case of a drive failure, would it still ensure that the files on the remaining HDD are enough for a full recovery? Or would you advice partitioning some of the 1xRAID1 for a third /grid/ drive for the hdfs? In this case it seems to introduce a lot of overhead due to RAID. What is the minimum amount of HDD's required for redundant hdfs operation? ... View more

We will be setting up a HDP 3 cluster, and the idea is to split 1xRAID1 and 2xHDD through 2 virtual machines. 1xRAID1 would handle the OS and local user files, while each of the 2xHDD's would be used for hdfs /grid/. My question: is 2xHDD enough to ensure file redundancy? I've read that Hadoop 3 does not require 3 replicas as it did in Hadoop 2, however, in case of a drive failure, would it still ensure that the files on the remaining HDD are enough for a full recovery? Or would you advice partitioning some of the 1xRAID1 for a third /grid/ drive for the hdfs? In this case it seems to introduce a lot of overhead due to RAID. What is the minimum amount of HDD's required for redundant hdfs operation? Big thanks! ... View more

Half of the resources on YARN in our cluster is dedicated to Hive LLAP, while the other half is left free for other various tasks, such as Spark or mapreduce containers. However, there are times when no other jobs are running, and LLAP is loaded to its fullest. Is there a setting allowing for dynamic allocation of all resources? That is, in such cases, would it be possible to allocate all, or i.e., 80% of the unused space for LLAP, and return it back to normal after finishing the jobs? As far as I know now, only way to do that would require a LLAP or YARN restart, which is not at all preferred. ... View more

Consider this example table: create table config.test (param string, value string) partitioned by (jobid string) clustered by (param) into 3 buckets stored as orcfile tblproperties('transactional'='true'); I want this to be an ACID table, don't really need the partitioning and clustering, but creating this following all the guides. Assume I want to update some values: insert into config.test partition(jobid) values ('1', '2', '3'); insert into config.test partition(jobid) values ('2', '3', '4'); insert into config.test partition(jobid) values ('4', '4', '4'); update config.test set value = '99' where jobid = '4' and param = '4'; Inserts work fine, Update returns the following "java.lang.NegativeArraySizeException" error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Reducer 2, vertexId=vertex_1535365185102_0002_140_01, diagnostics=[Task failed, taskId=task_1535365185102_0002_140_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1535365185102_0002_140_01_000000_0:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:218) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:442) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:366) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:556) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:508) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:213) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:188) ... 15 more Caused by: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:357) ... 21 more ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1535365185102_0002_140_01_000000_1:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:218) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:442) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:366) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:556) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:508) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:213) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:188) ... 15 more Caused by: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:357) ... 21 more ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1535365185102_0002_140_01_000000_2:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:218) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:442) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:366) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:556) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:508) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:213) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:188) ... 15 more Caused by: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:357) ... 21 more ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1535365185102_0002_140_01_000000_3:java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:218) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:172) at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73) at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61) at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37) at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36) at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:442) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:366) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:556) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:508) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376) at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.init(ReduceRecordProcessor.java:213) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:188) ... 15 more Caused by: java.lang.NegativeArraySizeException at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:357) ... 21 more ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1535365185102_0002_140_01 [Reducer 2] killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0 What could be the problem? I'm working on a perfectly running cluster and we do have some ACID tables working. But I can't figure out why this simple example doesn't work? What works: The UPDATE works when "jobid = '4'" is NOT in the WHERE clause, but it updates too many rows. Why is this a problem? I'm updating a "param" field which is neither bucketed nor a partition field, those two fields are only i the WHERE clause. If this is really because the "jobid" is a partitioning field, I could create dummy/mirror fields that would have the same info but not be partition keys, but that does seem redundant. Any Ideas? ... View more

When running a large Hive query that takes up the whole cluster (all LLAP executors are busy), simultaneously trying to run even a simplest query takes a lot of time, probably because the process is waiting for a free executor to work with. Is it possible to restrict/manage the queues in a similar manner as does YARN, i.e., so that there would always be at least one free executor for a quick query, or that the tasks from concurrent queries would be pushed in front of the line to guarantee some minimal amount of exposure to the executors these tasks would get? ... View more

We often drop and recreate Hive tables with new/recalculated data under same table names. However, after longer use I've found that fetching the data from these tables became broken: i.e., count(*) from that table estimates 2k rows, while export using beeline to csv returns >100k rows, most of which are NULL's at the end of the file. Usually, the problem does not repeat itself when trying to recreate the same tables under different names, but I don't think it's a good solution to create hundreds of tables ala table1, ..., table506 to be able to work with the file.. So my question: is there a fix for this? How can we be sure that dropping the old tables also drops the data? ... View more

We often drop and recreate Hive tables with new/recalculated data under same table names. However, after longer use I've found that fetching the data from these tables became broken: i.e., count(*) from that table estimates 2k rows, while export using beeline to csv returns >100k rows, most of which are NULL's at the end of the file. Usually, the problem does not repeat itself when trying to recreate the same tables under different names, but I don't think it's a good solution to create hundreds of tables ala table1, ..., table506 to be able to work with the file.. So my question: is there a fix for this? How can we be sure that dropping the old tables also drops the data? ... View more