Hi, I am seeing some situations where I have two Hive SQL commands running concurrently and I'm getting a lost update.  I am running Hive 2.3.6 on EMR with hive.support.concurrency = true and I believe this shouldn't be happening based on what I understand about Hive table locking.  (I am not using ACID transactions but the table locking should still prevent lost update as far as I know;)   Specifically I have a "load data" statement loading data into table T from an S3 location.  I have an "insert overwrite T select * from T" table running concurrently from another Hive connection that deletes some rows from T but should not be affecting rows from the load data statement.  I am seeing that the data from the load data statement disappears after the insert overwrite finishes.  My understanding is that the load data and insert overwrite should create an exclusive table lock on T so they should allow each other to finish before reading or writing data from T.  (I checked this using "show locks" and they do definitely create an exclusive lock.)     Has anyone seen this issue before and are there any Hive settings I can try changing to prevent this behavior? ... View more

When running Hive queries from NiFi 1.10 that contain backticks, I get the following error:   2019-12-23 15:17:00,191 WARN [Timer-Driven Process Thread-2] o.a.nifi.processors.hive.SelectHiveQL SelectHiveQL[id=075b3a1e-7632-1647-68d9-338231b5921b] Failed to parse query: select 1 as `asdf` due to java.lang.NullPointerException:   I thought Hive queries allowed backticks to escape column names, so I'm not sure why NiFi can't parse this.  The actual query runs fine on the Hive server, and I get a valid flow file with the query results, but it still raises a red NiFi bulletin (which we would prefer not to have if there is not a real problem).   The Hive server running the query is the one that comes with HDP 2.5. ... View more

We have an external hive table (let's call it example_table) created on top of an orc file in Hadoop. Doing a simple select query such as: select * from example_table works fine, but select * from example_table order by id returns lots of extra rows that look like corrupt data. We have seen cases where this returns 10x the number of rows as the query without the order by clause. If its relevant, the orc files were created by Spark and we are using Hive 1.2.1 on HDP 2.5. I suspect this may have something to do with the fact that the orc data has fields of string datatype that contain newlines. Is this a known bug and/or are there any Hive settings we could try changing to fix this? UPDATE: Here is a Hive script we came up with to reproduce the problem: create table default.test_errors(c1 string) stored as orc; --Make sure to include newlines here with CTE as (select 'a b c' as c1) insert into default.test_errors select c1 from CTE; --Returns 1 row select * from default.test_errors; --Returns 3 rows select * from default.test_errors order by c1; ... View more