Member since
12-09-2015
106
Posts
40
Kudos Received
20
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3060 | 12-26-2018 08:07 PM | |
2782 | 08-17-2018 06:12 PM | |
1480 | 08-09-2018 08:35 PM | |
12511 | 01-03-2018 12:31 AM | |
1058 | 11-07-2017 05:53 PM |
04-11-2020
09:16 PM
I was working on something unrelated, but I hit this same error, detailed the issue in Jira, and have proposed a workaround. The issue is that there is a feature in Hive called the REGEX Column Specification. IMHO this feature was ill conceived and is not standard SQL. It should be removed from Hive and this issue is yet another reason why. That's what I was working on when I hit this issue. When Hive looks at the table name surrounded by back ticks, it looks at that string and determines that it is a Regex. When Hive looks at the table name surrounded by quotes, it looks at that string and determines that it is a Table Name. The basic rule it uses is "most anything ASCII surrounded by back ticks is a Regex." However, when Hive sees the quotes, it sees the string as a table name. Using quotes (and technically back ticks too, but that's clearly broken) around table names can be allowed/disallowed with a feature in Hive called "hive.support.quoted.identifiers". This feature is enabled in the user's HS2 session by default. However, when performing masking, it is a multi step process: The query is parsed by HS2 The masking is applied The query is parsed again by HS2 The first parsing attempt respects the hive.support.quoted.identifiers configuration and allows a query with quotes to be parsed. However, the masking code does not pass this configuration information to the parser on the second attempt. And oddly enough, if the configuration information is not passed along, the parser will consider this feature to be disabled. So, it's actually on the second pass that it fails because the parser rejects the quotes. For the record, I hit this issue when I removed the Regex feature, because it forced all quoted strings to be considered table names (and subjected to this feature being enabled/disabled) instead of sneaking by as being considered a Regex. All the masking unit tests failed. https://issues.apache.org/jira/browse/HIVE-23182 https://issues.apache.org/jira/browse/HIVE-23176
... View more
08-20-2018
04:15 PM
3 Kudos
hive.merge.cardinality.check=false is a bad idea. The logic controlled by this property checks if the ON clause of your Merge statement is such that more than 1 row from source side matches the same row from target side (which only happens in WHEN MATCHED clause). Logically what this means is that the query is asking the system to update 1 existing row in target in 2 (or more) different ways. This check is actually part of SQL standard definition of how Merge should work. You either need examine your data or the ON clause but disabling this check, when it throws a cardinality_violation error, may lead to data corruption later.
... View more
04-11-2018
07:30 PM
1 Kudo
hive.support.concurrency property enables locking. When a queries is shutdown its locks should be released immediately. When dies abruptly it may leave locks behind. These will be cleaned up by a background process running from a standalone Hive metastore process. This process will consider locks abandoned if they have not heartbeated for (by default) 5 minutes. Metastore logfile should have entries from AcidHouseKeeperService - that is the clean up process.
... View more
11-15-2017
07:28 AM
Hi all! Where is I
can find information about limitation Hadoop Distcp ? (transactional / non-transactional etc )
... View more
03-19-2019
07:12 AM
Hello @Constantin Stanca , Where do we set these configuration variables for them to take effect?
... View more
03-01-2017
04:41 PM
you can use left join to get this Step5 can be replace with create table z_test2 as select a.LastName, case when b.FirstName is null then a.FirstName else b.FirstName end as firstname, a.address, a.city from z_test1 a left outer join z_test b on a.LastName = b.LastName
... View more
01-27-2017
05:45 PM
Thanks @Eugene Koifman
Can you point to an updated complete and updated documentation/book on Hive features ? (ACID, LLAP, etc)
... View more
06-14-2016
06:43 AM
colud you please type the command example to run a compaction for my table adn bucket in ORC format?? CREATE EXTERNAL TABLE IF NOT EXISTS canal(
CODTF string,
CANAL string,
FECHAOPRCNF string,
FRECUENCIA int,
CODNRBEENF string
)
COMMENT 'A buckered table canal by CODNRBEENF'
CLUSTERED BY (CODNRBEENF)
SORTED BY (FECHAOPRCNF)
INTO 60 BUCKETS
stored as ORC
LOCATION '/RSI/tables/logs/canal'
TBLPROPERTIES ("immutable"="false", "orc.create.index"="true","orc.compress"="ZLIB", "orc.stripe.size"="67108864","orc.row.index.stride"="25000");
... View more
10-17-2017
06:39 AM
For me this setting is disabled .I can not make any changes.Can you please let me know how to on the ACID properties?
... View more