Member since
11-12-2018
189
Posts
177
Kudos Received
32
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
513 | 04-26-2024 02:20 AM | |
667 | 04-18-2024 12:35 PM | |
3228 | 08-05-2022 10:44 PM | |
2938 | 07-30-2022 04:37 PM | |
6424 | 07-29-2022 07:50 PM |
12-26-2018
07:48 AM
3 Kudos
@Jack From your above output I can see it's MANAGED_TABLE. If the table has TBLPROPERTIES ("auto.purge"="true") the previous data of the table is not moved to Trash when INSERT OVERWRITE query is run against the table. This functionality is applicable only for managed tables and is turned off when "auto.purge" property is unset or set to false. For more detail HIVE-15880
... View more
12-26-2018
07:36 AM
3 Kudos
@Jack Please can you share output of below command. It is hive 3.x or hive 2.x ? DESCRIBE FORMATTED <table_name>
... View more
12-26-2018
06:45 AM
3 Kudos
@Ilia K You can manually configure your memory allocation settings based on the hardware specifications. Please refer below links for more tuning up RAM in yarn configuration. https://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/ https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.5/bk_command-line-installation/content/determine-hdp-memory-config.htm
... View more
12-26-2018
06:20 AM
3 Kudos
@Jack Seems your tables are not partitions. When you try to “INSERT OVERWRITE” to a partition of an external table under existing HDFS directory, depending on whether the partition definition already exists in the HIVE metastore or not, Hive will behave differently: Suppose if partition definition does not exist, it will not try to guess where the target partition directories are either static or dynamic partitions, so it will not be able to delete existing files/data under those partitions. So that will retained as historical data of HDFS like your case. But if partition definition does exist, it will attempt to remove all files/data under the target partition directory before writing new data into those directories. Reference here Please accept the answer you found most useful.
... View more
12-26-2018
06:02 AM
3 Kudos
@Michael Bronson As for automatic clean up not getting triggered, well it may be due to (or at-least related to) this unresolved bug reported on Yarn. https://issues.apache.org/jira/browse/YARN-4540
... View more
12-25-2018
07:41 AM
2 Kudos
Please can you attach full error logs to debug further, if issue persist..
... View more
12-25-2018
07:40 AM
3 Kudos
As mentioned in the article, did you modified KMS URL in Ambari and removed tailing "/' ? which resolved your issue or still facing same..?
... View more
12-24-2018
11:03 AM
3 Kudos
@Sundara Palanki Please can you check below article, seems you are facing similar issue. https://community.hortonworks.com/content/supportkb/150691/yarn-failed-to-renew-token-kind-timeline-delegatio.html
... View more
12-06-2018
06:03 AM
3 Kudos
@Junfeng Chen That's true, above property for local filesystem. For hdfs could you try to use Append instead of Overwrite ? But problem in this, we need to delete files manually from the temp directory.
... View more