Member since
12-09-2015
106
Posts
40
Kudos Received
20
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2944 | 12-26-2018 08:07 PM | |
2587 | 08-17-2018 06:12 PM | |
1346 | 08-09-2018 08:35 PM | |
12060 | 01-03-2018 12:31 AM | |
988 | 11-07-2017 05:53 PM |
06-16-2017
09:05 PM
Only managed tables can be transactional - not external.
... View more
04-19-2017
07:43 PM
2 Kudos
The short answer is you can ignore these. When you are using Hive Streaming Ingest (https://cwiki.apache.org/confluence/display/Hive/Streaming+Data+Ingest) which is used by Storm/Flume/NiFi, Hive creates these file for it's internal housekeeping for maintaining transactional consistency. They should normally be removed as soon as the TransactionBatch is closed (usually once transaction Y of delta_X_Y/ finishes). The flush_legth file may remain around if the Writer process crashes before TransactionBatch is closed. They will eventually be cleaned by the Compactor process.
... View more
03-28-2017
04:10 PM
This is not supported yet
... View more
03-21-2017
02:18 AM
1 Kudo
I'm a developer of this system - I believe this is safe (though I've not ran a full test to prove it)
... View more
03-16-2017
03:51 PM
it may be simpler to modify your table to add a primary key to NEXT_COMPACTION_QUEUE - any synthetic PK will work.
... View more
03-14-2017
06:29 PM
This is not streaming, but SQL merge command may be useful here: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Merge
... View more
03-06-2017
06:38 PM
Order by produces a total order for the result set. Sort By sorts the output of each reducer so in general this will not produce the same answer
... View more
02-23-2017
07:33 PM
2 Kudos
PutHiveStreaming relies on Streaming API which has 2 relevant concepts: number of events per transaction and number of transactions per batch. Generally, the more events you write per transaction the faster the ingest. I don't see the 1st of these properties in the NiFi doc referenced above - perhaps there is some NiFi specific property that controls this.
... View more
02-16-2017
05:08 PM
This _tmp file should be created in the Mapper of the compaction job. Is there anything about it in the job logs?
... View more