The short answer is you can ignore these. When you are using Hive Streaming Ingest (https://cwiki.apache.org/confluence/display/Hive/Streaming+Data+Ingest) which is used by Storm/Flume/NiFi, Hive creates these file for it's internal housekeeping for maintaining transactional consistency. They should normally be removed as soon as the TransactionBatch is closed (usually once transaction Y of delta_X_Y/ finishes). The flush_legth file may remain around if the Writer process crashes before TransactionBatch is closed. They will eventually be cleaned by the Compactor process. ... View more

This is not supported yet ... View more

it may be simpler to modify your table to add a primary key to NEXT_COMPACTION_QUEUE - any synthetic PK will work. ... View more

PutHiveStreaming relies on Streaming API which has 2 relevant concepts: number of events per transaction and number of transactions per batch. Generally, the more events you write per transaction the faster the ingest. I don't see the 1st of these properties in the NiFi doc referenced above - perhaps there is some NiFi specific property that controls this. ... View more