I'm not a Hive expert but I did author the original PutHive3Streaming processor for NiFi. My recommendation is setting Records Per Transaction greater than the number of records in a FlowFile (unless we are talking about super-huge files), and transactions per batch to 1. This makes the transaction semantics similar to how NiFi FlowFile sessions work (rollback, failure, success, e.g.). If the number of records is huge and is causing throughput problems, try dividing that number by 100 and making transactions per batch 100. When you multiply the two numbers together it should be greater than the total number of records in the FlowFile in order to avoid overhead with the Hive Metastore by requesting a large number of batches/transactions. ... View more

The operation to add an attribute to a FlowFile is on the ProcessSession object not the FlowFile itself (so the session can keep track of changes).  Try the following instead:   session.putAttribute(destFlowFile, , "logMsg", "Testing Msg") session.putAllAttributes(destFlowFile, backupAttributes) ... View more

You can refer to the "Fields" output field explicitly instead of needing another shift:   [ { "operation": "shift", "spec": { "*": { "CUST_AC_NO": "[&1].ExternalSystemIdentifier", "BRANCH_CODE": "[&1].Fields.FLD0001", "CUST_NO": "[&1].Fields.FLD0002", "AC_DESC": "[&1].Fields.FLD0003" } } }, { "operation": "default", "spec": { "*": { "InstitutionId": "1" } } } ] ... View more