Thanks for your in-depth response, Andy! This is great; it certainly clarified the gray areas for me... it sounds like Option 5, then, is simply passing around encrypted data as the FlowFile content. If a given processor needs to access the content, it would employ an EncryptedRecordReader (or however you choose to build it... your layered approach sounds good, I agree that it'd be better than the 2*n implementations...). Since the content is decrypted and re-encrypted as part of the @OnTrigger phase, plaintext data would never be written to the Content Repository. It sounds like the attributes would still be unencrypted, so processors that deal with those (e.g. RouteOnAttribute) could still function. ... View more

Hello @Andy LoPresto, After reading here (and on other forum posts), it appears that there's a group of us who are seeking encryption of their "data at rest" (i.e. encryption of the flowfile, content, and provenance repositories). You all at Hortonworks have recently provided an encrypted provenance repository (thank you!), but if we also want the other two to be encrypted, it seems that we currently have five options: 1) Create our own encrypted versions of the pluggable writers/readers for the Content and Flowfile repositories. 2) Wait for you/Hortonworks to finish implementing encrypted versions of the pluggable writers/readers for the Content and Flowfile repositories. 3) Use an encrypted volume. (The OS / disk-driver handles the encryption and decryption transparently) 4) Use a VolatileContentRepository (at our own risk of data loss) 5) Create a ScriptedRecordReader and ScriptedRecordSetWriter (in lieu of the pending EncryptedRecordReader and EncryptedRecordSetWriter controller services). You alluded to #5 in your response to Alvin, above. My question is: How does #5 differ from #1? More specifically, when does NiFi write content and flowfile data to disk? When queues get full? Would I be correct to assume that #5 only works if we're processing record-oriented data exclusively? For example, if our flow were to assemble record-oriented from one or more relations (i.e. from sources other than an EncryptedRecordReader), then wouldn't there be a possibility of plaintext data written to disk before it becomes associated/identified as a "record"? ... View more