Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi data encryption and PCI compliance

Solved Go to solution
Highlighted

NiFi data encryption and PCI compliance

Explorer

How does NiFi support data protection and encryption for PCI compliance? We are looking to ingest credit card transactions data which contains sensitive and PII information.

Option 1: encrypt data at source before sending to NiFi - how does NiFi handle encrypted data for making attribute-based routing decisions?

Option 2: use NiFi to ingest raw transaction data from a secure source -> use NiFi EncryptContent processor to encrypt the content of the flow file. How does it get stored in the content and flow file repositories? Is the data encrypted when stored on disk?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NiFi data encryption and PCI compliance

Currently, NiFi supports encrypting/decrypting data through the EncryptContent processor, but the pre/post state of the data would still be stored in plaintext in the content repository. In general, transparent disk encryption/OS-level data encryption is recommended in conjunction with strict OS-level/POSIX access controls.

There is a current effort to provide encrypted implementations of the flowfile (attribute), content, and provenance repositories.

As Dan mentioned, a combination of encrypted payload and plaintext metadata for routing can work very well if the payload does not need to be processed/transformed inside NiFi.

View solution in original post

2 REPLIES 2
Highlighted

Re: NiFi data encryption and PCI compliance

  1. NiFi would have to decrypt any encrypted data before it can be pulled into an attribute for decision making. This obviously assumes that all the data is encrypted, and you are not providing a payload to NiFI combining unencrypted meta-data and the encrypted payload, which works very well.
  2. You could use EncryptContent or a 3rd party protection service to encrypt the data within NiFi, this has the benefit of the action being included in the NiFi Provenance record. The Data would appear in the NiFi Content repository, so you would protect this, typically on an encrypted volume. The other repositories would not have sensitive data in them unless you copied it there, such as using ExtractText to write the unencrypted CC# to an Attribute.
  3. Bonus option: Use a tool which provides format preserving encryption to allow routing decisions without exposing sensitive data, either in or before NiFi.
Highlighted

Re: NiFi data encryption and PCI compliance

Currently, NiFi supports encrypting/decrypting data through the EncryptContent processor, but the pre/post state of the data would still be stored in plaintext in the content repository. In general, transparent disk encryption/OS-level data encryption is recommended in conjunction with strict OS-level/POSIX access controls.

There is a current effort to provide encrypted implementations of the flowfile (attribute), content, and provenance repositories.

As Dan mentioned, a combination of encrypted payload and plaintext metadata for routing can work very well if the payload does not need to be processed/transformed inside NiFi.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here