Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Integrate Apache Nifi with encrypted data in Amazon SQS and S3

Integrate Apache Nifi with encrypted data in Amazon SQS and S3

Contributor

Currently i have created a process group which would get the events from SQS via getSQS processor and retrieve the S3 data from fetchS3Object processor.

But now data is encrypted before placing in S3 and my Nifi need to decrypt the file collected using an AWS key which would be changing every 3 month so it should refresh the key by making an API call to AWS KMS.

I didn't find any document to encounter this scenario. Please share some hint or processor which could handle this situation.

1 REPLY 1

Re: Integrate Apache Nifi with encrypted data in Amazon SQS and S3

Contributor

Adding few more details to the query as in the current scenario , the data is encrypted using AWS KMS-Managed Keys (SSE-KMS) only using client side encryption. The client (Nifi) need to download the encrypted object from Amazon S3 along with the cipher blob version of the data encryption key stored as object metadata. The client then sends the cipher blob to AWS KMS to get the plain-text version of the key so that it can decrypt the object data.

Please have a look to the option 1 of the below link:

https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html

Not sure if we can do this with the existing Nifi processor with some customization OR we need to create a new processor altogether.

Please suggest.

Don't have an account?
Coming from Hortonworks? Activate your account here