Support Questions
Find answers, ask questions, and share your expertise

Option to acces AWS3 Secret without exposing keys


@MattWho  : Currently i am using PutS3 Object to transfer files from source to destination. I am passing secret key in property file and accessing via Credential File configuration. But our security team is not allowing this process in production as it is exposed for others to access. Can you suggest any other  approach to access these keys. Appreciate quick response as we have production deployment in < 5 days. 


Re: Option to acces AWS3 Secret without exposing keys

Master Guru



The Credentials File you created should be owned and only accessible by the NiFi service user.  Since all components in NiFi are executed by the NiFi service user, technically anyone who has access to the NiFi UI and knows where this credentials file exists on the server would be able to use it in a new processor they add to canvas.

Above being said, you should be setting up proper authorization policies on your NiFi to limit what users can actually view the configuration of your putS3Object processor.  If a user cannot view its configuration, that user will not know the bucket or location of your credentials file on disk.  Or you configure the access and secret key directly in processor so it never in a file on disk where unauthorized users could find it and use it.


You may also consider using the AWSCredentialsProviderControllerService as well and limited access to that controller service to only select users.  Then if others unauthorized users add S3 processors to the canvas they would not be able to select that AWSCredentialsProviderControllerService.


You can further protect  your access key and secret key by not using the credentials file and instead enter  those values directly in the corresponding properties in the processor or AWS controller service.  Then they do not exist on disk anywhere for a user to find and use. The advantage to using the AWS controller service is that you can still give users access to the processor but block all but specific users access to the AWS controller service.  This allows users to operate the processor, look at data provenance for that processor, etc while still not having access to view the secret and access keys in the referenced AWS controller service.


Hope this helps,