Looking to encrypt data while copying data into database

[#1]

Hi Team,

I'm new to NiFi looking for some help as I'm looking for a processor to read the Avro output and encrypt the data and transform the encrypted data into the database table. Extracting data using ExceuteSQL processor Is there any processor that I can use to achieve this?

[#2]

@Vas_R Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @cotopaul @SAMSAL @MattWho  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

@Vas_R 

There is not a lot of detail in your use case here.
Where are you reading the Avro from?  Did you already ingest Avro content from source into a FlowFile within NiFi?  Guessing you are using ExecuteSQL as first processor in your flow to read content from a source DB?
Are you looking to encrypt the entire Avro or only a portion of the Avro record content?
The more detail you provide, the better the chances are that a community member will be able to help you.

Thanks,
Matt

[#4]

@MattWho 
Yes, I'm reading the data from a source Database table and the content is written to an Avro FlowFile. Now I want to encrypt all the columns I'm getting in Avro but not the entire Flowfile (If I use EncryptContent processor it's encrypting the entire flow file which I'm not interested in) as I want to put these encrypted records into the destination table. 

Thanks,

Vas_R

[#5]

@Vas_R,

How would you like to encrypt that data, as I see you want to do this on each column and each row? Have you something in mind or are you searching for something built in NiFi directly? If you want something directly in NiFi, I do not think that you will find anything that specific.

You could try exporting your relevant data into attributes and use something like CryptographicHashAttribute Processor in order to use a hash algorithm on those attributes. Next, you can use an AttributesToCSV/AttributesToJSON Processor and generate a new FlowFile with the hashed data. If CSV or JSON is not the best format for you, you can add an extra ConvertRecord and transform your data in whatever format you want. But be careful as this solution will require many resources if you are playing with large amounts of data.

Another solution would be to find an encryption algorithm and implement it within a Script. Add an ExecuteStreamCommand Processor, which will read the AVRO File, perform the encryption and write out the newly generated AVRO File.

[#6]

@Vas_R Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks