question Re: kafka Data encryption at rest in Archives of Support Questions (Read Only)

kafka Data encryption at rest

sunile_manjee — Mon, 25 Jan 2016 13:17:13 GMT

I am looking for best practice with Kafka data encryption at rest. I am aware of custom solutions using AES for producer and consumer; however, looking for alternatives or best practice.

Re: kafka Data encryption at rest

aervits — Tue, 02 Feb 2016 23:20:06 GMT

@schintalapani

Re: kafka Data encryption at rest

sunile_manjee — Wed, 03 Feb 2016 00:13:29 GMT

we could in effect write a consumer and producer that can watch the folder directory on HDFS using maybe INotifiy and then pull that file off, if they have permissions it should decrypt while its collecting and then write the blob into Kafka. Then the Consumer can read from Kafka and write into HDFS again with TDE. There are more discussion here .

http://www.confluent.io/blog/apache-kafka-0.9-is-released

Re: kafka Data encryption at rest

jim_hoagland — Fri, 22 Jul 2016 04:10:18 GMT

An example of using RSA+AES: http://www.symantec.com/connect/blogs/end-end-encryption-though-kafka-our-proof-concept

Re: kafka Data encryption at rest

smartninja723 — Thu, 17 Nov 2016 00:16:49 GMT

Hi @Sunile Manjee , I am also looking for Encryption at rest for Kafka messages. But prefer not to pass on the messages to the HDFS. Consider our requirement more or less like "disk theft" which contains credit card transaction logs for 100s of credit card holders or military data.

Thanks in advance.