- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
kafka Data encryption at rest
Created 01-25-2016 05:17 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am looking for best practice with Kafka data encryption at rest. I am aware of custom solutions using AES for producer and consumer; however, looking for alternatives or best practice.
Created 02-02-2016 04:13 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
we could in effect write a consumer and producer that can watch the folder directory on HDFS using maybe INotifiy and then pull that file off, if they have permissions it should decrypt while its collecting and then write the blob into Kafka. Then the Consumer can read from Kafka and write into HDFS again with TDE. There are more discussion here .
Created 02-02-2016 03:20 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created 02-02-2016 04:13 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
we could in effect write a consumer and producer that can watch the folder directory on HDFS using maybe INotifiy and then pull that file off, if they have permissions it should decrypt while its collecting and then write the blob into Kafka. Then the Consumer can read from Kafka and write into HDFS again with TDE. There are more discussion here .
Created 11-16-2016 04:16 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @Sunile Manjee , I am also looking for Encryption at rest for Kafka messages. But prefer not to pass on the messages to the HDFS. Consider our requirement more or less like "disk theft" which contains credit card transaction logs for 100s of credit card holders or military data.
Thanks in advance.
Created 07-21-2016 09:10 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
An example of using RSA+AES: http://www.symantec.com/connect/blogs/end-end-encryption-though-kafka-our-proof-concept
![](/skins/images/C3EF05C688F0C29C1D3298241F61C2B3/responsive_peak/images/icon_anonymous_message.png)