Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to turn off snappy compression in Nifi PutKafka processor

Re: Unable to turn off snappy compression in Nifi PutKafka processor

After digging into the code, it does not seem possible. The Snappy compression is hard-coded. I have submitted a PR to allow users setting this compression codec (https://github.com/apache/nifi/pull/409). You can give it a try if you want.

Highlighted

Re: Unable to turn off snappy compression in Nifi PutKafka processor

Contributor

Saw that - thanks. Unfortunately being able to apply it is a bit beyond me.

Re: Unable to turn off snappy compression in Nifi PutKafka processor

Super Guru

So you see the dumped file with Compression on? Did you use the command line tools to check it: http://www.michael-noll.com/blog/2013/03/17/reading-and-writing-avro-files-from-the-command-line/

What kind of Kafka consumer?

Can you export your dataflow as a template and post it?

Can you view the file through another HDFS client?

Re: Unable to turn off snappy compression in Nifi PutKafka processor

Super Guru

Does HDFS in PutHDFS have compression set?

Did you look at this article: https://community.hortonworks.com/articles/28341/converting-csv-to-avro-with-apache-nifi.html