Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Can I store more than one record in a parquet file using ConvertAvroToParquet?

avatar
Contributor

I have created a flow to read json messages from kafka, and covert them to avro so that I can use ConvertAvroToParquet. My issue is that I can only store one avro message in a parquet file. Is there away to store multiple records within a parquet file using ConvertAvroToParquet?

1 ACCEPTED SOLUTION

avatar
Master Guru

You can use MergeContent or MergeRecord for this, it can take flow files each with a single record and combine them together to make a flow file containing many Avro records, then you can use ConvertAvroToParquet or PutParquet.

View solution in original post

1 REPLY 1

avatar
Master Guru

You can use MergeContent or MergeRecord for this, it can take flow files each with a single record and combine them together to make a flow file containing many Avro records, then you can use ConvertAvroToParquet or PutParquet.