Support Questions

Find answers, ask questions, and share your expertise

Nifi Convert Json to Parquet

avatar
Explorer

Hi I am trying to convert my Json to a Parquet file and then put in into an S3.

My Json is 2 GB large.

What I am doing is converting it to an Avro and then Parquet. However my Convert record processor is pending and doesn't seem to do any activity. Do you know why?

109241-1559641746182.png


Here is my Convert record Config:

109251-1559641782523.png

1 REPLY 1

avatar
Master Guru

@3nomis

ConvertRecord processor is still running as we can see running threads as one at top right corner of the processor.

If it is running for ever (or) you are not convinced with performance then you can try splitting the 2GB file into smaller chunks by using SplitRecord processor then use MergeRecord processor to merge them back by using Defragment as merge strategy.