Hi I am trying to convert my Json to a Parquet file and then put in into an S3. My Json is 2 GB large. What I am doing is converting it to an Avro and then Parquet. However my Convert record processor is pending and doesn't seem to do any activity. Do you know why? Here is my Convert record Config: ... View more

I am having trouble with my dataflow using Nifi, Sure because I didn't understand how Wit/Notify Works. Here is my basic data flow: - I listed my s3 bucket containing zip files. - Extracted my Json file from the zip. Each Json is 2Gb up to 3 Gb. - I then split my json several times to avoid 'Out of memory' to obtain sing records FlowFiles(Here comes the trouble). - I want to flatten each flowfile - I want to merge my flattened FF to re obtain the original Json. I know I cannot do with 'Merge Content' as my json goes through several split processors. Can you please explain how to use the Wait/Notify? Does the wait automatically merges the flowfiles? Because I did not understand. I also looked at this post but it is still not clear. Thanks ... View more