I hava use case to split a large file to 0.5 GB files,I was able to slit the file to 0.5 GB but thw split is not record oriented.I see records splitting in the middle.
I am using split text using Split Text Processor,I have attached the screnshot below.
What am I doing wrong? Can anyone direct me to examples / templates?
i think for this case we need to split the records one line each by using SplitRecord/SplitText processor.
Then Using MergeContent processor we can do 500 MB splits by using this way we are not going to have splitting records in between.
1.SplitRecord/SplitText //split the flowfile 1 line each 2.MergeRecord/MergeContent //to get 500MB filesize
To force merge flowfiles use MaxBigAge property like 30 mins..etc.
In case if you are using Record oriented processors we need to define Record Writer/Reader with avro schema to read/write the flowfile.
Refer to this link for more details regards to merge content processor.