Split Large Json file into multiple files with a specified number of records

[#1]

Hi, 

I am a newbie to Nifi and would like some guidance please. 

We want to split a large Json file into multiple files with a specified number of records. I am able to split a file into individual records using SplitJson and the Json Path Expression set as $.<objectthatcontainsthearray>.* I have also added an UpdateAttribute Processor with filename set to ${filename}_${fragment.index} so that we have the sequence of the files as order is important.

However, we might want to have say a 100,000 records split into 100 files of 1000 records each . What is the easiest way to do this ?

Thanks very much in advance

[#2]

@pxm 

You'll want to use the SplitRecord processor to accomplish what you are looking to do.

Configure it with a RecordReader and RecordWriter  for you input data and desired output data formats along with the desired 1000 records per split.

Hope this helps,

Matt