Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Multiple JSON to JSON Array Output file

Explorer

Hello All, 

 

I am exporting data from Mongo DB. In mongo db collection i have round 2000+ objects. These are read using GetMongo. Further processed by EvaluateJSON and then SplitJSON. 

 

Now post that i want to write all the JSON as single JSON array in a single JSON file. but not able to achieve it. 

 

Tried calling UpdateAttribute updating output file name and then call to Putfile. but putfile is writing only one record and then fails with "duplicate file or same file exist " error. 

1. Can anyone direct me on how to get these multiple JSON in to a single JSON array. 1. 

2. Fetch only 200 records by GetMongo so that each file is only 200 JSON Array. 

 

Please guide 

 

Thxs

Rups

 

 

  

2 REPLIES 2

@Rups Your GetMongo should have a Limit, Batch Size, and Results Per FlowFile.   Investigate using these to get the # of results you want per FlowFile.    If you do not see these configurations investigate nifi 1.9 or greater.  

 

I would recommend NOT using SplitJson to split results.

 

For putFile you need to update the filename to something unique per FlowFile to avoid the duplicate filename.  For example something like:

 

filename = ${filename:append(now():format('yyyy-mm-dd hh:mm:ss'))}

 

or use the FlowFile's Filename:

 

filename = ${Filename}

Explorer

Above solution has already been tried. Here with this I get individual file with one JSON in each. 

Whereas i need single file with multiple JSON in that. Is that possible ?

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.