Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 08-06-2019 07:52 PM
I have JSON input of the following format:
{
"Id": 1000000,
"ReportName": TestReport,
"Results": [{
"Id": 1,
"ResultId": "1000000-0",
"Query": {
"Id": 001,
"Name": "TestQuery0",
}
}, {
"Id": 2,
"ResultId": "1000000-1",
"Query": {
"Id": 002,
"Name": "TestQuery1",
}
}]
}These file can become quite large depending on the number of Results in the Report and I was hoping to convert the single Flowfile to multiple records for processing. However, due to the format of the JSON a SplitRecord will result in one record per split. There is one report per FlowFile and therefore only 1 root level element.
I am looking for a method or strategy to split the Flowfile into smaller Records while still maintaining the cohesiveness of the report in the end when it put in HDFS.
Current Strategy:
Any advice would be appreciated! Thank you
Created 08-09-2019 05:46 PM
If I am reading your use case correctly, I think you're looking for what the ForkRecord processor does; it allows you to fork a (usually single) record into multiple records based on a Record Path (similar to JSONPath but different syntax and expressiveness), possibly keeping the "root" elements common to each outgoing record.
Created 08-09-2019 05:46 PM
If I am reading your use case correctly, I think you're looking for what the ForkRecord processor does; it allows you to fork a (usually single) record into multiple records based on a Record Path (similar to JSONPath but different syntax and expressiveness), possibly keeping the "root" elements common to each outgoing record.
Created 10-08-2019 01:36 PM
Thank you for the response. This was the correct answer, but I was unable to verify until recently.