Created 08-06-2019 07:52 PM
I have JSON input of the following format:
{
"Id": 1000000,
"ReportName": TestReport,
"Results": [{
"Id": 1,
"ResultId": "1000000-0",
"Query": {
"Id": 001,
"Name": "TestQuery0",
}
}, {
"Id": 2,
"ResultId": "1000000-1",
"Query": {
"Id": 002,
"Name": "TestQuery1",
}
}]
}These file can become quite large depending on the number of Results in the Report and I was hoping to convert the single Flowfile to multiple records for processing. However, due to the format of the JSON a SplitRecord will result in one record per split. There is one report per FlowFile and therefore only 1 root level element.
I am looking for a method or strategy to split the Flowfile into smaller Records while still maintaining the cohesiveness of the report in the end when it put in HDFS.
Current Strategy:
Any advice would be appreciated! Thank you
Created 08-09-2019 05:46 PM
If I am reading your use case correctly, I think you're looking for what the ForkRecord processor does; it allows you to fork a (usually single) record into multiple records based on a Record Path (similar to JSONPath but different syntax and expressiveness), possibly keeping the "root" elements common to each outgoing record.
Created 08-09-2019 05:46 PM
If I am reading your use case correctly, I think you're looking for what the ForkRecord processor does; it allows you to fork a (usually single) record into multiple records based on a Record Path (similar to JSONPath but different syntax and expressiveness), possibly keeping the "root" elements common to each outgoing record.
Created 10-08-2019 01:36 PM
Thank you for the response. This was the correct answer, but I was unable to verify until recently.