Support Questions
Find answers, ask questions, and share your expertise

split json arrays into json objects and parse

New Contributor

How to split complexed json arrays into individual json objects with SplitJson processor in NIFI? I don't know how to configure the relationship original, split, failure. Json arrays is below


{

"scrollId1": "xyz",

"data": [

{

"id": "app-server-dev-glacier",

"uuid": "a0733c21-6044-11e9-9129-9b2681a9a063",

"name": "app-server-dev-glacier",

"type": "archiveStorage",

"provider": "aws",

"region": "ap-southeast-1",

"account": "164110977718"

},

{

"id": "abc.company.archive.mboi",

"uuid": "95100b11-6044-11e9-977a-f5446bd21d81",

"name": "abc.company.archive.mboi",

"type": "archiveStorage",

"provider": "aws",

"region": "us-east-1",

"account": "852631421774"

}

]

}

I need to split it into

{

"id": "app-server-dev-glacier",

"uuid": "a0733c21-6044-11e9-9129-9b2681a9a063",

"name": "app-server-dev-glacier",

"type": "archiveStorage",

"provider": "aws",

"region": "ap-southeast-1",

"account": "164110977718"

},

{

"id": "abc.company.archive.mboi",

"uuid": "95100b11-6044-11e9-977a-f5446bd21d81",

"name": "abc.company.archive.mboi",

"type": "archiveStorage",

"provider": "aws",

"region": "us-east-1",

"account": "852631421774"

}


Next, I need to insert another field "time" in front of "id", the first attribute of individual object.

I used SplitJson processor, and JSON Path Expression is $.data.id.*, but the relationship reports error. Don't know how to config relationship branches, original, split and failure. Any one have any advice? @Shu

3 REPLIES 3

New Contributor

Super Guru

The JsonPath Expression is meant to identify an array, then SplitJson will split each element into its own flow file. Try "$.data" as your JsonPath Expression, and use the "splits" relationship to send things downstream. The "original" relationship will contain the incoming flow file, which doesn't sound like what you want.

New Contributor

@Matt Burgess, thanks, I will try from my side. But which processors should I use if batch messages like below:

{"id":"D1-V11-id11","provider":"D1-V12-provider","type":"D1-V13-type","parentsource":"D1-V14-parentsource","account":"D1-V15-account","region":"D1-V16-region",...},{"id":"D1-V21-21","provider":"D1-V22-provider","type":"D1-V23-type","parentsource":"D1-V24-parentsource","account":"D1-V25-account","region":"D1-V26-region",...}...,{"id":"D1-Vn1-idn1","provider":"D1-Vn2-provider","type":"D1-Vn3-type","parentsource":"D1-Vn4-parentsource","account": "D1-Vn5-account","region": "D1-Vn6-region",...} CR

{"id":"D2-V11-id11","provider":"D2-V12-provider","type":"D2-V13-type","parentsource":"D2-V14-parentsource","account":"D2-V15-account","region":"D2-V16-region",...},{"id":"D2-V21-id21","provider":"D2-V22-provider","type":"D2-V23-type","parentsource":"D2-V24-parentsource","account":"D2-V25-account","region":"D2-V26-region",...}...,{"id":"D2-Vn1-idn1","provider":"D2-Vn2-provider","type":"D2-Vn3-type","parentsource":"D2-Vn4-parentsource","account": "D2-Vn5-account","region": "D2-Vn6-region",...} CR

… CR

{"id":"Dn-V11-id11","provider":"Dn-V12-provider","type":"Dn-V13-type","parentsource":"Dn-V14-parentsource","account":"Dn-V15-account","region":"Dn-V16-region",...},{"id":"Dn-V21-id21","provider":"Dn-V22-provider","type":"Dn-V23-type","parentsource":"Dn-V24-parentsource","account":"Dn-V25-account","region":"Dn-V26-region",...}...,{"id":"Dn-Vn1-idn1","provider":"Dn-Vn2-provider","type":"Dn-Vn3-type","parentsource":"Dn-Vn4-parentsource","account": "Dn-Vn5-account","region": "Dn-Vn6-region",...}


This is not the standard JSON format. Any advice?