question Re: Split JSON after Convert Record (CSVtoJSON) creating 10,000 duplicate split records in Archives of Support Questions (Read Only)

Split JSON after Convert Record (CSVtoJSON) creating 10,000 duplicate split records

rohancnaidu — Sun, 18 Aug 2019 02:20:35 GMT

Sample Data:-

1,Michael,Jackson

2,Jim,Morrisson

3,John,Lennon

4,Freddie,Mercury

5,Elton,John

refer image CSVtoJSONJSON created successfully

Result After Convert Record (CSVtoJSON)

[ { "id" : 1, "firstName" : "Michael", "lastName" : "Jackson" },

{ "id" : 2, "firstName" : "Jim", "lastName" : "Morrisson" },

{ "id" : 3, "firstName" : "John", "lastName" : "Lennon" },

{ "id" : 4, "firstName" : "Freddie", "lastName" : "Mercury" },

{ "id" : 5, "firstName" : "Elton", "lastName" : "John" } ]

Applying SplitJSON to the convertrecord processor with jsonpath = $.* creates 10,000 splits in the queue

refer image SplitJSON10000splits

I need to split the array of JSON into individuall JSON records and apply some transformation to these records

Re: Split JSON after Convert Record (CSVtoJSON) creating 10,000 duplicate split records

Shu_ashu — Sun, 18 Aug 2019 02:20:21 GMT

@Rohan Naidu

You have connected Failure and Original relationships of split json looped back to same processor,

What does Original relationship means The original FlowFile that was split into segments. If the FlowFile fails processing, nothing will be sent to this relationship i.e your original flowfile will be transferred to this relationship.

Example of Original Flowfile:-

This message will be original flowfile

[{"id":1,"fname":"Michael","lname":"Jackson"},{"id":2,"fname":"Jim","lname":"Morrisson"},{"id":3,"fname":"John","lname":"Lennon"},{"id":4,"fname":"Freddie","lname":"Mercury"},{"id":5,"fname":"Elton","lname":"John"}]

in your case original relation loop back to split json and generating duplicates.

To resolve this issue auto terminate original relation by

Right Click on the Split json processor
Goto Settings tab
Click on Check box before Original relation
Then click on Apply button Right Below of the screen.

Auto terminate original Relationship:-

Splitjson Configs:-

As you can see above screenshot only failure relationship is loop back to the processor and we have auto terminated original relationship.

Re: Split JSON after Convert Record (CSVtoJSON) creating 10,000 duplicate split records

rohancnaidu — Sun, 10 Dec 2017 01:13:03 GMT

@shu Thanks. It worked.