@ShuThank you very much ... View more

How to split complexed json arrays into individual json objects with SplitJson processor in NIFI? I don't know how to configure the relationship original, split, failure. Json arrays is below { "scrollId1": "xyz", "data": [ { "id": "app-server-dev-glacier", "uuid": "a0733c21-6044-11e9-9129-9b2681a9a063", "name": "app-server-dev-glacier", "type": "archiveStorage", "provider": "aws", "region": "ap-southeast-1", "account": "164110977718" }, { "id": "abc.company.archive.mboi", "uuid": "95100b11-6044-11e9-977a-f5446bd21d81", "name": "abc.company.archive.mboi", "type": "archiveStorage", "provider": "aws", "region": "us-east-1", "account": "852631421774" } ] } I need to split it into { "id": "app-server-dev-glacier", "uuid": "a0733c21-6044-11e9-9129-9b2681a9a063", "name": "app-server-dev-glacier", "type": "archiveStorage", "provider": "aws", "region": "ap-southeast-1", "account": "164110977718" }, { "id": "abc.company.archive.mboi", "uuid": "95100b11-6044-11e9-977a-f5446bd21d81", "name": "abc.company.archive.mboi", "type": "archiveStorage", "provider": "aws", "region": "us-east-1", "account": "852631421774" } Next, I need to insert another field "time" in front of "id", the first attribute of individual object. I used SplitJson processor, and JSON Path Expression is $.data.id.*, but the relationship reports error. Don't know how to config relationship branches, original, split and failure. Any one have any advice? @Shu ... View more

@Derek Calderon - Short answer is no. The ExecuteSQL processor is written to write the output to the FlowFile's content. - There is an alternative solution. You have some processor currently feeding FlowFiles to your ExecuteSQL processor via a connection. My suggestion would be to feed that same connection to two different paths. The first connection feeds to a "MergeContent" processor via a funnel and the second feeds to your "ExecuteSQL" processor. The ExecuteSQL processor performs the query and retrieves the data you are looking for writing it to the content of the FlowFile. You then use a processor like "ExtractText" to extract that FlowFIles new content to FlowFile Attributes. Finally you use a processor like "ModifyBytes" to remove all content of this FlowFile. Finally you feed this processor to the same funnel as the other path. The MergeContent processor could then merge these two flowfiles using the "Correlation Attribute Name" property (assuming "filename" is unique, that could be used), min/max entries set to 2, and "Attribute Strategy" set to "Keep All Unique Attributes". The result should be what you are looking for. - Flow would look something like following: Having multiple identical connections does not trigger NiFi to write the 200 mb of content twice to the the content repository. a new FlowFile is created but it points to the sam content claim. New content is only generated when the executeSQL is run against one of the FlowFiles. So this flow does not produce any additional write load on the content repo other then when the executeSQL writes its output which i am assuming is relatively small? - Thank you, Matt ... View more

@Vengai Magan Please refer to this and this links describes how to install NiFi as Service and this link to setup high performance NiFi. ... View more

Perfect! Thanks! I'll try QueryDatabaseTable for it. It'll be better! ... View more

@Murat Menteşe As your xml doc having array [] in it and i'm not sure how to write matching xslt. As the current xslt converts the array xml into object/element and adding "" for array[]. In case of large data you have to increase the Maximum Buffer Size 1 MB //increase based on your flowfile size as this processor works takes the whole flowfile into memory and does all replacing based on our configs. ... View more

@Amira khalifa Use one of the way from the above shared link to take out only the header from the csv file then in replace text keep the then search for (&|\(|\)|\/_|\s) and in Replacement value keep as empty string, now we are searching for all the special characters in the header flowfile then replacing with empty string.Now add this header flowfile with the other non header flowfile. all the explanation and template.xml are shared in this link. ... View more