@Anderosn  1. If the content of your flow file is too large to be inserted into a single CLOB column, you can split it into smaller chunks and insert each chunk into the database separately. 2. Instead of storing the content in a CLOB column, you can consider storing it in a BLOB (Binary Large Object) column in your database. BLOB columns can store binary data, including large files, without the size limitations of CLOB columns. 3. Store the content of the flow file in an external storage system (e.g., HDFS, Amazon S3) and then insert the reference (e.g., file path or URL) into the database. This approach can be useful if the database has limitations on the size of CLOB or BLOB columns 4. If ExecuteScript is not approved, consider using an external script or application to perform the insertion into the database. You can trigger the script or application from NiFi using ExecuteProcess or InvokeHTTP processors   Regards, Chethan YM   ... View more

Hi, Have you looked into the EnforceOrder processor : https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.23.2/org.apache.nifi.processors.standard.EnforceOrder/index.html Based on your description it should do the job. One thing to keep in mind if you want this to work is what is stating in the description : "... [IMPORTANT] In order to take effect of EnforceOrder, FirstInFirstOutPrioritizer should be used at EVERY downstream relationship UNTIL the order of FlowFiles physically get FIXED by operation such as MergeContent or being stored to the final destination. " The group identifier will be in this case the value of the fragment.identifier: ${fragment.identifie} The Order Attribute will be the name of the fragment.index: fragment.index If you find this helpful please accept the solution. Thanks ... View more

Hi @Anderosn , From previous posts it seems this is a common behavior for COLB column data type and it doesnt seem like you can avoid it. To extract the json value please refer to the following post: https://community.cloudera.com/t5/Support-Questions/Avro-to-Json-adding-extra-delemeters/m-p/380646#M244113   ... View more

Can you post screenshot of the UpdateRecord processor configuration? Also you have to be careful with the provided input because there is an extra comma after last Garry value which makes the json invalid. ... View more

@Anderosn  So MergeContent does just that, merges the content of all FlowFiles being merged. I am not sure how often your GenerateFlowFile processor executes, but when it does execute it will create a FlowFile with a unique filename (unless you set the filename in the GenerateFlowFile processor via a dynamic property).  The produced data by the GenerateFlowFile is routed is routed as a FlowFile to one of the success relationships and a clone FlowFile is routed to the other success relationship in your dataflow (both FlowFiles have same "filename" but different flowfile uuids). The "filename" attribute can be used in the MergeContent processor in the "Correlation Attribute Name" property.  Then you can set min num entries to "2".  This will make sure both FlowFiles with same value in the filename attribute will get allocated to same bin.  The MergeContent property "Attribute Strategy" will need to be set to "Keep All Unique Attributes" so that the final merged FlowFile will include the new token attribute. Now we have to deal with the content.  What we need to make sure is that the FlowFile used to fetch the token has no content before being routed to mergeContent processor. For that you can use the ModifyBytes processor and set "Remove all content" to "true" after your EvaluateJsonPath processor.  Removing the content does not remove the FlowFile metadata/attributes, so this now 0 byte FlowFile will still have its filename value and token attribute with value. ------- Now with above suggestion for your existing dataflow as an option, there are probably many other dataflow designs to accomplish this. Since you are using GenerateFlowFile to create the content needed for your final invokeHTTP, I'd go a different route that does not need a MergeContent processor. GenerateFlowFIle (custom content needed to fetch token) --> InvokeHTTP (get Token) --> EvaluateJsonPath (extract token from content to attribute) --> replaceText ( ("Replacement Strategy"="always replace", "Evaluation mode"="Entire text", "replacement value"=<content needed for your final rest-api call>) --> InvokeHTTP (you final rest-api endpoint request). The above removes need for MergeContent or dealing with multiple paths.  You have a single process flow where any failure in along the path does not result in potential of orphaned binned FlowFile at your MergeContent processor. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

Hi,   You can try the following spec:   [ { "operation": "shift", "spec": { "*": "&", "responseData": { "responseList": { "*": { "individualInfo": { "#${UUID()}": "responseData.responseList.[&2].individualInfo.activityUID", "firstName": "responseData.responseList.[&2].individualInfo.&", "middleName": "responseData.responseList.[&2].individualInfo.&", "lastName": "responseData.responseList.[&2].individualInfo.&", "dateOfBirth": "responseData.responseList.[&2].individualInfo.&" } } } } } } ]   If that helps please accept solution. Thanks ... View more