Hi @Colombia , First of all you are right , the documentation seems to be lacking around this call. When I check the Nifi Rest API documentation under (Nifi-Api ) here is what I find for the upload api : For some reason the Name of all the parameters are not shown ! Not sure if they were missed by mistake or I'm missing something here @MattWho , @steven-matison might know. After trial and error using Postman I was able to get it working and here is the curl command as provided by postman:   curl --location 'https://{url}:9443/nifi-api/process-groups/17baf155-018c-1000-7bde-586d185d1d0d/process-groups/upload' \ --header 'Content-Type: multipart/form-data' \ --header 'Authorization: Bearer ...' \ --form 'positionX="11.0"' \ --form 'clientId="4036074c-018c-1000-3e06-aaaaaaaaaaaa"' \ --form 'disconnectNode="true"' \ --form 'groupName="testapi"' \ --form 'positionY="557.0"' \ --form 'file=@"/C:/nifi-2.0.0-M1-bin/nifi-2.0.0-M1/test123.json"'       If that helps please accept solution. Thanks ... View more

@scoutjohn, I can see the full problem now. I wish I have considered it in my first response because it will change things a little bit. Since you have mentioned that you could have other arrays in each level, the current spec basically wont work unless you create different specs for different scenarios which is not a good idea. This can be resolved in two shift transformations: 1- Bring those array objects from different levels  into "serviceCharacteristic" array. 2- Apply the name, value assignment based on each object array index in "serviceCharacteristic" array. In this case no need to figure out how many arrays objects you might have in the each level, nor you have to assign hard coded index. Here is the final spec:   [ // First transformation, group array objects under service // and serviceOrderItem into one array "serviceCharacteristic" { "operation": "shift", "spec": { "*": "&", "serviceOrderItem": { "*": { "id": "serviceOrderItem.[&1].&", "action": "serviceOrderItem.[&1].&", "service": { "id": "serviceOrderItem.[&2].service.&", "state": "serviceOrderItem.[&2].service.&", //group any array objects at the supportingService level into serviceCharacteristic "*": "serviceOrderItem.[&2].service.serviceCharacteristic[].&" }, //group any array objects at the modifyPath level into serviceCharacteristic "*": "serviceOrderItem.[&1].service.serviceCharacteristic[].&" } } } }, // 2ed transformation to assign name, value for each // serviceCharacteristic array object { "operation": "shift", "spec": { "*": "&", "serviceOrderItem": { "*": { "*": "serviceOrderItem.[&1].&", "service": { "*": "serviceOrderItem.[&2].service.&", "serviceCharacteristic": { "*": { "*": { "$": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].name", "@": "serviceOrderItem.[&5].service.serviceCharacteristic[&2].value" } } } } } } } } ]   Note: Not sure why you need the "recursivelySquashNulls"  transformation. I did not see any use case for here but you can add it if it solves other issues. Hope that helps.   ... View more

@ariel12, If I understood correctly you need to see if the record exist in the DB based on certain unique value\s before inserting it to prevent duplicates. If that is the case, then you can utilize processor called LookupRecord that can lookup record from the different sources including DB and then route the result to match or unmatched where you can decide the proper action. To see an example on how you can set up the lookupRecord and the lookupRecordService you can watch the following video: https://www.youtube.com/watch?v=17KG3kKQkOs More info about RecordLookup: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.23.2/org.apache.nifi.processors.standard.LookupRecord/index.html I Hope that will help. Thanks   ... View more

Hi @ariel12 , This is happening because the PutSQL expects the timestamp (if no format is provided) to be in the format of "yyyy-MM-dd HH:mm:ss.SSS" . That is the format passed to the Java timestamp constructor by default when trying to parse the date string. Since the date in your input json is missing the milliseconds ".SSS" and no format is provided the process will fail with the "Unparseable date" error. To fix this,  you can use different processors as follows: 1- UpdateAttribute You need to find what sql.args.N.value attribute generated by the ConvertJsonToSQL processor that corresponds to the value of the alertDate, then you can update this value using to UpdateAttribute to the proper expected format by PUTSql. For example, if you find that the alertDate value is stored in sql.args.1.value then , then you can update it using Expression Language to the following: ${sql.args.1.value:toDate("yyyy-MM-dd HH:mm:ss"):format("yyyy-MM-dd HH:mm:ss.SSS")} And the UpdateAttribute will look like: Alternatively:  Instead of updating the value, you can update the type for the sql.args.N.type to set it to be as varchar ( value of 12) instead of timestamp (value of 93), this will fix the problem too. Also you can assign format using sql.args.N.format -not provided by default - where you set the format value to "yyyy-MM-dd HH:mm:ss" , the PutSQL will respect the provided format and use it.   2- UpdateRecord: If you dont like purging default data , or figuring out which sql args the timestamp is getting stored into specially when you have large data set and many timestamp fields where it becomes challenging to do or figure out index based on changing order, Instead you want to do it on the source data before it gets converted to SQL, then you can use UpdateRecord where you have to provide Json reader\writer and add dynamic property that sets the path to the target field and set it accordingly. In your case the UpdateRecod will look like this: The Value field has the following: ${field.value:toDate("yyyy-MM-dd HH:mm:ss"):format("yyyy-MM-dd HH:mm:ss.SSS")} Note: the field.value in the expression is provided by this processor to reference the original value of the given path. please refer to UpdateRecod documentation under Additional Details. 3- Jolt Transformation: Since you asked about using Jolt, you can use it as well to set the data in the source. Since Jolt doesnt have datetime functions you just use string concat to add default value for the .SSS like .000. here is the spec : [ { "operation": "modify-overwrite-beta", "spec": { "alertDate": "=concat(@(1,alertDate),'.000')" } } ] If that helps please accept solution. Thanks       ... View more

Hi @Alevc , It seems the problem with the syntax "&[&3]"  that is used inside the RUBRICS elements transformation. Notice how its always producing an array of three elements for each RUBRICS element index, this happen to matche the number of objects in the parent array at level &3. Since all parent object have at least 1 RUBRIC object at index 0, they will be grouped together under 0, however only the 3rd Parent object has more than one elements so when it try to group under index 1,2,3...10 only one object is found so its adds two null values to the array to keep total at 3. Im not sure why its doing this, I thought I understand jolt but sometimes I find that Im still scratching the surface :). The good news is I have seen this pattern  before and I know how to resolve. The challenge here is how to combine both child and parent attributes together to form a unique object against each child. You can do this by creating a unique key to help you group each parent & child attributes under one object assigned to that key, that will be the first transformation. The second transformation is to dump all parent & child object into the final array leaving the assigned unique key behind. The unique key in this case can be combination of parent index and child index. I used underscore (_) to separate.   [ // First transformation to group each parent\child attributes // under a unique key using combination of parent index and // chilld index , for example the first child under the first // parent will have 0_0, first child of second parent will be 1_0, // 2ed child of the 3rd parent will be 2_1 and so on { "operation": "shift", "spec": { "*": { "RUBRICS": { "*": { "@name": "[#].&1_&3.RUBRIC", "@business_unit": "[#].&1_&3.COMPENSATION_GROUP", "@classification": "[#].&1_&3.CLASSIFICATION", "@code": "[#].&1_&3.CODE", "@amount": "[#].&1_&3.AMOUNT", "@type": "[#].&1_&3.TYPE", "@aggregation_key": "[#].&1_&3.AGGREGATION_KEY", "@(2,ENTITY_TYPE)": "[#].&1_&3.ENTITY_TYPE", "@(2,ENTITY_ID)": "[#].&1_&3.ENTITY_ID", "@(2,ENTITY_DOCUMENT)": "[#].&1_&3.ENTITY_DOCUMENT", "@(2,CALCULATION_DATE)": "[#].&1_&3.CALCULATION_DATE" } } } } }, // Second trabsforamtion is to dump all combined parent & // child objects into the final array without the keys. { "operation": "shift", "spec": { "*": { "*": "[]" } } } ]   If you find this helpful please accept solution. Thanks ... View more