Hi @Crags , It looks like you were close....maybe not 🙂 , though heading in the right direction ...will kind of still far a way 😞 . The problem looks easy but is it really ?! Welcome to the world jolt , whenever you think you got it, you face problem that makes you re think your understanding 🙂 I dont mean to discourage you or scare you from jolt . I was in your shoes when I first started where I only knew what {"*":"&"} means . However with practice and a lot of problem solving I got much better and right now I enjoy solving jolt problems . You can read as much tutorial online or lookup some examples  but I can assure you the only way to learn it is through practice. A good  place to look for the latest information is the Jolt github repository. For quick cheat sheet I use this link. Now back to your problem , The challenge is to be able to group extracted attributes from each zone together, for that I use the first shift spec to create each zone  attribute and store them under the same zone parent object. Then I use a second shift to bucket each zone object value into the prices array with their product info.  The spec will look like this:   [ { "operation": "shift", "spec": { "*": { "CODE": "[&1].code", "PRODUCT": "[&1].product", "Zone*": { "$": "[&2].&1.name", "@": "[&2].&1.price", "@(1,Valid)": "[&2].&1.valid" } } } }, { "operation": "shift", "spec": { "*": { "*": "[&1].&", "Zone*": "[&1].prices[]" } } } ]   Since you are new to this , I would also consider looking into another transformation language called jslt and Nifi has processor for that as well. If you are familiar with Xquery language you probably find it easier to learn and sometimes the spec is much simpler like in this scenario where it looks like the following :   [ for (.) let valid=.Valid { "Code": .CODE, "Product":.PRODUCT, "prices" :[ for (.) { "name" :.key, "value": .value, "valid" : $valid } if(test(.key,"Zone\\s\\d+")) ] } ]   make sure to remove the expression .!=null from the filter property if you want to have null values in the result. Hope that helps. If it does, please accept the solution.   ... View more

Hi @Rashad_K , I dont believe such feature exists out of the box. Processors are designed to be isolated from each other and only can communicate via relationships, therefore there is no such thing as rollback which will reset the state of the QDT based on the result of the PutSQL. If I may suggest the following: 1- Depending on why would some records fail using PutSql, you can use the PutSql Failure relationship to handle errors. For example if the failure is due to connectivity issue you can retry it by looping the failure rel back to the PutSql or you can use ControlRate processor before retrying to wait sometime . If the failure is data related or syntax you can store those records in another table for farther processing and re import them back once the data is fixed. The idea from QDT is that once data is inserted into the target table it wont be changed (like logs for example) and any correction need to be sent as new record hence the max value doesnt have to be changed on data error issues.   2- Second option will only work if you have control over the target table in the QDT where records processed successfully are removed from the table. In this case if records failed for some reason and you want to re read then you can invoke the API to reset the state , which means the QDT will start from the top and re read failed records. However in this approach you have to be aware of concurrency issues in case the state is reset while some reocrds are being processed and have not been removed yet from the QDT table where you might end up re reading those records again. That is why I dont recommend this approach.  Of course there could be other options out there but my advise and based on experience is try to pick a solution where isolation is in mind to avoid conflicts and unpredictable behavior. Hope that helps. If it does, please accept solution. Thanks ... View more

Hi, It seems the data is being converted into none datetime format . when I try a simple input with the date you provided and then do an Update Record using the toDate function I get an an epoch time which is a long integer value, Im not sure why this is happening but if you are using records reader like avro or json and you use the infer schema setting the output can be unpredictable. I would suggest to not do an update record on the date values and just try to insert them as they come since they look to be a valid datetime format string. ... View more