After testing, in order to remove a variable you need to send an update request where that variable's value is null. ... View more

For your attempt to use only a CSVReader, have you configured the "timestamp" property in the controller service to describe a format that includes microseconds?   I have encountered a very similar issue in the past. In my case, I was writing avro files using PutDatabaseRecord which already had their schema as part of the file. The schema itself was incorrect and would describe date fields as strings - as such, when I'd write to my DB (which had a date with milliseconds percision type), the milliseconds would get cut out and presumably some default parsing would allow the rest of the date to be written correctly.   I the solution I found was to turn my avro files into CSV, then use a CSVReader in my PutDatabaseRecord processor. In my case, configuring the timestamp format to include milliseconds ( .SSS in the end) would end up allowing the processor to write the records to the DB correctly without any data loss. ... View more

Perhaps JoltTransformRecord could help you.   The jolt transformation "shift" let's you rename fields - in a csv file's case, headers. If you know all your replaces ahead of time, you could define a transformation like: { operation: shift spec: {     "kgg":"g",     "c34":"c",     "*":"&"     } }   Note the last shift, "*":"&" ,  which would transfer over the rest of the headers you didn't specifically rename.  ... View more

Perhaps you can use the processor CalculateRecordStats to count how many records of each situation code you have in a given file, then once that is put in your attributes you can try a couple of different methods to decide what to do, for example you can use RouteOnAttribute and then check if attribute sit_code.pod>0 then send to extract_one_pod_element and there you could use QueryRecord to take out exactly one POD element, otherwise route to LIV, AAR, etc. ... View more