"createdOn": {"$date":"${now():format('YYYY-MM-dd HH:mm:ss'):replace(' ', 'T'):append('.000+00:00')}"} used this to store ISO date format in mongo db from Jolt and nifi expression language. ... View more

Assuming that there would be a way to check for flow files which went through failure, how would you extract them? Manual search within the Data Provenance Menu from within the GUI? Or using the REST API? Now, what I do know is that you cannot identify those files that easy, because they do not write any specific lines within the Data Provenance. If you have a processor right after the failure queue (or if you terminate the failure queue in the processing processor) you can query using that Component ID and identify the type = DROP, meaning that those files have been "processed" (Indicates a provenance event for the conclusion of an object's life for some reason other than object expiration). More about the types can be found here: https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.1.2/bk_user-guide/content/provenance_events.html ... View more