You need to keep the NIFi copy of the data even after writing a copy of it out via the putSFTP? If you need to retain a local copy of the data, route success twice form your putSFTP processor. So you should be able to do this with..... The UpdateAttribute processor can be used to update the filename by adding the following new property to its configuration: The Local copy of your Files remain unchanged down the success relationship to the left. The copy sent down the path to the right will have its content cleared, filename changed, and then sent via another PutSFTP. Thanks, Matt ... View more

Using commons-collections-3.2.1 solved the issue.Thanks, @Matt Burgess. ... View more

Correct, SelectHiveQL is for statements that return ResultSets (like SELECT *), those results are converted to Avro records. PutHiveQL is for executing statements (except callable statements like stored procedures) that do not return results, such as your ALTER TABLE example. ... View more

Thanks, now I can load from local ... View more

Hi, Matt! very useful and informative articles. Thank you veru much! Could you tell me how do I read content of a flowFile, transform it the way I like, and write the output to a new flowFile attribute (not back to the content)? I was trying to return transformation result from the callback but caught the error: None required for void return which is fairly expected behaviour - callback returns to the session.read function, but the latter does not return anything, I assume. So we get the flowFile itself and its content residing in different namespaces and I can't figure out how can I use content of the flowFile to place it into the attribute. Could you kindly help me, Matt? ... View more

Since this isn't related to the original question, please ask this as its own standalone question and I'd be happy to answer it. (The short answer is you might be able to use UpdateAttribute to change the 4 to the right column number for Table B if you can figure out whether a flow file is for Table A or B) ... View more

@Matt Burgess If the query returns too many rows in QueryDatabaseTable, can you set Max Rows Per Flow File and Fetch Size to something like 10000? -> yes, I have do that, but it doesn't work... I have 30 000 000 + data in postgreSQL so an exception of OutOfMemory is normal. I'm looking forward to your answer for PutElasticSearchHTTP. ... View more

Very interesting article, well done ! Have you done something similar with jython? I'm currently trying to use an execute script, with jython, to put key/values onto the distributed map cache but can't find a way to do it... BR, Paulo Rente ... View more

The UI uses the REST API so you can do it programmatically the same way: /flowfile-queues/{id}/drop-requests ... View more