Excellent article as usual @Matt Clarke. As an additional side note and to keep this excellent article evolving I did notice when recently configuring our local NiFi instance that the HDF3 doco now suggests better performance using the WriteAheadProvenanceRepository instead of the default PersistentProvananceRepository particularly for environments which consume a lot of small files (available for NiFi v1.2 and above). "When used in a NiFi instance that is responsible for processing large volumes of small FlowFiles, the PersistentProvenanceRepository can quickly become a bottleneck. The WriteAheadProvenanceRepository was then written to provide the same capabilities as the PersistentProvenanceRepository while providing far better performance. Changing to the WriteAheadProvenanceRepository is easy to accomplish, as the two repositories support most of the same properties." (https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.0.0/bk_administration/content/provenance-repository.html). NiFi users meeting this criteria and starting with an instance greater than v1.2 might want to consider this configuration as well. ... View more