Testing Nifi Unit & Integration Testing & automating it Any Ideas?

[#1]

Just wondering if anyone has any suggestions on Unit Testing & Integration Testing Nifi. Want to make these tests automated as well. Is there any framework or examples currently to acheive these. Thank you.

[#2 · Accepted solution]

For unit testing custom processors, nifi has a test framework, see https://docs.hortonworks.com/HDPDocuments/HDF1/HDF-1.2.0.1/bk_DeveloperGuide/content/testing.html for some details about the TestRunner and nifi-mock package. This allows building of quick mock flows and provides a number of assertions for testing processors.

This approach allows fully automated testing of processor units.

If you're looking for more of an integration testing approach a flow, this is a little more involved. You could use the nifi framework to programmatically create flows around mock repositories. However, for a full integration test in a live nifi instance, I would recommend using tracer messages or object in the relevant ingest points, and testing against results, possibly using the LogAttributes, or Routes based on metadata to check the integrity of your results. GenerateFlowFile can also be useful for testing flows. To automate these in a CI environment you might look at deploying Templates with a script into a CI hosted nifi instance.

[#3]

If you're looking for more of an integration testing flow testing I have built a framework that empowers flow developers do flow testing without writing single line of code by deploying target flow on Nifi Server running in docker container, run tests, generate test report and delete the test flow.

Medium Article: https://medium.com/@dharmachand/comprehensive-guide-to-nifi-flow-testing-with-nipyapi-e44a61975be9

GitHub Repo: https://github.com/dharmachand/nifi-testing

Regards,

Chand