Support Questions

Find answers, ask questions, and share your expertise

Nifi Testing Challenges

avatar
New Contributor

Hi, our team is researching the Nifi ETL tool as a candidate. I am trying to find challenges or drawbacks with respect to testing capabilities. 

 

Can anyone guide me to some useful links regarding testing challenges, pros or cons within Nifi

1 REPLY 1

avatar
Master Mentor

@uzi1990   Can you provide more detail around the type of testing you are referring to?  Testing what specifically?

NiFi is a flow based programming ETL tool. As a user you add and configure components (processors,  RPGs, Remote ports, funnels, etc..) to the NiFi canvas.  Then interconnecting those added components via connections containing component relationships.  Processor components (currently in excess of 300 unique processor available) can be started and stopped one by one or on groups.  When a component executed in generates or passes a FlowFile to a downstream relationship.  Via the NiFi UI, users can list the contents of a downstream connection and view/download the content of the FlowFile for inspection and also view any metadata/attributes NiFi has set for those FlowFiles.  This is how you would validate the processor configuration produced the expected output you want.  You can then start the next processor component in your dataflows and do the same processor over.  Assuming you have content repository archive enabled, you can also execute an entire flow and examine the generated data provenance for any FlowFile(s) that traversed that dataflow.  You can see the content and meatadata/attributes as they existed at each generated provenance event.

Example Data Provenance lineage:

MattWho_0-1659553149946.png

You can right click on any event dot and view the details:

MattWho_1-1659553232874.png

 

If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Matt