Support Questions

Find answers, ask questions, and share your expertise

How to see the Data Provenance and Lineage in Data Flow on Public Cloud?

avatar
Visitor

This video (timestamped) shows you can list the queue on connections, and see provenance and lineage in flow designer: https://youtu.be/8cZJ9CyLYyI?t=5904 But in the public cloud version of Cloudera Data Flow, that functionality is missing. I can list queue and see data in many formats, but no provenance and lineage. Do we need Data Hub to do this or am I missing something? I am doing this from the Web console of Data Flow Designer.

I am trying this out on a trial account of the Cloudera DataFlow, so not sure if that affects my experience. Provenance and replay was an important function I wanted to try. 

 

Screenshots (Imgur link) 

1 ACCEPTED SOLUTION

avatar
Master Guru

Provenance/lineage is not currently visible from the Flow Designer. This is intended because the Flow Designer UI is for flow design regardless of whether there is a Test Session or deployment active. Provenance and lineage is associated with actual data running through a deployment, so to view these you'll need to navigate to the Cloudera Flow Management (NiFi) canvas from the deployment view once your flow has been deployed. From the canvas you can proceed as the video instructs and hopefully it looks familiar at that point.

View solution in original post

2 REPLIES 2

avatar
Community Manager

@mansmaan Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @MattWho @SAMSAL @mburgess  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Senior Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Master Guru

Provenance/lineage is not currently visible from the Flow Designer. This is intended because the Flow Designer UI is for flow design regardless of whether there is a Test Session or deployment active. Provenance and lineage is associated with actual data running through a deployment, so to view these you'll need to navigate to the Cloudera Flow Management (NiFi) canvas from the deployment view once your flow has been deployed. From the canvas you can proceed as the video instructs and hopefully it looks familiar at that point.