Hi, I am trying to obtain data ingestion lineage of Nifi via Cloudera Navigator. I have not come across any solution or documentation for the same. Has anyone experimented with this successfully?
Is there an existing solution for this?
At present Navigator doesn't support gathering lineage from NiFi - however within nifi there is lineage of flowfile. You can get the steps from this link https://docs.cloudera.com/HDPDocuments/HDF3/HDF-22.214.171.124/getting-started-with-apache-nifi/content/lin...
Thank you for the clarification.
But I was asking more from the perspective of having 1 central place where the data lineage can be highlighted. Ingestion -> Transformation (all steps in between) -> Visualization.
Right, In CDH its not available but with CDP you have options to install Atlas which already have integration with NiFi
Ingest, transform, and analyze data.
Services: HDFS, YARN, YARN Queue Manager, Ranger, Atlas, Hive Metastore, Hive on Tez, Spark, Oozie, Hue, and Data Analytics Studio
Browse, query, and explore your data in an interactive way.
Services: HDFS, YARN, YARN Queue Manager, Ranger, Atlas, Hive Metastore, Impala, and Hue
Low latency writes, reads, and persistent access to data for Online Transactional Processing (OLTP) use cases.
Services: HDFS, Ranger, Atlas, and HBase