Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Obtain Data Ingestion lineage from Apache Nifi via Cloudera Navigator

Solved Go to solution
Highlighted

Obtain Data Ingestion lineage from Apache Nifi via Cloudera Navigator

New Contributor

Hi, I am trying to obtain data ingestion lineage of Nifi via Cloudera Navigator. I have not come across any solution or documentation for the same. Has anyone experimented with this successfully? 

Is there an existing solution for this?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Obatin Data Ingestion lineage from Apache Nifi via Cloudera Navigator

Expert Contributor

Yeah, thats right. Unfortunately there is no feature available to gather Nifi Lineage in Navigator. 

View solution in original post

5 REPLIES 5

Re: Obatin Data Ingestion lineage from Apache Nifi via Cloudera Navigator

Expert Contributor

At present Navigator doesn't support gathering lineage from NiFi - however within nifi there is lineage of flowfile. You can get the steps from this link https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.4.1.1/getting-started-with-apache-nifi/content/lin...

Highlighted

Re: Obatin Data Ingestion lineage from Apache Nifi via Cloudera Navigator

New Contributor

Thank you for the clarification.

But I was asking more from the perspective of having 1 central place where the data lineage can be highlighted. Ingestion -> Transformation (all steps in between) -> Visualization.

Highlighted

Re: Obatin Data Ingestion lineage from Apache Nifi via Cloudera Navigator

Expert Contributor

Right, In CDH its not available but with CDP you have options to install Atlas which already have integration with NiFi

https://docs.cloudera.com/cdpdc/7.0/overview/topics/cdpdc-overview.html

  • Data Engineering

    Ingest, transform, and analyze data.

    Services: HDFS, YARN, YARN Queue Manager, Ranger, Atlas, Hive Metastore, Hive on Tez, Spark, Oozie, Hue, and Data Analytics Studio

  • Data Mart

    Browse, query, and explore your data in an interactive way.

    Services: HDFS, YARN, YARN Queue Manager, Ranger, Atlas, Hive Metastore, Impala, and Hue

  • Operational Database

    Low latency writes, reads, and persistent access to data for Online Transactional Processing (OLTP) use cases.

    Services: HDFS, Ranger, Atlas, and HBase

https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.4.1.1/installing-hdf/content/configure_nifi_for_at...

Highlighted

Re: Obatin Data Ingestion lineage from Apache Nifi via Cloudera Navigator

New Contributor

Thank you again. Yes I know we can use Atlas in CDP.

So, all in all, there is not even a workaround to get Nifi lineage on Navigator right?

Highlighted

Re: Obatin Data Ingestion lineage from Apache Nifi via Cloudera Navigator

Expert Contributor

Yeah, thats right. Unfortunately there is no feature available to gather Nifi Lineage in Navigator. 

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here