Support Questions
Find answers, ask questions, and share your expertise

Adding Atlas Classification Tags for HDFS Data

Adding Atlas Classification Tags for HDFS Data

New Contributor

I've read the following article to add Atlas classification tags during data ingestion via NiFi

https://community.hortonworks.com/articles/229220/adding-atlas-classification-tags-during-data-inges...

It works as long as the data flow ends with the last processor. But in this article there is a loop, starting with a delay, follow by a getter for the Atlas lineage and if the guid from the lineage isn't found, it started again with the delay. The flow doesn't end and the lineage isn't written. Endless loop.

If the flow ends with the PutHDFS Processor or with the DelayForSomeTime Processor (I delete the rest from the flow), then Atlas generates the lineage.

Is there a switch that I overlooked while reading this article?

I've used this articel to configure NiFi for Atlas Integration https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.2/bk_installing-hdf-and-hdp/content/nifi-atla...

Thanks for help