Support Questions
Find answers, ask questions, and share your expertise

Nifi ListS3 processors not reporting bucket source to Atlas

Nifi ListS3 processors not reporting bucket source to Atlas

New Contributor

[Environment]

HDP:3.1.0.0-78

HDF:3.3.1.0-10

 

[Problem]

When Nifi Flow with ListS3 processor reports to Atlas, the lineage doesn't show the bucket source(type in Atlas: aws_s3_bucket )

 

My workflow is 

  1. ListS3 
  2. RouteOnAttribute 
  3. FetchS3Object 
  4. UpdateRecord 
  5. UpdateAttribute 
  6. PutS3Object 

In my imagination, ListS3 should work like GetFile. When reporting Nifi flow with GetFile processor, the lineage in Atlas would show the file source. It looks like below:

26.PNG

 

However, my Nifi lineage in Atlas looks like below:

 

25.PNG

 

 

Is there something I forgot to set up or ListS3 does not provide this feature?