Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

What is the Best Practice for Querying NiFi 1.0.0 REST API for Provenance Events

avatar
Master Guru

I would like to get a list of all provenance events

https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

Ideally I would like to ingest them with NIFI and store them in HDFS.

Is there an example of POST /provenance?

1 ACCEPTED SOLUTION

avatar

Hi @Timothy Spann, take a look at this answer I just provided for a similar question.

Although, you should also look at SiteToSiteProvenanceReportingTask, which allows you "export" them from the NiFi provenance repository and ingest them into a NiFi instance (could be remote, could be same) as pure data and then manipulate them as you would any other data (e.g. feed to PutHDFS).

View solution in original post

2 REPLIES 2

avatar

Hi @Timothy Spann, take a look at this answer I just provided for a similar question.

Although, you should also look at SiteToSiteProvenanceReportingTask, which allows you "export" them from the NiFi provenance repository and ingest them into a NiFi instance (could be remote, could be same) as pure data and then manipulate them as you would any other data (e.g. feed to PutHDFS).

avatar
Expert Contributor

Hi @Timothy Spann, this is an old post of yours, but I've a similar need as you posted here; I wanted to know if you were able to come up with a working version of a dataflow that queries NiFi API provenance events; if you did, I appreciate if you could share some info about that. Thanks in advance.