Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

[NIFI] : dataprovenance API : value "percentCompleted"

Highlighted

[NIFI] : dataprovenance API : value "percentCompleted"

New Contributor

hi all,

Somebody could explain what is mean the value "percentCompleted" when I make a POST /nifi-api/provenance ?

Must I wait percentCompleted egal 100% before make a GET /nifi-api/provenance/${id} ?

Here

percentCompleted = 33%
{"provenance":{"id":"a0e66f97-b318-1fa4-0000-0000241111aa","uri":"https://nifi001:9443/nifi-api/provenance/a0e66f97-b318-1fa4-0000-0000241111aa","submissionTime":"09/12/2017 11:29:16.606 CEST","expiration":"09/12/2017 11:59:16.606 CEST","percentCompleted":33,"finished":false,"request":{"searchTerms":{"Component ID":"7176ddeb-015e-1000-ffff-ffff8dee922b","Event Type":"DROP"},"startDate":"09/12/2017 10:00:00 CEST","endDate":"09/12/2017 11:59:59 CEST","maxResults":1000},"results":{"provenanceEvents":[],"total":"0","totalCount":0,"generated":"11:29:16 CEST","oldestEvent":"09/11/2017 12:00:06 CEST","bash-4.1# curl -v --compress -H Content-type:application/json --cacert nifi-cert.pem --cert ./nifi001.p12 -XGET https://nifi001:9443/nifi-api/provenance/a0e66f97-b318-1fa4-0000-0000241111aa

thanks

1 REPLY 1

Re: [NIFI] : dataprovenance API : value "percentCompleted"

Rising Star
@mayki wogno Depending on which implementation you're using and how large your Provenance Repository is, some queries may take a while to complete. As a result, they are run in the background. Once you post to /nifi-api/provenance, you get back a "Provenance Request" that tells you how close your search is to completing. So in the example above, it is 33% of the way finished searching through all of your Provenance Repository. When you make a GET request to /nifi-api/provenance/${id} you are fetching an updated result. So what is typically done, such as in the UI, is to poll that GET endpoint every couple of seconds and look at the response to see if the query has completed. If the query has completed, then the "provenanceEvents" element will be populated. If the query has not yet completed, it may or may not be populated, depending on the value of the "incrementResults" query parameter that you pass to the GET call (if not specified, it defaults to true, but if you intend to wait for the results to complete, you may not want the overhead of sending back those intermediate results every time you poll).
Don't have an account?
Coming from Hortonworks? Activate your account here