Created 10-28-2016 10:42 AM
Is there any way to get provenance event id? In nifi-rest api ,I want to get a provenance event use /provenance-events/{id}, What should I do? Thanks.
Created 10-29-2016 03:38 AM
You can send a
POST
request to https://host:port/nifi-api/provenance
which will submit a provenance query. The contents of this request should be {"provenance":{"request":{"maxResults":1000}}}
(with a configurable count). The response will contain an identifier for the query (as it may take a long time to execute). You can then submit a GET
request to https://host:port/nifi-api/provenance/{query-id}
which will respond with the results of the query. From this response, you can iterate/extract specific provenance event IDs and request more information using the GET /provenance-events/{id}
method. You can also add additional filters and search terms in the initial query to refine it (see GET /provenance/search-options
for available options).
Example response to search options (
GET https://nifi.nifi.apache.org:9443/nifi-api/provenance/search-options
):
{"provenanceOptions":{"searchableFields":[{"id":"EventType","field":"eventType","label":"Event Type","type":"STRING"},{"id":"FlowFileUUID","field":"uuid","label":"FlowFile UUID","type":"STRING"},{"id":"Filename","field":"filename","label":"Filename","type":"STRING"},{"id":"ProcessorID","field":"processorId","label":"Component ID","type":"STRING"},{"id":"Relationship","field":"relationship","label":"Relationship","type":"STRING"}]}}
Example response to initial query submission (
POST https://nifi.nifi.apache.org:9443/nifi-api/provenance
):
{"provenance":{"id":"0e74ca7f-0158-1000-e780-8ec16cb486ba","uri":"https://nifi.nifi.apache.org:9443/nifi-api/provenance/0e74ca7f-0158-1000-e780-8ec16cb486ba","submissionTime":"10/28/2016 20:21:24.866 PDT","expiration":"10/28/2016 20:51:24.868 PDT","percentCompleted":0,"finished":false,"request":{"searchTerms":{},"maxResults":1000},"results":{"provenanceEvents":[],"total":"0","totalCount":0,"generated":"20:21:24 PDT","oldestEvent":"10/28/2016 20:14:44 PDT","timeOffset":-25200000}}}
Example response to query update request (
GET https://nifi.nifi.apache.org:9443/nifi-api/provenance/0e74ca7f-0158-1000-e780-8ec16cb486ba
):
{"provenance":{"id":"0e74ca7f-0158-1000-e780-8ec16cb486ba","uri":"https://nifi.nifi.apache.org:9443/nifi-api/provenance/0e74ca7f-0158-1000-e780-8ec16cb486ba","submissionTime":"10/28/2016 20:21:24.866 PDT","expiration":"10/28/2016 20:51:24.876 PDT","percentCompleted":100,"finished":true,"request":{"searchTerms":{},"maxResults":1000},"results":{"provenanceEvents":[{"id":"13","eventId":13,"eventTime":"10/28/2016 20:15:43.606 PDT","lineageDuration":11,"eventType":"DROP","flowFileUuid":"66d1354f-1c0d-4658-9263-17d77ef741df","fileSize":"0 bytes","fileSizeBytes":0,"groupId":"0947f405-0158-1000-d643-3299cb111b40","componentId":"0e6f37b9-0158-1000-254c-d79aeb605761","componentType":"LogAttribute","componentName":"LogAttribute","attributes":[{"name":"filename","value":"1234593201638265","previousValue":"1234593201638265"},{"name":"path","value":"./","previousValue":"./"},{"name":"uuid","value":"66d1354f-1c0d-4658-9263-17d77ef741df","previousValue":"66d1354f-1c0d-4658-9263-17d77ef741df"}],"parentUuids":[],"childUuids":[],"details":"Auto-Terminated by success Relationship","contentEqual":false,"inputContentAvailable":false,"outputContentAvailable":false,"outputContentClaimFileSize":"0 bytes","outputContentClaimFileSizeBytes":0,"replayAvailable":false,"replayExplanation":"Cannot replay data from Provenance Event because the event does not contain the required Content Claim","sourceConnectionIdentifier":"0e6f4ed9-0158-1000-0ab4-97a391fb36b8"}, ... {"id":"0","eventId":0,"eventTime":"10/28/2016 20:14:44.820 PDT","lineageDuration":10,"eventType":"CREATE","flowFileUuid":"3e5d9a82-f185-48d5-b0e4-f3b818f81a13","fileSize":"0 bytes","fileSizeBytes":0,"groupId":"0947f405-0158-1000-d643-3299cb111b40","componentId":"0e6e6df1-0158-1000-e997-ec080b4cdd98","componentType":"GenerateFlowFile","componentName":"GenerateFlowFile","attributes":[{"name":"filename","value":"1234534415887832"},{"name":"path","value":"./"},{"name":"uuid","value":"3e5d9a82-f185-48d5-b0e4-f3b818f81a13"}],"parentUuids":[],"childUuids":[],"contentEqual":false,"inputContentAvailable":false,"outputContentAvailable":false,"outputContentClaimFileSize":"0 bytes","outputContentClaimFileSizeBytes":0,"replayAvailable":false,"replayExplanation":"Cannot replay data from Provenance Event because the event does not contain the required Content Claim"}],"total":"14","totalCount":14,"generated":"20:21:25 PDT","oldestEvent":"10/28/2016 20:14:44 PDT","timeOffset":-25200000}}}
Response to query delete request when processing is complete (
DELETE https://nifi.nifi.apache.org:9443/nifi-api/provenance/0e74ca7f-0158-1000-e780-8ec16cb486ba
):
{}
Created 10-29-2016 03:38 AM
You can send a
POST
request to https://host:port/nifi-api/provenance
which will submit a provenance query. The contents of this request should be {"provenance":{"request":{"maxResults":1000}}}
(with a configurable count). The response will contain an identifier for the query (as it may take a long time to execute). You can then submit a GET
request to https://host:port/nifi-api/provenance/{query-id}
which will respond with the results of the query. From this response, you can iterate/extract specific provenance event IDs and request more information using the GET /provenance-events/{id}
method. You can also add additional filters and search terms in the initial query to refine it (see GET /provenance/search-options
for available options).
Example response to search options (
GET https://nifi.nifi.apache.org:9443/nifi-api/provenance/search-options
):
{"provenanceOptions":{"searchableFields":[{"id":"EventType","field":"eventType","label":"Event Type","type":"STRING"},{"id":"FlowFileUUID","field":"uuid","label":"FlowFile UUID","type":"STRING"},{"id":"Filename","field":"filename","label":"Filename","type":"STRING"},{"id":"ProcessorID","field":"processorId","label":"Component ID","type":"STRING"},{"id":"Relationship","field":"relationship","label":"Relationship","type":"STRING"}]}}
Example response to initial query submission (
POST https://nifi.nifi.apache.org:9443/nifi-api/provenance
):
{"provenance":{"id":"0e74ca7f-0158-1000-e780-8ec16cb486ba","uri":"https://nifi.nifi.apache.org:9443/nifi-api/provenance/0e74ca7f-0158-1000-e780-8ec16cb486ba","submissionTime":"10/28/2016 20:21:24.866 PDT","expiration":"10/28/2016 20:51:24.868 PDT","percentCompleted":0,"finished":false,"request":{"searchTerms":{},"maxResults":1000},"results":{"provenanceEvents":[],"total":"0","totalCount":0,"generated":"20:21:24 PDT","oldestEvent":"10/28/2016 20:14:44 PDT","timeOffset":-25200000}}}
Example response to query update request (
GET https://nifi.nifi.apache.org:9443/nifi-api/provenance/0e74ca7f-0158-1000-e780-8ec16cb486ba
):
{"provenance":{"id":"0e74ca7f-0158-1000-e780-8ec16cb486ba","uri":"https://nifi.nifi.apache.org:9443/nifi-api/provenance/0e74ca7f-0158-1000-e780-8ec16cb486ba","submissionTime":"10/28/2016 20:21:24.866 PDT","expiration":"10/28/2016 20:51:24.876 PDT","percentCompleted":100,"finished":true,"request":{"searchTerms":{},"maxResults":1000},"results":{"provenanceEvents":[{"id":"13","eventId":13,"eventTime":"10/28/2016 20:15:43.606 PDT","lineageDuration":11,"eventType":"DROP","flowFileUuid":"66d1354f-1c0d-4658-9263-17d77ef741df","fileSize":"0 bytes","fileSizeBytes":0,"groupId":"0947f405-0158-1000-d643-3299cb111b40","componentId":"0e6f37b9-0158-1000-254c-d79aeb605761","componentType":"LogAttribute","componentName":"LogAttribute","attributes":[{"name":"filename","value":"1234593201638265","previousValue":"1234593201638265"},{"name":"path","value":"./","previousValue":"./"},{"name":"uuid","value":"66d1354f-1c0d-4658-9263-17d77ef741df","previousValue":"66d1354f-1c0d-4658-9263-17d77ef741df"}],"parentUuids":[],"childUuids":[],"details":"Auto-Terminated by success Relationship","contentEqual":false,"inputContentAvailable":false,"outputContentAvailable":false,"outputContentClaimFileSize":"0 bytes","outputContentClaimFileSizeBytes":0,"replayAvailable":false,"replayExplanation":"Cannot replay data from Provenance Event because the event does not contain the required Content Claim","sourceConnectionIdentifier":"0e6f4ed9-0158-1000-0ab4-97a391fb36b8"}, ... {"id":"0","eventId":0,"eventTime":"10/28/2016 20:14:44.820 PDT","lineageDuration":10,"eventType":"CREATE","flowFileUuid":"3e5d9a82-f185-48d5-b0e4-f3b818f81a13","fileSize":"0 bytes","fileSizeBytes":0,"groupId":"0947f405-0158-1000-d643-3299cb111b40","componentId":"0e6e6df1-0158-1000-e997-ec080b4cdd98","componentType":"GenerateFlowFile","componentName":"GenerateFlowFile","attributes":[{"name":"filename","value":"1234534415887832"},{"name":"path","value":"./"},{"name":"uuid","value":"3e5d9a82-f185-48d5-b0e4-f3b818f81a13"}],"parentUuids":[],"childUuids":[],"contentEqual":false,"inputContentAvailable":false,"outputContentAvailable":false,"outputContentClaimFileSize":"0 bytes","outputContentClaimFileSizeBytes":0,"replayAvailable":false,"replayExplanation":"Cannot replay data from Provenance Event because the event does not contain the required Content Claim"}],"total":"14","totalCount":14,"generated":"20:21:25 PDT","oldestEvent":"10/28/2016 20:14:44 PDT","timeOffset":-25200000}}}
Response to query delete request when processing is complete (
DELETE https://nifi.nifi.apache.org:9443/nifi-api/provenance/0e74ca7f-0158-1000-e780-8ec16cb486ba
):
{}
Created 10-31-2016 06:49 AM
Thanks a lot ,I've managed to solve this problem with your help
Created 11-02-2016 01:13 PM
Hi @m mary / @Andy LoPresto: As mentioned above i got provenance id(ex id : 0e74ca7f-0158-1000-e780-8ec16cb486ba). Now i want to run "/provenance-events/{id}" rest API. So i have replced above id with this rest API(/provenance-events/0e74ca7f-0158-1000-e780-8ec16cb486ba). But it is throwing "HTTP/1.1 404 Not Found" error. Please help me which id I need to replace here to get the provenance-events details.
Created 11-03-2016 02:36 AM
hi. provenance id is changing.every id can be used once,you can't use the provenance id to get "/provenance-events/{id}" again,you can get id and get the url directly.
Created 11-03-2016 06:55 AM
Hi, As per your suggestion I have replaced above rest API (/provenance-events/{id}) ID with UUID/Component Id/Identifier. Still it is throwing "404 Not Found" error.
Created 11-04-2016 05:33 PM
Can you please post the complete output of curl or whatever REST client you are using? The complete URI to use is http(s)://host:port/nifi-api/provenance-events/{id}
.
Created 04-19-2017 07:43 AM
I got below ouput after executing GET
request to http://<hostname>:9090/nifi-api/provenance/query-id :
{"provenanceEvents":[{"id":"a9d352d4-20e9-4fbb-851c-3191204d58ba4","eventId":4,"eventTime":"04/18/2017 09:30:14.775 UTC","lineageDuration":1,"eventType":"CREATE","flowFileUuid":"c87d810d-17b7-4c5f-b313-dd4653e81a8d","fileSize":"0 bytes","fileSizeBytes":0,"clusterNodeId":"a9d352d4-20e9-4fbb-851c-3191204d58ba","clusterNodeAddress":"amnifi1.openstacklocal:9090","groupId":"b90e8d65-015a-1000-22ce-e993c787469e","componentId":"61665eb8-015b-1000-0000-0000653a830f","componentType":"ListFTP","componentName":"AAAAAAAAAAAA","attributes":[{"name":"file.group","value":"0"},{"name":"file.lastModifiedTime","value":"2017-04-18T09:30:00+0000"},{"name":"file.owner","value":"0"},{"name":"file.permissions","value":"rw-r--r--"},{"name":"filename","value":"jj1.txt"},{"name":"ftp.listing.user","value":"akshay"},{"name":"ftp.remote.host","value":"amnifi3.openstacklocal"},{"name":"ftp.remote.port","value":"21"},{"name":"path","value":"ALL_DAY_WORK"},{"name":"uuid","value":"c87d810d-17b7-4c5f-b313-dd4653e81a8d"}],"parentUuids":[],"childUuids":[],"contentEqual":false,"inputContentAvailable":false,"outputContentAvailable":false,"outputContentClaimFileSize":"0 bytes","outputContentClaimFileSizeBytes":0,"replayAvailable":false,"replayExplanation":"Cannot replay data from Provenance Event because the event does not contain the required Content Claim”},
Now when I am trying to submit GET request to get more info about provenance event I am getting below error message :
$ curl -i -X GET http://<hostname>:9090/nifi-api/provenance-events/{4}
HTTP/1.1 400 Bad Request Date: Wed, 19 Apr 2017 07:21:13 GMT Content-Type: text/plain Transfer-Encoding: chunked Server: Jetty(9.3.9.v20160517) The cluster node identifier must be specified.
Nifi is clustered. Not sure where I should mentioned cluster node id. Please help.
Created on 04-19-2017 05:38 PM - edited 08-18-2019 05:31 AM
As noted in the REST API, clusterNodeId
is a required field in the GET /provenance-events/{id}
request when running in clustered mode. Use the Developer Tools of your browser to watch the requests that the UI fires when you get the details of a provenance event and you should see where to include the clusterNodeId
in the request (my instance is not running in a cluster so it is not present in my request).
Created 04-23-2017 03:58 PM
Please refer below link in order to get more information about a specific provenance event for nifi which is running in clustered mode : - How to get information of a specific provenance event when nifi is running in standalone/clustered...