Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

[NIFI-API] Why only one node send the result of data provenance

[NIFI-API] Why only one node send the result of data provenance

New Contributor

Hi all,

I Have a cluster NIFI with 3 node, when I'm making GET dataprovenance with python script, only nifi-node003 sent me the result.

So can't receive dataprovenance from nifi-node001 and nifi-node002

What's wrong ?

thanks for helps

"{\"provenance\":{\"request\":{\"maxResults\":1000,\"startDate\":\"08/23/2017 03:00:00 CEST\",\"endDate\":\"08/23/2017 03:59:59 CEST\",\"searchTerms\":{\"EventType\":\"SEND\",\"ProcessorID\":\"0dd63d71-8620-1ce1-be9d-587c5f6ec679\"}}}}"

    conn=httplib.HTTPSConnection(clusterNode, 9443, key_file=None, cert_file="/var/opt/hosting/nifi/conf/nifi-001.pem")

    headers = {"Content-Type": "application/json", "Accept": "application/json, text/javascript"}
    conn.request("POST", "/nifi-api/provenance", json.dumps(params), headers)
    response = conn.getresponse()
    result = json.loads(response.read())
    conn.close()

    conn.request("GET", "/nifi-api/provenance/" + result['provenance']['id'])

    responseGet = conn.getresponse()
    resultGet = json.loads(responseGet.read())

    conn.request("DELETE", "/nifi-api/provenance/" + resultGet['provenance']['id'])
    res = conn.getresponse()
    conn.close()

5 REPLIES 5

Re: [NIFI-API] Why only one node send the result of data provenance

New Contributor

Hi @mayki wogno,

The POST /provenance endpoint submits a provenance request. The GET /provenance/{id} endpoint retrieves the provenance request object, and the clusterNodeId is the id of the cluster node where the request exists (ie, the node doing the aggregation processing).

Does that answer your question?

Re: [NIFI-API] Why only one node send the result of data provenance

New Contributor

@kdoran:

With the web-ui I got 18 result with my search data provenance, but when i'll doing with python script i got only 5 (only has the result from node003)

38388-web-ui-dataprovenance.jpg

38389-cli-provenance.jpg

Re: [NIFI-API] Why only one node send the result of data provenance

New Contributor

Hi @mayki wogno. Can you post the API call you make and the full JSON response?

Re: [NIFI-API] Why only one node send the result of data provenance

Master Guru

@mayki wogno

My guess here is that your API call is only directed at nifi003.

When you run a query from within the UI, your query request is replicated all nodes. The results from each of those individual queries are returned to the node your browser is currently displaying. The results are correlated in the provenance results UI you see above.

You can use the developer tools in your browser to see all the rest-api calls that are made when you do your provence query via the UI. You will need to make sure you make all the same queries when interacting with the nodes rest-api directly.

Thanks,

Matt

Highlighted

Re: [NIFI-API] Why only one node send the result of data provenance

New Contributor

@matt, by default the api get a query from cluster except if we use clusternodeid in the parameter of query, right ?

Don't have an account?
Coming from Hortonworks? Activate your account here