Member since
03-03-2025
5
Posts
0
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
347 | 05-25-2025 11:54 PM | |
925 | 03-04-2025 08:02 PM |
05-25-2025
11:54 PM
Hi All, I got the answer from this https://community.cloudera.com/t5/Support-Questions/Nifi-how-to-get-provenance-event-id-in-nifi/m-p/154682#M117123 thread it got resolved my query. Thank you @alopresto
... View more
05-22-2025
11:11 PM
Hi all, I have the requirement that to do the unit testing for each nifi processors while send hl7 message via a python script. The following are the needs: 1. After sent the HL7 test message it should receive the ACK/NACK data= Done 2. After processed the validation check this flow file will move into another upstream processors = [ Here I need the status of the flow file on each upstream processors whether it successful / fails. For that I am trying to extract or fetch the provenance data through nifi rest api, I am using NiFi version 2.0.0 with the python script I sent the payload as payload = { "provenance": { "request": { "componentId": "f6ca227e-0196-1000-13c2-d66cf56e69f2", "sort": "eventTime", "maxResults": 1 } } } with url https://localhsot:port/nifi-api/provenance I am getting this output : :clipboard: :clipboard: Showing 1 recent provenance events: :three_o_clock: 05/14/2025 15:48:24.041 IST :wrench: Component: ExecuteGroovyScript (ExecuteGroovyScript) :page_facing_up: Type: CLONE 🧾 UUID: 1f7c9140-f09d-4da3-bd6c-075ac8e1444b :package: Size: 126 bytes bytes but the issue here is it shows the end of the provenance data not the latest one. the below is my python script method : def fetch_provenance_events(limit=10😞 hdr = { "Authorization": f"Bearer {token}", "Content-Type": "application/json" } print(f"token : {hdr}") try: payload = { "provenance": { "request": { "componentId": "f6ca227e-0196-1000-13c2-d66cf56e69f2", "sort": "eventTime", "maxResults": 1 } } } url = f"{NIFI_BASE_URL}/nifi-api/provenance" response = requests.post(url, headers=hdr,json=payload, verify=False) print(":magnifying_glass_tilted_left: Raw response:", response.status_code) # 👈 Add this print(":magnifying_glass_tilted_left: Text response:", response.text) # 👈 Add this response.raise_for_status() submission = response.json()["provenance"] print(f"submission: {submission}") query_id = submission["id"] print(f" Qry ID : {query_id}") # Poll until complete while not submission.get("finished", False😞 time.sleep(1) resp = requests.get(f"{NIFI_BASE_URL}/provenance-events/latest/{query_id}", headers=hdr, verify=False) print('prv_event_resp : {resp.text}') submission = resp.json()["provenance"] # Now retrieve the results events = submission["results"]["provenanceEvents"] return sorted(events, key=lambda e: e["eventTime"], reverse=True) #events = response.json().get("provenanceEvents", []) #return events except requests.exceptions.HTTPError as e: print(f":cross_mark: HTTP Error: {e}") except Exception as e: print(f":cross_mark: Other Error: {e}") return [] Here is my View Data Provenance (UI) the latest records shows on the top. But it fetches the random data as attached UI screenshot of provenance data: I tried with all combinations of reverse=False but no luck, if someone already tried and got the success result please let me know where I made the wrong or anything that i missed out here. Thanks in advance. VAX
... View more
Labels:
- Labels:
-
Apache NiFi
03-11-2025
09:50 PM
Hi All, I am trying to delete my nifi existing process group through nifi-api curl but unfortunately it gives me error that "The revision must be specified" but everything specified as per the documentation but still not working, below is the curl command what I used for delete the process group(flow) to my target system and also I can able to delete the same through UI but not works with rest api. curl -k -X DELETE "https://localhost:28443/nifi-api/process-groups/836e216e-0195-1000-d3b8-771b257f1fe6" \ -H "Content-Type: application/json" \ -H "Authorization: Bearer Token" \ --data '{ "revision": { "version": 0 }, "disconnectedNodeAcknowledged": false }' ERROR: 400 - The revision must be specified. Also find the below is the output of the process_groups: {'permissions': {'canRead': True, 'canWrite': True}, 'processGroupFlow': {'id': '187acb23-0195-1000-1908-1543c761f376', 'uri': 'https://localhost:28443/nifi-api/flow/process-groups/187acb23-0195-1000-1908-1543c761f376', 'parameterContext': {'id': 'f24131aa-0fad-3876-f76a-591e810f66da', 'permissions': {'canRead': True, 'canWrite': True}, 'component': {'id': 'f24131aa-0fad-3876-f76a-591e810f66da', 'name': 'NiFi_Engine_Parameters'}}, 'breadcrumb': {'id': '187acb23-0195-1000-1908-1543c761f376', 'permissions': {'canRead': True, 'canWrite': True}, 'breadcrumb': {'id': '187acb23-0195-1000-1908-1543c761f376', 'name': 'NiFi Flow'}}, 'flow': {'processGroups': [{'revision': {'version': 0}, 'id': '836e216e-0195-1000-d3b8-771b257f1fe6', 'uri': 'https://localhost:28443/nifi-api/process-groups/836e216e-0195-1000-d3b8-771b257f1fe6', 'position': {'x': 392.0, 'y': 240.0}, 'permissions': {'canRead': True, 'canWrite': True}, 'bulletins': [], 'component': {'id': '836e216e-0195-1000-d3b8-771b257f1fe6', 'versionedComponentId': '3bcf353e-04bc-39e1-85fd-879f7318bb8a', 'parentGroupId': '187acb23-0195-1000-1908-1543c761f376', 'position': {'x': 392.0, 'y': 240.0}, 'name': 'NIFI_HL7_INTEGRATION_ENGINE_PG_ES_100325_latest', 'comments': '', 'parameterContext': {'id': 'f24131aa-0fad-3876-f76a-591e810f66da', 'permissions': {'canRead': True, 'canWrite': True}, 'component': {'id': 'f24131aa-0fad-3876-f76a-591e810f66da', 'name': 'NiFi_Engine_Parameters'}}, 'flowfileConcurrency': 'UNBOUNDED', 'flowfileOutboundPolicy': 'STREAM_WHEN_AVAILABLE', 'defaultFlowFileExpiration': '0 sec', 'defaultBackPressureObjectThreshold': 10000, 'defaultBackPressureDataSizeThreshold': '1 GB', 'executionEngine': 'INHERITED', 'maxConcurrentTasks': 1, 'statelessFlowTimeout': '1 min', 'runningCount': 0, 'stoppedCount': 542, 'invalidCount': 6, 'disabledCount': 0, 'activeRemotePortCount': 0, 'inactiveRemotePortCount': 0, 'upToDateCount': 0, 'locallyModifiedCount': 0, 'staleCount': 0, 'locallyModifiedAndStaleCount': 0, 'syncFailureCount': 0, 'localInputPortCount': 0, 'localOutputPortCount': 0, 'publicInputPortCount': 0, 'publicOutputPortCount': 0, 'statelessGroupScheduledState': 'STOPPED', 'inputPortCount': 0, 'outputPortCount': 0}, 'status': {'id': '836e216e-0195-1000-d3b8-771b257f1fe6', 'name': 'NIFI_HL7_INTEGRATION_ENGINE_PG_ES_100325_latest', 'statsLastRefreshed': '10:16:27 IST', 'aggregateSnapshot': {'id': '836e216e-0195-1000-d3b8-771b257f1fe6', 'name': 'NIFI_HL7_INTEGRATION_ENGINE_PG_ES_100325_latest', 'statelessActiveThreadCount': 0, 'flowFilesIn': 0, 'bytesIn': 0, 'input': '0 (0 bytes)', 'flowFilesQueued': 0, 'bytesQueued': 0, 'queued': '0 (0 bytes)', 'queuedCount': '0', 'queuedSize': '0 bytes', 'bytesRead': 0, 'read': '0 bytes', 'bytesWritten': 0, 'written': '0 bytes', 'flowFilesOut': 0, 'bytesOut': 0, 'output': '0 (0 bytes)', 'flowFilesTransferred': 0, 'bytesTransferred': 0, 'transferred': '0 (0 bytes)', 'bytesReceived': 0, 'flowFilesReceived': 0, 'received': '0 (0 bytes)', 'bytesSent': 0, 'flowFilesSent': 0, 'sent': '0 (0 bytes)', 'activeThreadCount': 0, 'terminatedThreadCount': 0, 'processingNanos': 0}}, 'runningCount': 0, 'stoppedCount': 542, 'invalidCount': 6, 'disabledCount': 0, 'activeRemotePortCount': 0, 'inactiveRemotePortCount': 0, 'upToDateCount': 0, 'locallyModifiedCount': 0, 'staleCount': 0, 'locallyModifiedAndStaleCount': 0, 'syncFailureCount': 0, 'localInputPortCount': 0, 'localOutputPortCount': 0, 'publicInputPortCount': 0, 'publicOutputPortCount': 0, 'parameterContext': {'id': 'f24131aa-0fad-3876-f76a-591e810f66da', 'permissions': {'canRead': True, 'canWrite': True}, 'component': {'id': 'f24131aa-0fad-3876-f76a-591e810f66da', 'name': 'NiFi_Engine_Parameters'}}, 'inputPortCount': 0, 'outputPortCount': 0}], 'remoteProcessGroups': [], 'processors': [], 'inputPorts': [], 'outputPorts': [], 'connections': [], 'labels': [], 'funnels': []}, 'lastRefreshed': '10:16:27 IST'}} Awaiting for your response. Thanks Vijay Amirtharaj
... View more
Labels:
- Labels:
-
Apache NiFi
03-04-2025
08:02 PM
Hi Matt, Thank you so much for the solution you provided and you saved my day. It is working fine. Thanks Vijay
... View more
03-03-2025
11:08 PM
Hi All, I have created my processor group in a local instance of NiFi 2.0.0-M4 and I have downloaded the json definition file and also I can able to import that json manually to a target nifi system (production), but my ultimate goal is to deploy this json using NiFi RestAPI and I have accomplished the following tasks 1. Generate the NiFi Token 2. Create a ProcessorGroup to the target NiFi system But I could not find a way to import the json definition file through REST API and when I search on the apache NiFi documentation of API and in could not find any proper api call for NiFi version 2.0.0-M4 but there is only for version 1.28 supported Api endpoints for template concepts but as of NiFi version 2.0.0-M4 it was changed from xml template into json definition concept. Now how do I import the json file to the target system processGroup which i created through rest api. Please anyone can give your suggestion asap. Thanks Vijay.AX
... View more
Labels:
- Labels:
-
Apache NiFi