About samrathal

samrathal · ‎10-18-2023

I need to retrieve all flowfiles in a specific queue in NiFi, but the API only returns up to 100 results at a time. My queue contains 358 flowfiles, so I need a way to retrieve all of them Below are the API am using for get the flowfiles. API : https://myserver.com:myport/nifi-api/flowfile-queues/ba619122-3c65-3279-a2ba-b3ad89f8a429/listing-requests/903d485f-d4aa-102a-0000-0112e4a15ea { "listingRequest": { "id": "903d485f-d4aa-102a-0000-0112e4a15ea", "uri": "https://myserver.com:myport/nifi-api/flowfile-queues/ba619122-3c65-3279-a2ba-b3ad89f8a429/listing-requests/903d485f-d4aa-102a-0000-0112e4a15ea", "submissionTime": "10/18/2023 17:07:57.632 IST", "lastUpdated": "17:07:57 IST", "percentCompleted": 1, "finished": true, "maxResults": 100, "state": "Completed successfully", "queueSize": { "byteCount": 3792, "objectCount": 350 }, "flowFileSummaries": [ { "HERE IS THE FLOW FILES TILL 100TH POSITIONS": 100 } ] } }

samrathal · ‎08-03-2023

I have a usecase, where i have to read file from the remote windows server, my NiFi is deployed on linux machine and I have tried to use GetSFTP, FetchSFTP and ExecuteProcess but didn't able to even reach the server.

samrathal · ‎02-03-2023

Hi Everyone, the API would be : GET https://<URL>/nifi-api/flowfile-queues/<id>/flowfiles/<flowfile-uuid>/content?clusterNodeId=<clusterNodeId> You'll get the content of the FlowFile in the queue.

samrathal · ‎02-02-2023

@steven-matison Yes, I have the clusterNodeId, but not understand where to fit in this API. Can you please share the dummy API

samrathal · ‎02-02-2023

Hi, I have a use case, where i have to see the content of each FlowFiles by triggering the API call. API i have explored so far : https://<URL>/nifi-api/flowfile-queues/<Queue-ID>/flowfiles/<FlowFile-UUID>/content I got the response as below : "The id of the node in the cluster is required." Note : My NiFi is setting up in three node cluster. Please suggest.

samrathal · ‎12-21-2022

@MattWho Thanks for the response, Below is the inline comments and my scenario. 1. Is this a one time execution flow? -Not 2. if not, how do you differentiate between different complete batches (when does new one merge bundle end and another begin?)? - I'm calling the same rest-api with the record limit of 1000, and let suppose my attribute "X-Total-Count" is 5600 (record), then am going to call the rest-api 6 times. (1000*6) to capture the complete record. so till reach to 5600 records i want "Wait" processor before merge it all. 3. Are all 1000 records from each rest-api call going into a single NiFi FlowFile or 1 FlowFile per record? -One FlowFile per record, as after "invokeHTTP" am using "splitjson". 4. Is there some correlation identifier as a rest of rest-api call that identifies all 1000 Record batch pulls as part of same complete bundle? -i'm using offset for this. What I have : rest-api with limit set to 1000, total number of records is not always same, capturing from "X-Total-Count". What am looking for : Before merge the whole data, i want "Wait" processor for waiting the completion of API (api could be run 2 times or 15times) after getting the whole data that "FlowFile" (fragment.count) is equals to "X-Total-Count" i.e ${fragment.count:equals(${X-Total-Count})}. then it notify to pass all data "MergerRecord" and a single FlowFile would generates.

samrathal · ‎12-21-2022

I have a scenario where am triggering API in batches (1000 records in each). from the first API I get the value of total records in attribute list X-Total-Count, i wants to merge them all. Please let me know how i can leverage the wait processor in it to wait until we get the whole data till value of X-Total-Count. Wait Processor properties, i have configured.

samrathal · ‎11-13-2022

Hi, I have a use, where i need to upload the processor group (.json) file from the curl / API. Please suggest. OR can we do this operation in NiFi CLI without using NiFi registry.

samrathal · ‎10-11-2022

Hi Team, Am using self signed certificates, for API calling but while import the configuration passwords are not stick with the properties, can someone please suggest a method, How we can add truststore and keystore password using NiFi CLi or via API call to SSL ContextService.

samrathal · ‎09-05-2022

@araujo No, I'm getting the same current time in "timeOneMinu", Subtraction is not performing.

Online	Offline
Last Visited	‎07-23-2024 08:19 AM

Member Since	‎07-28-2022 05:57 AM
Last Visited	‎07-23-2024 08:19 AM
Posts	25
Kudos received	3

Cloudera Community

Re: How can i see the Content of Flowfiles in queu...

How to retrieve a complete list of flowfiles in a ...

How can we get or read file from a remote windows ...

Re: How can i see the Content of Flowfiles in queu...

Re: How can i see the Content of Flowfiles in queu...

How can i see the Content of Flowfiles in queue us...

Re: How to apply wait processor for capture comple...

How to apply wait processor for capture complete r...

How to upload .json processor group using curl / A...

How to add truststore and keystore password using ...

Re: How to decrease Epoch time length and perform ...