Member since
06-27-2019
7
Posts
0
Kudos Received
0
Solutions
01-25-2024
07:01 AM
if Google sends you here, most probably you're looking for https://community.cloudera.com/t5/Support-Questions/PUTSQL-which-cannot-be-converted-to-a-timestamp/m-p/292827
... View more
01-25-2024
06:54 AM
@joseomjr, I'm pretty sure it's a component that has been around from some time: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache.nifi.processors.standard.ConvertJSONToSQL/index.html
... View more
01-13-2023
04:12 AM
Even though you could send 5 at a time, you cannot wait for any of them (e.g., sequentilally) for allowing the next batch to be sent, at least that I know, using only this processor. I'd play with the idea of routing any response of this processor (retry, fail, success?) to a RouteOnAttribute processor that evaluates a flag for governing the InvokeHTTP, or better yet, use the Wait/Notify processors as explained in https://community.cloudera.com/t5/Support-Questions/Retrieve-Value-of-Signal-Counter-in-Wait-Notify-Processes/m-p/301917
... View more
04-26-2022
07:01 AM
Thanks for your replies @steven-matison!!! great help indeed. Anyway, a little more detail on the 3rd point please: Run Schedule sets how long a process will operate before a new instance is necessary I have this setting in other processors for marking the frequency of "hey, start working!" for each processor. In this InvokeHTTP case, having the specific configuration of "5 concurrent tasks" and "run schedule 0.2 sec", my doubt is: will it wait 0.2s between request and request, on each thread? Since you just explained that >1 task means "don't even wait", I guess that this setting will make the processor to send away 5 requests every 0.2s, so 25 requests per second, without even caring about if the remote server replied or not. Is this the case? Otherwise, if concurrent tasks were set to 1, then the processor would wait for a response, THEN wait 0.2s, and THEN send away the next one? My confusion comes from your "how long a process will operate" versus "how long will it wait after processing the previous flowfile. Thanks again!
... View more
04-25-2022
05:40 AM
Hi. I haven't found any documentation for these internals about the InvokeHTTP processor in NiFi. My requirement is to throttle requests to and endpoint (which we also control) that will accept, let's say, 5 simultaneous connections in an address and port, but I need to wait for each of their responses before letting another flowfile to be sent, so: Does this translate to simply using 1 InvokeHTTP processor configured to 5 "Concurrent Tasks" and that's it? Will the processor wait for the remote endpoint's request before sending the next one? How does the "Run Schedule" works together with the previous settings? (if I had, e.g.: 1 sec) I've been proposed with splitting the incoming queue and put 5 InvokeHTTP processors in parallel, each one attending 1/5 of the incoming flowfiles (I'd do the pre-partitioning before with some RouteOnAttribute trick), but I think it's exactly the same outcome as the 1. above. Is it? This all boils down to not knowing exactly how this processor work under the hood. Any insight would be much appreciated, thanks in advance!
... View more
Labels:
- Labels:
-
Apache NiFi