Member since
06-27-2019
9
Posts
1
Kudos Received
0
Solutions
05-07-2024
01:30 AM
1 Kudo
Thanks @MattWho for the quick response. The short version of "why does it doesn't work" means is that nifi keeps working, recognizes the addition of the NARs but no "1.24" versions of components are available in the Processor selection window. I promptly tried the "M1" download workaround and found this as soon as I wget'ed the NAR: text version of the above: INFO Found ./extensions/nifi-standard-shared-nar-2.0.0-M1.nar in auto-load directory
INFO Starting load process for 1 NARs...
INFO Creating class loaders for 1 NARs...
WARN Unable to resolve required dependency 'nifi-standard-services-api-nar'. Skipping NAR '/usr/local/lib/nifi-2.0.0-M2/./work/nar/extensions/nifi-standard-shared-nar-2.0.0-M1.nar-unpacked'
INFO Successfully created class loaders for 0 NARs, 1 were skipped
INFO Finished NAR loading process! so, since it asks for nifi-standard-shared, I'll download it also and: text version: INFO Found ./extensions/nifi-standard-services-api-nar-2.0.0-M1.nar in auto-load directory
INFO Starting load process for 1 NARs...
INFO Including 1 previously skipped bundle(s)
INFO Creating class loaders for 2 NARs...
WARN While loading 'org.apache.nifi:nifi-standard-services-api-nar:2.0.0-M1' unable to locate exact NAR dependency 'org.apache.nifi:nifi-jetty-bundle:2.0.0-M1'. Only found one possible match 'org.apache.nifi:nifi-jetty-bundle:2.0.0-M2'. Continuing...
INFO Loaded NAR file: /usr/local/lib/nifi-2.0.0-M2/./work/nar/extensions/nifi-standard-services-api-nar-2.0.0-M1.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-services-api-nar-2.0.0-M1.nar-unpacked]
INFO Loaded NAR file: /usr/local/lib/nifi-2.0.0-M2/./work/nar/extensions/nifi-standard-shared-nar-2.0.0-M1.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-shared-nar-2.0.0-M1.nar-unpacked]
INFO Successfully created class loaders for 2 NARs, 0 were skipped EDIT: I also downloaded the jetty package, but: INFO Starting load process for 1 NARs...
ERROR Found a Jetty NAR, will not auto-load /usr/local/lib/nifi-2.0.0-M2/./extensions/nifi-jetty-bundle-2.0.0-M1.nar
INFO No NARs were unpacked, nothing to do so, it looks like it's acknowledging the NARs I provide, but after reloading the interface: You've been very helpful (and quick) providing the causes and several workarounds (I'll go the "build your URL on the fly" way), so thank you a lot and my double "accept as solution" click 🙂
... View more
05-06-2024
09:30 AM
Hi @MattWho! I tried to download the 1.24 InvokeHTTP to my M2 but doesn't seem to work: While loading 'org.apache.nifi:nifi-standard-shared-nar:1.24.0' unable to locate exact NAR dependency 'org.apache.nifi:nifi-standard-services-api-nar:1.24.0'. Only found one possible match 'org.apache.nifi:nifi-standard-services-api-nar:2.0.0-M2'. Continuing... then: Loaded NAR file: /usr/local/lib/nifi-2.0.0-M2/./work/nar/extensions/nifi-standard-shared-nar-1.24.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-shared-nar-1.24.0.nar-unpacked] then: Loaded extensions for org.apache.nifi:nifi-standard-shared-nar:1.24.0 in 23 millis and finally: Found existing bundle with coordinate org.apache.nifi:nifi-standard-shared-nar:1.24.0, will not load /usr/local/lib/nifi-2.0.0-M2/./extensions/nifi-standard-shared-nar-1.24.0.nar Downloading also the mentioned nifi-standard-services-api-nar logs this: org.apache.nifi.nar.NarAutoLoaderTask Found ./extensions/nifi-standard-services-api-nar-1.24.0.nar in auto-load directory and: Starting load process for 1 NARs...
Creating class loaders for 1 NARs...
While loading 'org.apache.nifi:nifi-standard-services-api-nar:1.24.0' unable to locate exact NAR dependency 'org.apache.nifi:nifi-jetty-bundle:1.24.0'. Only found one possible match 'org.apache.nifi:nifi-jetty-bundle:2.0.0-M2'. Continuing...
Loaded NAR file: /usr/local/lib/nifi-2.0.0-M2/./work/nar/extensions/nifi-standard-services-api-nar-1.24.0.nar-unpacked as class loader org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-services-api-nar-1.24.0.nar-unpacked]
Successfully created class loaders for 1 NARs, 0 were skipped
Loaded extensions for org.apache.nifi:nifi-standard-services-api-nar:1.24.0 in 16 millis
Finished NAR loading process! any ideas how this could actually be done? Thanks in advance!
... View more
01-25-2024
07:01 AM
if Google sends you here, most probably you're looking for https://community.cloudera.com/t5/Support-Questions/PUTSQL-which-cannot-be-converted-to-a-timestamp/m-p/292827
... View more
01-25-2024
06:54 AM
@joseomjr, I'm pretty sure it's a component that has been around from some time: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.6.0/org.apache.nifi.processors.standard.ConvertJSONToSQL/index.html
... View more
01-13-2023
04:12 AM
Even though you could send 5 at a time, you cannot wait for any of them (e.g., sequentilally) for allowing the next batch to be sent, at least that I know, using only this processor. I'd play with the idea of routing any response of this processor (retry, fail, success?) to a RouteOnAttribute processor that evaluates a flag for governing the InvokeHTTP, or better yet, use the Wait/Notify processors as explained in https://community.cloudera.com/t5/Support-Questions/Retrieve-Value-of-Signal-Counter-in-Wait-Notify-Processes/m-p/301917
... View more
04-26-2022
07:01 AM
Thanks for your replies @steven-matison!!! great help indeed. Anyway, a little more detail on the 3rd point please: Run Schedule sets how long a process will operate before a new instance is necessary I have this setting in other processors for marking the frequency of "hey, start working!" for each processor. In this InvokeHTTP case, having the specific configuration of "5 concurrent tasks" and "run schedule 0.2 sec", my doubt is: will it wait 0.2s between request and request, on each thread? Since you just explained that >1 task means "don't even wait", I guess that this setting will make the processor to send away 5 requests every 0.2s, so 25 requests per second, without even caring about if the remote server replied or not. Is this the case? Otherwise, if concurrent tasks were set to 1, then the processor would wait for a response, THEN wait 0.2s, and THEN send away the next one? My confusion comes from your "how long a process will operate" versus "how long will it wait after processing the previous flowfile. Thanks again!
... View more
04-25-2022
05:40 AM
Hi. I haven't found any documentation for these internals about the InvokeHTTP processor in NiFi. My requirement is to throttle requests to and endpoint (which we also control) that will accept, let's say, 5 simultaneous connections in an address and port, but I need to wait for each of their responses before letting another flowfile to be sent, so: Does this translate to simply using 1 InvokeHTTP processor configured to 5 "Concurrent Tasks" and that's it? Will the processor wait for the remote endpoint's request before sending the next one? How does the "Run Schedule" works together with the previous settings? (if I had, e.g.: 1 sec) I've been proposed with splitting the incoming queue and put 5 InvokeHTTP processors in parallel, each one attending 1/5 of the incoming flowfiles (I'd do the pre-partitioning before with some RouteOnAttribute trick), but I think it's exactly the same outcome as the 1. above. Is it? This all boils down to not knowing exactly how this processor work under the hood. Any insight would be much appreciated, thanks in advance!
... View more
Labels:
- Labels:
-
Apache NiFi