Member since
09-29-2021
3
Posts
0
Kudos Received
0
Solutions
02-09-2022
05:26 AM
I'm developing a NIFI flow with multiple sub-flows. I need the ability to control the execution between sub-flows; i.e. FLOW C cant start until the last queue in FLOW B is empty, but it can't start until the last queue in FLOW A is empty.... The NIFI node is secured and interactions via web are authenticated using OpenId Connect SSO... I've read some support forums that describe capturing and passing a token when using the INVOKEHTTP processor. But these solutions employ external scripting, using CURL to perform token management. But I'm not sure how to incorporate those steps in my flow and I'm looking for some help if it's out there 🙂 I'm trying to use the NIFI REST API, which provides methods that will allow me to monitor and control flow execution. But when I attempt to make any such calls a NO RETRY response is returned and the invokehttp.response.body attribute value is: Unknown user with identity 'anonymous'. Contact the system administrator. The solution described here: https://community.cloudera.com/t5/Community-Articles/NiFi-REST-API-FlowFile-Count-Monitoring/ta-p/244208 Is the sort of thing I'm trying to accomplish. However authentication is probably not a problem here because it's HTTP not HTTPS. (edited)
... View more
Labels:
- Labels:
-
Apache NiFi
02-07-2022
07:02 AM
Can the same be done if the NIFI cluster is HTTPS? I've read some other posts that appear to use CURL, presumably in some scripted process, which then creates tokens that get passed in subsequent REST calls. But I need to be able to gather NIFI processor/queue status all within NIFI- no external scripting. Is this possible?
... View more
09-29-2021
03:42 AM
I've created a flow that includes a LookupRecord service which is not behaving as expected. The processor is configured to use a DatabaseLookupService. All work fine, so long as the key being looked up exists in the database. With RoutingStrategy set to "Route to matched or unmatched", I'm expecting a "no match" condition. But instead an ERROR is being thrown. This is preventing any flow file to be produced, and processing halts. Is this a big? If not, can someone explain the behavior and what's needed to trigger a "nomatch" instead? Thank You Nifi Version: 1.11.4 Database: SQLServer Debug log is shown below: \2021-09-29 06:20:02,506 ERROR [Timer-Driven Process Thread-9] o.a.n.processors.standard.LookupRecord LookupRecord[id=c05c15bf-147d-117b-d23f-57e27be472d3] Failed to process StandardFlowFileRecord[uuid=66461123-15cb-4132-a57c-dbb4512ea740,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1632910350773-2219, container=default, section=171], offset=965984, length=2093],offset=0,name=5d3ead60-facb-4ee2-8af1-c8c86da190da,size=2093]: org.apache.nifi.processor.exception.ProcessException: Failed to lookup coordinates {key=be623387-27ff-40a4-b3bb-6b7438877ad2APUALL} in Lookup Service org.apache.nifi.processor.exception.ProcessException: Failed to lookup coordinates {key=be623387-27ff-40a4-b3bb-6b7438877ad2APUALL} in Lookup Service at org.apache.nifi.processors.standard.LookupRecord.doResultPathReplacement(LookupRecord.java:395) at org.apache.nifi.processors.standard.LookupRecord.route(LookupRecord.java:303) at org.apache.nifi.processors.standard.LookupRecord.route(LookupRecord.java:68) at org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:134) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2324) at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2292) at org.apache.nifi.processors.standard.AbstractRouteRecord.onTrigger(AbstractRouteRecord.java:121) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.nifi.lookup.LookupFailureException: Error executing SQL statement: SELECT "Id" FROM "TABLE"."V_NIFI_Component" WHERE lookupkey = ?for value be623387-27ff-40a4-b3bb-6b7438877ad2XYZALL : ResultSet not positioned properly, perhaps you need to call next. at org.apache.nifi.lookup.db.DatabaseRecordLookupService.lookup(DatabaseRecordLookupService.java:172) at sun.reflect.GeneratedMethodAccessor639.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87) at com.sun.proxy.$Proxy229.lookup(Unknown Source) at org.apache.nifi.processors.standard.LookupRecord.doResultPathReplacement(LookupRecord.java:393) ... 18 common frames omitted
... View more
- Tags:
- LookupRecord
- NiFi
Labels:
- Labels:
-
Apache NiFi