Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Have Nifi's InvokeHTTP Processor execute script right before it POSTs

Highlighted

Have Nifi's InvokeHTTP Processor execute script right before it POSTs

New Contributor

One of our APIs requires an HMAC signature for an REST call.

Currently I have a ExecuteScript processor calling some python code to generate this.

This sets the sig in an attribute that can be used by the InvokeHTTP Processor.

In this case I'm calling an API call for each record in a result set I got earlier.

The execute script would get called for each record really fast, but the API calls are much slower. So each flowfile would get an HMAC signature, but that flowfile wouldn't get processed for a few minutes.

The server doesn't like old HMAC signatures. So if a flowfile was backed up behind the invokeHTTP for too long it would expire.

So what I need is the InvokeHTTP Processor to call the ExecuteScript processor as a sort of "preProcess" execute just before it executes.

What is the best way to accomplish this.

For now I've put a delay on both processors of 2 seconds so the ExecuteScript doesn't run ahead of the InvokeHTTP, but it doesn't work very well. It gets out of sync pretty fast and I end up with a queue of flowfiles that have the HMAC signature generated.

Thanks!

1 REPLY 1

Re: Have Nifi's InvokeHTTP Processor execute script right before it POSTs

On the connection between ExecuteScript and InvokeHttp, you could try setting the Back Pressure Object Threshold to 1. This says that ExecuteScript will not be scheduled when there is a flow file in the queue between the two. Then when the flow file is processed, ExecuteScript will be scheduled, and the flow file will be immediately transferred to the (empty) queue for InvokeHttp to process.