Skip to main content
mtigua
mtigua
Silver ⭐️
May 3, 2019
Solved

How to initialize a processor via NIFI API

Hi everyone, please help!

I'm using ListSFTP processor to get files and process them. In the first execution the processor take the files normally, but when you start the processor the second time it doesn't. I know this is the correct behavior for this processor. Is there any way to start and initialize the processor via NIFI API in such a way that It takes the files again? I need this approach because I need to do testing.

Thanks!

Wynner
WynnerAnswer
Platinum ⭐️
May 5, 2019

@Mario Tigua

You can use the REST api to clear state on the processor. You can get the curl command to do this using the Developer tools available in the Chrome browser. Then pass that command, in a script, to a ExecuteStreamCommand processor that is triggered after the files are fetched and clear the state of the ListSFTP processor. For example:

This simple flow shows how you could list the files, fetch them and then after they're fetched, use the results flow files to trigger the state being cleared in the ListSFTP processor:

108426-screen-shot-2019-05-06-at-92743-am.png

The configuration of the ExecuteStreamCommand processor looks like:

108427-screen-shot-2019-05-06-at-93005-am.png


The contents of the curl-script.sh file, of course you would fill in the appropriate host and port for your environment as well as the uuid of the processor.

#!/bin/bash

# curl command to stop processor
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c' -X PUT -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' --data-binary '{"revision":{"clientId":"8440f58c-016a-1000-f56a-4b8ca84fb669","version":30},"component":{"id":"84415a50-016a-1000-0000-00001257b02c","state":"STOPPED"}}' --compressed

# curl command to clear state
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c/state/clear-requests' -X POST -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'Content-Length: 0' --compressed


The curl command can be obtained by

108436-screen-shot-2019-05-06-at-93143-am.png

Then using the Developer tool, get the curl command when you manually clear the state of the processor

108517-screen-shot-2019-05-06-at-93457-am.png


This example was done with an insecure instance of NiFi, if NiFi is secure, you'll have to get a token to run the curl commands.

View in thread

3 replies

LesterMartin
Gold ⭐️
LesterMartin
Gold ⭐️
May 5, 2019

I didn't see such a property when I looked at http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.9.2/org.apache.nifi.processors.standard.ListFTP/index.html, but some quick solutions to jumpstart this could be to simply replace this processor with a new one (which will have its own state management) or if you using "timestamps" for the "Listing Strategy" property then you could always do a linux "touch" command on the files on the FTP server which should trick the processors to grab them again.

Good luck and happy Flowfiling!

Translate
Wynner
Platinum ⭐️
WynnerAnswer
Platinum ⭐️
May 5, 2019

@Mario Tigua

You can use the REST api to clear state on the processor. You can get the curl command to do this using the Developer tools available in the Chrome browser. Then pass that command, in a script, to a ExecuteStreamCommand processor that is triggered after the files are fetched and clear the state of the ListSFTP processor. For example:

This simple flow shows how you could list the files, fetch them and then after they're fetched, use the results flow files to trigger the state being cleared in the ListSFTP processor:

108426-screen-shot-2019-05-06-at-92743-am.png

The configuration of the ExecuteStreamCommand processor looks like:

108427-screen-shot-2019-05-06-at-93005-am.png


The contents of the curl-script.sh file, of course you would fill in the appropriate host and port for your environment as well as the uuid of the processor.

#!/bin/bash

# curl command to stop processor
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c' -X PUT -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' --data-binary '{"revision":{"clientId":"8440f58c-016a-1000-f56a-4b8ca84fb669","version":30},"component":{"id":"84415a50-016a-1000-0000-00001257b02c","state":"STOPPED"}}' --compressed

# curl command to clear state
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c/state/clear-requests' -X POST -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'Content-Length: 0' --compressed


The curl command can be obtained by

108436-screen-shot-2019-05-06-at-93143-am.png

Then using the Developer tool, get the curl command when you manually clear the state of the processor

108517-screen-shot-2019-05-06-at-93457-am.png


This example was done with an insecure instance of NiFi, if NiFi is secure, you'll have to get a token to run the curl commands.

Translate
    mtigua
    mtiguaAuthor
    Silver ⭐️
    May 7, 2019

    Thanks @Wynner, it works!

    Translate
    Terms & ConditionsAccessibility statement