Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to initialize a processor via NIFI API

Solved Go to solution

How to initialize a processor via NIFI API

New Contributor

Hi everyone, please help!

I'm using ListSFTP processor to get files and process them. In the first execution the processor take the files normally, but when you start the processor the second time it doesn't. I know this is the correct behavior for this processor. Is there any way to start and initialize the processor via NIFI API in such a way that It takes the files again? I need this approach because I need to do testing.

Thanks!

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to initialize a processor via NIFI API

@Mario Tigua

You can use the REST api to clear state on the processor. You can get the curl command to do this using the Developer tools available in the Chrome browser. Then pass that command, in a script, to a ExecuteStreamCommand processor that is triggered after the files are fetched and clear the state of the ListSFTP processor. For example:

This simple flow shows how you could list the files, fetch them and then after they're fetched, use the results flow files to trigger the state being cleared in the ListSFTP processor:

108426-screen-shot-2019-05-06-at-92743-am.png

The configuration of the ExecuteStreamCommand processor looks like:

108427-screen-shot-2019-05-06-at-93005-am.png


The contents of the curl-script.sh file, of course you would fill in the appropriate host and port for your environment as well as the uuid of the processor.

#!/bin/bash

# curl command to stop processor
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c' -X PUT -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' --data-binary '{"revision":{"clientId":"8440f58c-016a-1000-f56a-4b8ca84fb669","version":30},"component":{"id":"84415a50-016a-1000-0000-00001257b02c","state":"STOPPED"}}' --compressed

# curl command to clear state
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c/state/clear-requests' -X POST -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'Content-Length: 0' --compressed


The curl command can be obtained by

108436-screen-shot-2019-05-06-at-93143-am.png

Then using the Developer tool, get the curl command when you manually clear the state of the processor

108517-screen-shot-2019-05-06-at-93457-am.png


This example was done with an insecure instance of NiFi, if NiFi is secure, you'll have to get a token to run the curl commands.

3 REPLIES 3

Re: How to initialize a processor via NIFI API

I didn't see such a property when I looked at http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.9.2/org.apache.... but some quick solutions to jumpstart this could be to simply replace this processor with a new one (which will have its own state management) or if you using "timestamps" for the "Listing Strategy" property then you could always do a linux "touch" command on the files on the FTP server which should trick the processors to grab them again.

Good luck and happy Flowfiling!

Re: How to initialize a processor via NIFI API

@Mario Tigua

You can use the REST api to clear state on the processor. You can get the curl command to do this using the Developer tools available in the Chrome browser. Then pass that command, in a script, to a ExecuteStreamCommand processor that is triggered after the files are fetched and clear the state of the ListSFTP processor. For example:

This simple flow shows how you could list the files, fetch them and then after they're fetched, use the results flow files to trigger the state being cleared in the ListSFTP processor:

108426-screen-shot-2019-05-06-at-92743-am.png

The configuration of the ExecuteStreamCommand processor looks like:

108427-screen-shot-2019-05-06-at-93005-am.png


The contents of the curl-script.sh file, of course you would fill in the appropriate host and port for your environment as well as the uuid of the processor.

#!/bin/bash

# curl command to stop processor
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c' -X PUT -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' --data-binary '{"revision":{"clientId":"8440f58c-016a-1000-f56a-4b8ca84fb669","version":30},"component":{"id":"84415a50-016a-1000-0000-00001257b02c","state":"STOPPED"}}' --compressed

# curl command to clear state
curl 'http://nifi-host:nifi-port/nifi-api/processors/84415a50-016a-1000-0000-00001257b02c/state/clear-requests' -X POST -H 'Origin: http://nifi-host:nifi-port'; -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://nifi-host:nifi-port/nifi/'; -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'Content-Length: 0' --compressed


The curl command can be obtained by

108436-screen-shot-2019-05-06-at-93143-am.png

Then using the Developer tool, get the curl command when you manually clear the state of the processor

108517-screen-shot-2019-05-06-at-93457-am.png


This example was done with an insecure instance of NiFi, if NiFi is secure, you'll have to get a token to run the curl commands.

Re: How to initialize a processor via NIFI API

New Contributor

Thanks @Wynner, it works!