start/stop processor via nifi api

[#1]

Hi,

I'm trying to start and stop processors via the nifi api version 1.0.0. I'm getting status code 409 returned and the message is "afb7dcf1-0157-1000-9450-ea0931a67e0f is not stopped."

I have read previous articles about the optimistic locking and I am supplying the version and client id but I'm still getting this error. Any ideas? Here is a snippet of my python code:

process_group = nifiapi.find_process_group(process_group_name)
if process_group is None:
    return None
processors = nifiapi.get_processors_by_pg(process_group["id"])
if processors is None:
    return None
for processor in processors['processors']:
    processor["revision"]["clientId"] = str(uuid.uuid4())
    processor["status"]["runStatus"] = "STOPPED"
    logging.debug("Updating processor: {}".format(json.dumps(processor)))
    nifiapi.update_processor(processor) # this makes the put request

[#2 · Accepted solution]

ok I figured it out. Since I was taking the processor object returned from my 'get' call and just modifying a few fields, it thought I was trying to do an update on the object. You cannot do this while it is running so that was the error message. I modified my request to only contain the minimum number of fields (i think) to stop the processor. I'm still unclear whether I need to set status.runStatus and/or component.state to STOPPED to get what I want as they both seem to indicate the same thing. Anyway, the below request works:

modified_processor = {
    'revision': {
        'version': processor["revision"]["version"],
        'clientId': str(uuid.uuid4())
    },
    'status': {
        'runStatus': status
    },
    'component': {
        'id': processor['id'],
        'state': status
    },
    'id' : processor['id']
}

[#3]

It gave JSON syntax error, with double quotes - worked fine.

{ "revision": { "version": 8, "clientId": "test" }, "status": { "runStatus": "RUNNING" }, "component": { "id": "8bb725ef-0158-1000-478b-da5903184807", "state": "RUNNING" }, "id": "8bb725ef-0158-1000-478b-da5903184807" }

But sometimes the GET request gets timed out - is the nifi-api service reliable?

[#4]

@ Frank Maritato Is there a way to start and stop processors using nifi - api??

Can you please elaborate with the endpoints to start/stop processors?

[#5]

Sure! The endpoint for updating a processor is a PUT request against /processors/[processor_id] where you would post json similar to what I have above. The runStatus should be either "STOPPED" or "RUNNING". You have to include the clientId or the version in the request. The 'component' field is also required. I discovered all this by reading the code since there really isn't a user guide on how to use these endpoints.

As part of my script, I was getting to the list of processors by the name of the process group. So my flow is:

GET /flow/search_results?q=[process_group_name]

Then I get a list of processors for that group with:

GET /process-groups/[process_group_id]/processors

I loop through each of those results with the PUT request I mentioned above. Hope this helps!

[#6]

Thanks for the reply @Frank Maritato

I tried using Put request against /processors/"processor_id" and the json is

{ "component":{ "id":"5444685a-0158-1000-0407-0bd70a8688ef" }, "status":{ "runStatus":"STOPPED" } }.

But got an error saying "Revision must be specified." What is Revision? Also , you specified that clientID or version should be included. Can you please define these? and/or From where we can get these values?

Thanks

[#7]

You will need to do a Get on that processor first. The revision/version is contained in that json and you just pass it thru for your Put request.

GET /processors/{id}

returning json will have something like this:

{
  "status": { ... },
  "component": { ... },
  "revision": {
    "version": 1
  }
}

So you would just take that whole revision element and add it to your Put payload.

clientId is any string that identifies your tool/app. In my example above, I was just generating a random uuid.

Hope this helps!

[#8]

@Frank Maritato Thanks for the Information.

[#9]

@Frank Maritato I am trying to start a processor group.Here is the command

curl -i -X PUT -H 'Content-Type:application/json' -d '{"component": {"id":"<processorgroup_id>","parentGroupId":"<parentgroup_id>","state":"RUNNING"},"revision": {"clientId":"<client_id>","version": 2}}' http://hostname:port/nifi-api/process-groups/>

I am facing:

HTTP/1.1 400 Bad Request Date: Tue, 20 Dec 2016 06:43:43 GMT Content-Type: text/plain Transfer-Encoding: chunked Server: Jetty(9.3.9.v20160517) Message body is malformed. Unable to map into expected format.

is there any issue in above api request??

[#10]

@Gayathri Rajendran My processor update json looks like this:

{
  "status": {
    "runStatus": "RUNNING"
  },
  "component": {
    "state": "RUNNING",
    "id": "01571000-f7fb-1da4-7d70-b6be89100354"
  },
  "id": "01571000-f7fb-1da4-7d70-b6be89100354",
  "revision": {
    "version": 1,
    "clientId": "c4cd2dc5-a57f-4025-aa4b-7e29118ce795"
  }
}

You may need to add the separate "status" object in there. Also, I've never specified a parentGroupId so I'm not sure if that is necessary.

Lastly, you may need to add a -H 'Accept:application/json' to your curl command as well.

Hope this helps!

[#11]

@Frank Maritato Thanks for the response

Here is my api call, I have updated my call like the above format. Even i am facing the same issue

curl -i -X PUT -H 'Content-Type:application/json' -H 'Accept: application/json' -d '{"status": {"runStatus": "RUNNING"},"component":{"state":"RUNNING","id":"<processorgroup_id>"},"id": "<processorgroup_id>", "revision": {"version": 2,"clientId":"<client_id>"}}' http://host:port/nifi-api/process-groups/>

HTTP/1.1 400 Bad Request Date: Wed, 21 Dec 2016 11:08:27 GMT Content-Type: text/plain Transfer-Encoding: chunked Server: Jetty(9.3.9.v20160517) Message body is malformed. Unable to map into expected format

[#12]

is there a way to start stop processors group?

[#13]

No, not from what I was able to find. If you want to start all processors in a processor group, you need to get the list of all processors in that group by calling

GET /process-groups/{id}/processors

and then one by one set them to RUNNING or STOPPED by doing

PUT /processors/{id}

Hope this helps!

[#14]

@Sanjeev Verma

yes its possible, pls see https://community.hortonworks.com/questions/69905/can-i-start-a-processgroup-using-rest-api.html#ans...

[#15]

Hi all,

any new ? somebody has the correct syntax of json for running a processor ?

i've used this json with HTTP CODE 405

# more startListHDFS_1.json
{
  "status": {
    "runStatus": "RUNNING"
  },
  "component": {
    "state": "RUNNING",
    "id": "b0c63454-89a8-15ef-be22-959bd4a72e5a"
  },
  "id": "b0c63454-89a8-15ef-be22-959bd4a72e5a",
  "revision": {
    "version": 1,
    "clientId": "test"
  }
}
# curl -H "Content-type: application/json" --cacert nifi-cert.pem --cert ./client.p12 -XPOST https://localhost:9443/nifi-api/processors/b0c63454-89a8-15ef-be22-959bd4a72e5a -d @startListHDFS_1.json -vv

help appreciate. thanks

[#16]

Hi all,

for me it is working fine :

HTTP method : PUT

{ "status": { "runStatus": "STOPPED" }, "component": { "state": "STOPPED", "id": "b2e9b4eb-015b-1000-765d-c01c0a2fa6ce" //processor id }, "id": "b2e9b4eb-015b-1000-765d-c01c0a2fa6ce", //processor id "revision": { "clientId": "b2e5a8bb-015b-1000-31c1-0ee3b952dcbe", "version": 8 } }

[#17]

in which processor we need to use? how to specify property of that processor?

[#18]

@Frank Maritato Can i run your python code in ExecuteScript processor - which means i am stopping an processor from another one within nifi. If yes? will your code work if security for nifi is enabled like LDAP authentication. Can you show light on "nifiapi" is it an api provided by nifi.