How to Start/Stop processor in Minifi

abyphil · ‎11-13-2018

I am using Minifi 0.5.0, and use FileChangeIngestor to update the dataflow yaml file. Is there a way to start/stop processors. I see answers on how to do the same with Nifi in this forum and i tried the equilvalent constructs in the yaml file before replacing the file (as shown below) but that has not helped.

- id: d8e5af9f-921e-3737-0000-000000000000
  status:
    runStatus: STOPPED
  component:
    state: STOPPED
    id:  d8e5af9f-921e-3737-0000-000000000000 
  name: TailFile
  class: org.apache.nifi.processors.standard.TailFile

Any help is appreciated

abyphil · ‎11-15-2018

I have some update. I think i have a workaround. Let me describe what i tried -

1. Using <state> - (failed)

I noticed that the template.xml file has a field <state>, which is not present in the corresponding yml file for Minifi. Possible values are - RUNNING, STOPPED, DISABLED

<name>TailFile</name>
<state>RUNNING</state>

I tried adding a corresponding field in the yml file file, manually

- id: 2438e3c8-015a-1000-79ca-83af40ec1991
  name: TailFile
  state: STOPPED

When the yml file was replaced with this, Minifi flowStatus indicated for a short while that, the processor(s) had indeed stopped

{"controllerServiceStatusList":null,"processorStatusList":[{"id":"0f84dc2f-c883-317f-0000-000000000000","name":"PutFile","processorHealth":{"runStatus":"Stopped","hasBulletins":false,"validationErrorList":[]},"processorStats":null,"bulletinList":null},{"id":"d8e5af9f-921e-3737-0000-000000000000","name":"TailFile","processorHealth":{"runStatus":"Stopped","hasBulletins":false,"validationErrorList":[]},"processorStats":null,"bulletinList":null}],"connectionStatusList":null,"remoteProcessGroupStatusList":null,"instanceStatus":null,"systemDiagnosticsStatus":null,"reportingTaskStatusList":null,"errorsGeneratingReport":[]}

However, subsequent flowStatus responses showed the processor status as Running. So i think, Minifi automatically starts all processors.

2. Use 'max concurrent tasks'=0 (failed)

I tried setting the max concurrent tasks for the processor as 0. But this did not seem to have an effect.

3. Change 'scheduling strategy' and 'scheduling period' (success)

I set the values as follows for the processor -

scheduling strategy: CRON_DRIVEN
  scheduling period: '0 0 0 1 * ?'

effectively scheduling the processor to run only on the first of every month. This effectively prevented the processor from running. When i subsequently, changed the values as

scheduling strategy: TIMER_DRIVEN
scheduling period: 1 sec

Minifi detected the changes, restarted and hence the processor started reading from where it left off.

I realize this is a hack, and would love to have a better solution, but thought i would update in case it is useful.

Any comments?