Executing apache NIFI through a shell script

[#1]

I have designed a NIFI dataflow using 2 processors to take input from the local file system and load into Hadoop File system. I have the following queries with respect to the above:

1. How can I execute the data flow inside a shell script instead of executing it via the GUI?
2. Does apache NIFI provides guaranteed delivery. In case the job fails how the recovery is done? Do we need to restart the process from the start all over again? Say there are 100000 records in a file and the job fails after reading 20000 files.In this case once we restart does it execute the process from the beginning?

Regards,

I Roy

[#2]

• You can start a processor programatically using Nifi APIs - here is the link to the API documentation (specifically you will be interested in the updateProcessor API within the processors

PUT/controller/process-groups/{process-group-id}/processors/{id}    Updates a processor

• Nifi keeps track of the processed messages, so when the process restarts it starts from where it left.. also you have an option to terminate the flow file upon failure (by message) - and you can always inspect the messages using 'data provenance' feature, and replay the messages selectively, if the need be.

[#3]

Can you provide an example or any link where we are programming via NIFI API

[#4]

Hi @INDRANIL ROY,

Take a look at NiFi Api documentation below, you can control the processors and do a lot more:

https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

For examples to start stop and change configuration, check out my answer to below question in HCC

https://community.hortonworks.com/questions/41532/nifi-job-control-and-date-parameter.html#answer-41...

Thanks,

Jobin George