python script to fetch files in NiFi

Thanks. Using examples from that cookbook, I could make it work.

Here is my code which works:

from org.apache.commons.io import IOUtils
from java.nio.charset import StandardCharsets
from org.apache.nifi.processor.io import OutputStreamCallback

class PyOutputStreamCallback(OutputStreamCallback):
  def __init__(self):
        pass
  def process(self, outputStream):
    with open("D:\\Work\\nifi test\\custom processor input\\random_json.json") as f:
        file_content = f.read()
    outputStream.write(bytearray(file_content.encode('utf-8')))

flowFile = session.create()
if(flowFile != None):
    flowFile = session.write(flowFile, PyOutputStreamCallback())
    flowFile = session.putAttribute(flowFile, "filename", 'input_file.json')

session.transfer(flowFile, REL_SUCCESS)
session.commit()

Next, I will figure out if I can add the local directory path as a property and read that, instead of hardcoding it in the script.

As part of a POC, we want to explore the capabilities of a custom script to fetch files from a source, that pre-existing processors cannot. As a starting step, we are trying to try out writing a script that can fetch file similar to GetFile, to get some basic idea of how that would work. Then we will try out more complicated sources, of which nifi perhaps doesn't provide support by default.