Support Questions
Find answers, ask questions, and share your expertise

Write to ADLS through Python Script in NIFI

Write to ADLS through Python Script in NIFI

All, I am trying to write contents oh HTTP response in JSON to ADLS. The JSON is modified for few fields using a Python script in before writing to ADLS. My idea is to once the JSON is modified, connect to ADLS through the Python script itself and write the data. I just wrote the script and am able to write the modified JSON response into my local disk. But I am not sure how can I modify the outputstream.write so that it can write to ADLS. Below is the code.

class PyStreamCallback(StreamCallback): def __init__(self): pass def process(self, inputStream, outputStream): text = IOUtils.toString(inputStream, StandardCharsets.UTF_8) obj = json.loads(text) outputStream.write(bytearray(json.dumps(obj).encode('utf-8')))

flowFile = session.get() if (flowFile != None): flowFile = session.write(flowFile,PyStreamCallback()) session.transfer(flowFile, REL_SUCCESS)

Any lead is appreciated as I am stuck. I have the credentials needed for connecting to Data Lake.