Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to Iterate the Flow file in Nifi using Python

Highlighted

How to Iterate the Flow file in Nifi using Python

New Contributor

HI All, I am trying to iterate the nifi flow file and modify each line in the flow file using python script in 'ExecuteScript'.But it is not working as expected. For example i would like to append 'Rajiv' string at the end for each line in the flow file.but it is adding only at the end of the end of the flow file.

Code109388-nifi.jpg

Execute script content:

import json

import java.io

from org.apache.commons.io import IOUtils

from java.nio.charset import StandardCharsets

from org.apache.nifi.processor.io import StreamCallback


class PyStreamCallback(StreamCallback):

def __init__(self, flowfile):

self.ff = flowfile

pass

def process(self, inputStream, outputStream):

text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)

text = text + 'rajiv'

outputStream.write(bytearray(text.encode('utf-8')))


flowFile = session.get()

if (flowFile != None):

flowFile = session.write(flowFile,PyStreamCallback(flowFile))

session.transfer(flowFile, REL_SUCCESS)


Generate flow file input:

1234

5678

output:

1234

5678rajiv

Expected output:

1234rajiv

5678rajiv


3 REPLIES 3

Re: How to Iterate the Flow file in Nifi using Python

New Contributor

Please let me know if the question is not clear. I will re-write the question again.

Re: How to Iterate the Flow file in Nifi using Python

New Contributor

Hi,

try :

text = text.replace('\n','rajiv') 

instead of  

text = text + 'rajiv'

 

maddalaus

Re: How to Iterate the Flow file in Nifi using Python

Cloudera Employee

Hi @rajivswe_2k7,

 

Alternatively, you can run your FlowFile through a ReplaceText processor and configuring it with replacement value of "ravi" / replacement strategy of "append" and Evaluation mode of "Line-by-line"

 

Screen Shot 2019-10-11 at 10.55.31 AM.png