Support Questions
Find answers, ask questions, and share your expertise

Split Objects inside File - Apache Nifi

Split Objects inside File - Apache Nifi

New Contributor

Hello everyone, everything good?

So, I send mine website's clickstream events to Kinesis Data Stream, and Kinesis Firehose saves in a file without extension concatenating objects(events), for example:

{obj1}{obj2}{obj3}{obj4}{obj5}

 

I've already done all the pipeline to receive 1 Object and with JoltTransformJson I handle it,
in this case, how would I split the objects in this file? And is this the best approach in case of Performance?

 

I tried with Split Content with these configurations:

Ditadi_0-1600364030100.png

 

But the first object is coming out {obj1}{ and the rest of the objects are coming out obj2}{ obj3}{

 

Thank you guys!

1 REPLY 1

Re: Split Objects inside File - Apache Nifi

Explorer

Hello,

 

one way to do it is to use the ReplaceText processor to add a line break after each object, and so having one line for each object in your FlowFile. To do so just replace "}{" by "} -linebreak- {".

 

Note that you have to escape brackets in your Search Value :

\}\{

And that you have to use Shift+Enter in your Replacement Value to add the line break :

}
{

 

Then just use the SplitText processor with Line Count Split to 1 to split your input flow file into one flow file for each line.

 

flow.png

 

 

Hope it helps