Hello everyone, everything good?
So, I send mine website's clickstream events to Kinesis Data Stream, and Kinesis Firehose saves in a file without extension concatenating objects(events), for example:
{obj1}{obj2}{obj3}{obj4}{obj5}
I've already done all the pipeline to receive 1 Object and with JoltTransformJson I handle it,
in this case, how would I split the objects in this file? And is this the best approach in case of Performance?
I tried with Split Content with these configurations:
But the first object is coming out {obj1}{ and the rest of the objects are coming out obj2}{ obj3}{
Thank you guys!
Hello,
one way to do it is to use the ReplaceText processor to add a line break after each object, and so having one line for each object in your FlowFile. To do so just replace "}{" by "} -linebreak- {".
Note that you have to escape brackets in your Search Value :
\}\{And that you have to use Shift+Enter in your Replacement Value to add the line break :
}
{
Then just use the SplitText processor with Line Count Split to 1 to split your input flow file into one flow file for each line.
Hope it helps 🙂
