- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Split Objects inside File - Apache Nifi
- Labels:
-
Apache NiFi
Created on ‎09-17-2020 10:38 AM - edited ‎09-17-2020 12:33 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello everyone, everything good?
So, I send mine website's clickstream events to Kinesis Data Stream, and Kinesis Firehose saves in a file without extension concatenating objects(events), for example:
{obj1}{obj2}{obj3}{obj4}{obj5}
I've already done all the pipeline to receive 1 Object and with JoltTransformJson I handle it,
in this case, how would I split the objects in this file? And is this the best approach in case of Performance?
I tried with Split Content with these configurations:
But the first object is coming out {obj1}{ and the rest of the objects are coming out obj2}{ obj3}{
Thank you guys!
Created ‎09-17-2020 11:54 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
one way to do it is to use the ReplaceText processor to add a line break after each object, and so having one line for each object in your FlowFile. To do so just replace "}{" by "} -linebreak- {".
Note that you have to escape brackets in your Search Value :
\}\{
And that you have to use Shift+Enter in your Replacement Value to add the line break :
}
{
Then just use the SplitText processor with Line Count Split to 1 to split your input flow file into one flow file for each line.
Hope it helps 🙂
