how to configure mergecontent processor

[#1]

Hi All,

I have json data in multiple small files (some times only one line in a file).

I want to merge all small files in to single large file.

I am getting a large file in an unexpected format.

ex:

file 1:

{"code"="1", "color"="green"}
{"code"="2", "color"="blue"}
{"code"="3", "color"="orange"}

file 2:

{"code"="4", "color"="yellow"}
{"code"="5", "color"="red"}

I am getting the below output after using MergeContent

{"code"="1", "color"="green"}
{"code"="2", "color"="blue"}
{"code"="3", "color"="orange"}{"code"="4", "color"="yellow"}
{"code"="5", "color"="red"}

Expected output

{"code"="1", "color"="green"}
{"code"="2", "color"="blue"}
{"code"="3", "color"="orange"}
{"code"="4", "color"="yellow"}
{"code"="5", "color"="red"}

[#2 · Accepted solution]

I answered this on stackoverflow:

https://stackoverflow.com/questions/52188619/mergecontent-processor-is-not-giving-expected-result

[#3]

@Bryan Bende

I have checked my data, it has no new blank spaces, but was arriving like batches.

I am merging the files and appending using puthdfs.

When I use the configurations you suggested, some times I am getting a new blank line at the beginning of the file which is appended using puthdfs.

Can you please help me how to avoid the blank line at the beginning of the file, also the file is big (1GB).