Support Questions
Find answers, ask questions, and share your expertise

how to configure mergecontent processor

Solved Go to solution

how to configure mergecontent processor

Expert Contributor

Hi All,

I have json data in multiple small files (some times only one line in a file).

I want to merge all small files in to single large file.

I am getting a large file in an unexpected format.

ex:

file 1:
{"code"="1", "color"="green"}
{"code"="2", "color"="blue"}
{"code"="3", "color"="orange"}

file 2:
{"code"="4", "color"="yellow"}
{"code"="5", "color"="red"}
I am getting the below output after using MergeContent
{"code"="1", "color"="green"}
{"code"="2", "color"="blue"}
{"code"="3", "color"="orange"}{"code"="4", "color"="yellow"}
{"code"="5", "color"="red"}
Expected output
{"code"="1", "color"="green"}
{"code"="2", "color"="blue"}
{"code"="3", "color"="orange"}
{"code"="4", "color"="yellow"}
{"code"="5", "color"="red"}
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: how to configure mergecontent processor

2 REPLIES 2
Highlighted

Re: how to configure mergecontent processor

Highlighted

Re: how to configure mergecontent processor

Expert Contributor

@Bryan Bende

I have checked my data, it has no new blank spaces, but was arriving like batches.

I am merging the files and appending using puthdfs.

When I use the configurations you suggested, some times I am getting a new blank line at the beginning of the file which is appended using puthdfs.

Can you please help me how to avoid the blank line at the beginning of the file, also the file is big (1GB).

Don't have an account?