Member since
08-17-2017
2
Posts
0
Kudos Received
0
Solutions
08-17-2017
01:43 PM
We have a requirement to concatenate huge number of small ORC files into small number of big ORC files, through this post I was encouraged to try it on our external table and it just worked fine. This is a test that we ran on our dev environment, we created 100 small ORC files and then used the concatenate command it turned them into 5 big files (next we'll try to figure out how to influence the number of files/size) Our config: HDP: 2.5.3
... View more
08-17-2017
08:38 AM
Hi @saravanan p Wondering if this might be of interest to you: I was able to concatenate an external table which originally had large number of small orc files into small number of huge orc files. I suppose it might work on internal tables as well Our HDP is: 2.5.3
... View more