We have a requirement to concatenate huge number of small ORC files into small number of big ORC files, through this post I was encouraged to try it on our external table and it just worked fine. This is a test that we ran on our dev environment, we created 100 small ORC files and then used the concatenate command it turned them into 5 big files (next we'll try to figure out how to influence the number of files/size) Our config: HDP: 2.5.3 ... View more

Hi @saravanan p Wondering if this might be of interest to you: I was able to concatenate an external table which originally had large number of small orc files into small number of huge orc files. I suppose it might work on internal tables as well Our HDP is: 2.5.3 ... View more