@nikkie_thomas You can set below if you are using Tez   set hive.merge.mapfiles=true; set hive.merge.mapredfiles=true; set hive.merge.smallfiles.avgsize=<some value>; set hive.merge.size.per.task=<some value>; set hive.merge.tezfiles=true; ... View more

Hi guys, I still don't get the point of specifying the variable while you provide entire path to the spark2 client. Could you please give me a reason for doing so? On HDP 2.6.2 I use there is enough to specify a path to appropriate spark client and then the version is chosen automatically. ... View more