I can see you have already partitioned your tables. One important thing is if you are using hive you need to make sure that the tables are partitioned on the column where most of your data is clubbed. So the date Column in my suggestion is the best option here.  Also, for those who are new to hive/hdfs, multiple files are created when you load data in hive tables. so don't get confused just pick the right partition and your files will be reduced in hdfs where hive data is being saved.  ... View more

Just tried repo.hortsonwork.com, it is accessible.  I have noticed that most of the links other than nexus-private.hortonworks.com and dev.hortonwork.com.s3.awsamazon.com are no more accessible as all the public REPO links are no longer private. Even the new links shared by the CDH/HDP require credentials to get the repos.   Still looking for an alternative. if anyone has found any solution, please guide us the way forward. ... View more