Hello All We are trying to generate trend of hdfs usage / consumption This way, we can plan when to add capacity. Is there anyway we can get data for the last 6 months or so. This will help us in forecasting ... View more

hello All We are analyzing all the long running queries/application ids from the history server. AS on the history server, The hive queries are not completed posted there, so we pulled the application id /job id from there but when we tried to lookup on tez view, for the same application id, the record was not found. it seems tez view, at our end, purges data in few hours itself. it shows only the queries that completed just now. is there a way to increase the retention of queries that are visible on tez view also, where else can we find the queries that are running for long time. We are checking the very next day but not finding anything. Any pointers will help. ... View more

We are daily copying 100+ tables/ data between production cluster and DR cluster. these tables are growing 0.5-1% daily. The tables that used to take 5 mins few months back are now taking 15 minutes to copy. We understand volume growth can cause performance issues but not to this drastic extend. Collectively, this is causing lot of delay. Copying entire database is now taking 3-4 more hours Can you please suggest how can we improve performance of the copy operation ... View more

We are trying to find the top 10 queries in hive that use lot of resources ( memory CPU etc ) we go this data using history server API and using RM but we only have the application id from the above data and do not have the hive query that we need to work on. can you please help me to meet this use case. ... View more