Support Questions
Find answers, ask questions, and share your expertise

Removing what appear to be old HortonWorks YARN Application Logs....

New Contributor

Hi,

 

I'm new to HortonWorks and have "inherited" the support of a small HDP 2.6.5 cluster that runs in AWS (I've got 6 years experience with Cloudera 5.x and 6.1 clusters, just not HortonWorks HDP and I have been given absolutely no documentation on this HDP cluster).

 

I was just wondering if anyone out there in the community could confirm whether the location in HDFS "/app-logs/hdfs/logs-ifile/" is where HDP keeps old YARN logs for applications that have executed successfully ?

 

Any advice would be greatly appreciated.

 

I'd like to remove these old logs....but I'm not 100% confident in doing so 😞

 

As per below, there are 10's of thousands of YARN logs dating back to 2019 and 2020 that I would like to delete and free up ~300GB of space.

 

[{hostname}]$ hdfs dfs -ls /app-logs/hdfs/logs-ifile/ | grep 2019 | wc -l

47873

 

[{hostname}]$ hdfs dfs -ls /app-logs/hdfs/logs-ifile/ | grep 2020 | wc -l

88602

 

[{hostname]$ hdfs dfs -ls /app-logs/hdfs/logs-ifile/ | head -5

 

drwxrwx---   - hdfs hadoop          0 2019-02-15 19:33 /app-logs/hdfs/logs-ifile/application_1549062956375_0694

drwxrwx---   - hdfs hadoop          0 2019-02-15 19:33 /app-logs/hdfs/logs-ifile/application_1549062956375_0812

drwxrwx---   - hdfs hadoop          0 2019-02-15 19:33 /app-logs/hdfs/logs-ifile/application_1549062956375_0899

drwxrwx---   - hdfs hadoop          0 2019-02-15 19:33 /app-logs/hdfs/logs-ifile/application_1549062956375_1066

drwxrwx---   - hdfs hadoop          0 2019-02-15 19:33 /app-logs/hdfs/logs-ifile/application_1549062956375_1178

.....

.....

 

 

 

Kind Regards,

 

Damion

0 REPLIES 0
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.