@shrinivas acharya
You can set the following properties inside the spark2-defaults to Specifies whether the History Server should periodically clean up event logs from storage. For more information on the same please refer to: https://spark.apache.org/docs/latest/monitoring.html
spark.history.fs.cleaner.enabled=true
spark.history.fs.cleaner.interval=1d
spark.history.fs.cleaner.maxAge=7d
In the Ambari UI you can achieve it by the following approach:
Ambari UI --> Spark2 --> Configs (Tab) --> Advanced (Sub Tab) --> "Custom spark2-defaults"
then click on "Add Property" then add the above properties there followed by a restart of the affected services.
.