Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to tune a hive on spark which is taking too many resources ? To limit the resources and optimize the job

How to tune a hive on spark which is taking too many resources ? To limit the resources and optimize the job

I have a hive on spark job which is taking too many res sources at the peak 1tb it has a few joins which causes the performance to lag. (It is only joining on a few million records not more than that)

How can I improve the performance with various configs in hive to

1. Reduce the resources consumed by this job is their away to limit the vcores and executor memory so that is doesnt take all the cluster ressources?

2. Are there any hive on spark specific configs to modify for this case lots of joins with say 5m records.

Thank you

Don't have an account?
Coming from Hortonworks? Activate your account here