Skip to main content
rajukasula3
Silver ⭐️
May 15, 2024 6:02 PM UTC
Question

Challenging issues solved on spark

Hello All, 

Can anyone provide me the most challenging real time issues solved on Spark, YARN and Hive ? 

@RangaReddy 

2 replies

robert2001b
Silver ⭐️
May 27, 2024 10:34 AM UTC

Tuning Spark jobs to minimize execution time and resource usage. Handling out-of-memory errors and optimizing memory allocation. Addressing issues where a disproportionate amount of work is done by a small number of tasks. Ensuring Spark applications can recover gracefully from worker failures. Efficiently distributing resources among various applications running in a cluster. Managing the lifecycle of containers that execute application tasks. 

Translate
rajukasula3
Silver ⭐️
Jun 1, 2024 6:40 PM UTC

Thank you

Translate