Skip to main content
pankshiv1809
pankshiv1809
Gold ⭐️
January 12, 2023
Solved

How to configure DYNAMIC Spark Job

Hi All,

 

Need some guideline for "how to define/design Dynamic SPARK job" any parameter. Please suggest me, it would be more appreciate in advance.

 

Regards,

Pankaj

smdas
smdasAnswer
Gold ⭐️⭐️
January 13, 2023

Hello @pankshiv1809 

 

Thanks for using Cloudera Community. Based on your Post, Assuming [1] would help i.e. Using Dynamic Allocation to allow Spark adjust resources based on Workload requirement. 

 

Regards, Smarak

 

[1] Job Scheduling - Spark 3.3.1 Documentation (apache.org)

View in thread

3 replies

smdas
Gold ⭐️⭐️
smdasAnswer
Gold ⭐️⭐️
January 13, 2023

Hello @pankshiv1809 

 

Thanks for using Cloudera Community. Based on your Post, Assuming [1] would help i.e. Using Dynamic Allocation to allow Spark adjust resources based on Workload requirement. 

 

Regards, Smarak

 

[1] Job Scheduling - Spark 3.3.1 Documentation (apache.org)

Translate
smdas
Gold ⭐️⭐️
smdas
Gold ⭐️⭐️
January 16, 2023

Hello @pankshiv1809 

 

Hope you are doing well. We wish to follow-up on the Post & confirm whether your Team was requesting information into 

Dynamic Allocation to allow Spark adjust resources based on Workload requirement. 

 

Regards, Smarak

 

[1] Job Scheduling - Spark 3.3.1 Documentation (apache.org)

Translate
smdas
Gold ⭐️⭐️
smdas
Gold ⭐️⭐️
January 18, 2023

Hello @pankshiv1809 

 

Since we haven't heard from your side concerning the Post, We are marking the Post as Solved. If you have any further ask, Feel free to update the Post & we shall get back to you accordingly.

 

Regards, Smarak

Translate
Terms & ConditionsAccessibility statement