question How to configure DYNAMIC Spark Job in Support Questions

How to configure DYNAMIC Spark Job

pankshiv1809 — Thu, 12 Jan 2023 08:33:30 GMT

Hi All,

Need some guideline for "how to define/design Dynamic SPARK job" any parameter. Please suggest me, it would be more appreciate in advance.

Regards,

Pankaj

Re: How to configure DYNAMIC Spark Job

smdas — Fri, 13 Jan 2023 09:59:10 GMT

Hello @pankshiv1809

Thanks for using Cloudera Community. Based on your Post, Assuming [1] would help i.e. Using Dynamic Allocation to allow Spark adjust resources based on Workload requirement.

Regards, Smarak

[1] Job Scheduling - Spark 3.3.1 Documentation (apache.org)

Re: How to configure DYNAMIC Spark Job

smdas — Mon, 16 Jan 2023 09:55:43 GMT

Hello @pankshiv1809

Hope you are doing well. We wish to follow-up on the Post & confirm whether your Team was requesting information into

Dynamic Allocation to allow Spark adjust resources based on Workload requirement.

Regards, Smarak

[1] Job Scheduling - Spark 3.3.1 Documentation (apache.org)

Re: How to configure DYNAMIC Spark Job

smdas — Wed, 18 Jan 2023 07:03:36 GMT

Hello @pankshiv1809

Since we haven't heard from your side concerning the Post, We are marking the Post as Solved. If you have any further ask, Feel free to update the Post & we shall get back to you accordingly.

Regards, Smarak