Member since
08-19-2020
2
Posts
0
Kudos Received
0
Solutions
08-26-2020
01:22 AM
Hello @KSKR , thank you for raising the question on "how to fetch the CPU utilization for a Spark job programmatically". One way to do this is via the Spark REST API. You should consider if you need the "live data" or you are looking for analysis once the application finished running. While the application is running, you can consider to connect to the driver and fetch the live data. Once the application finished running, you can consider parse the JSON files (the event log files) for the CPU time or use the Spark REST API and let the Spark History Server serve you with the data. What is your exact requirement? What would you like to achieve? Thank you: Ferenc
... View more