Member since
09-20-2018
334
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2298 | 05-14-2019 10:47 AM |
01-05-2020
06:33 AM
Hi, This parameter spark.executor.memory (or) spark.yarn.executor.memoryOverhead can be set in Spark submit command or you can set it Advanced configurations. Thanks AKR
... View more
01-05-2020
06:17 AM
Hi, Hope this below links helps in deciding the Configurations apart from the previous comments https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-2/ https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-1/ Thanks AKR
... View more
01-05-2020
05:39 AM
Hi, if there are more no of files are present in spark history Server, then FS would not be working as expected. In that case, We may need to write a script to delete the old files that are more then 7 days ( or as per your requirement) from the Spark history server location . Thanks Arun
... View more
01-05-2020
04:15 AM
Hi, This seems to be a Kerberos authentication problem. is this issue happens only for a SHS2 UI or does it happens for other URLs too? You could do a1. kinit username 2. kinit -kt service.keytab 3. send keytab file along with the submit command Thanks AKR
... View more
01-05-2020
04:03 AM
Hi, Whether those links that was provided helped to solve the issue http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/ http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/ if it was fine can you please mark this forum as solved. Thanks AKR
... View more
12-10-2019
01:24 AM
Hi, Have you tried submitting applications to other queue? Could you tell us does it worked on other queues? Queue.default has reached 99.7% and this may cause issues. Thanks AK
... View more
12-09-2019
11:32 PM
Hi, The correct way to pass the multiple configurations is that it should be passed along with the --conf. Ex: spark-submit --conf org.spark.metadata =false --conf spark.driver.memory=10gb Thanks AK
... View more
12-09-2019
11:07 PM
Hi, Hope this document will clarify your doubts. This was a tuning document. https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-2/ Thanks AK
... View more
12-09-2019
10:31 PM
Hi, If you are looking for 2.4 on HDP alone means, as mentioned in previous emails, don't expect any new HDP version anytime before the release of the combined new offering Cloudera Data Platform (CDP) sometime in 2020 or thereafter. As an alternative CDH 6.2+ supports spark 2.4 version. Thanks AK
... View more
11-12-2019
06:46 AM
Hi, Have you tried passing the Teradata jdbc Driver jar file that with --jar option and whether the issue got solved? Please share the error messages if you are still facing the issue. Thanks AKR
... View more