Member since
04-04-2022
62
Posts
4
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
179 | 12-08-2022 08:57 PM | |
139 | 12-07-2022 02:34 AM | |
199 | 12-06-2022 05:34 AM | |
229 | 12-01-2022 02:17 AM | |
190 | 11-25-2022 06:09 AM |
01-03-2023
09:03 AM
Hi @Tellyou6bu6 Did you got chance to check capacity-schedular.xml file?
... View more
01-03-2023
09:02 AM
@aval Can you please confirm on which version of Cloudera you are currently on ? Basically HWC is required when you want to access Managed tables via Spark. Also use of spark.sql.hive.hwc.execution.mode is deprecated as per CDP 7.1.7 https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/integrating-hive-and-bi/topics/hive-hwc-reader-mode.html
... View more
01-03-2023
08:58 AM
@myzard Can you please upload full error message which is above Traceback?
... View more
01-03-2023
08:55 AM
@Mostafa12345678 Can you share full stack trace for review?
... View more
01-02-2023
11:48 PM
yarn.nodemanager.disk-health-checker.enable is only to Enable or disable the disk health checker service and will not fix this issue. Please share NM logs snippet for review.
... View more
01-02-2023
06:24 AM
Hi @Tellyou6bu6 Regarding your Q1 can you please share capacity-schedular.xml file from YARN for review?
... View more
01-02-2023
06:23 AM
Hi @rudi101101 Can you please confirm if you are using any timeout settings from PowerBI end?
... View more
01-02-2023
06:05 AM
@hanumanth Can you please share top command output from NM's, need to check which processes are consuming resources on NM
... View more
01-02-2023
06:02 AM
Hi @SantoshB You can see such messages when you have reached user factor limit/resource limit on queue level. You can check tune user-limit factor or check queue utilization to schedule applications accordingly. Also seems application has failed with exit-code 13, can you please share YARN trace to identify reason for failure?
... View more
01-02-2023
05:57 AM
Hi @sanjaysubs Seems you don't have any healthy NM currently thats why you don't have any resources available to run any application. Please fix issue with your Node Managers it should solve your problem.
... View more
12-25-2022
11:35 PM
Hi @hanumanth There can be many reasons , are you observing this on specific node or all node managers?
... View more
12-25-2022
11:34 PM
Hi @reca If you are on CDH you can use navigator [1] and if you are on CDP you can use Ranger [2] [1] https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cn_iu_audit_usecase.html [2] https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/security-ranger-auditing/topics/security-ranger-audit-managing.html
... View more
12-08-2022
08:57 PM
1 Kudo
Hi You can check below document for better understanding of how queue manager works. https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/yarn-allocate-resources/topics/yarn-set-user-limits.html Also if you are Cloudera customer you can check below KB's https://my.cloudera.com/knowledge/How-does-user-limit-factor-impact-capacity-scheduler?id=270996 https://my.cloudera.com/knowledge/How-is-Max-Schedulable-Applications-Per-User-calculated-by?id=271520 https://my.cloudera.com/knowledge/Tuning-the-YARN-Capacity-Scheduler?id=276877 Hope this answers your question, please mark solution as accepted if it resolves your issue and hit like.
... View more
12-07-2022
02:34 AM
@neters Hope this answers your query please mark solution as accepted and hit like if you find it helpful
... View more
12-06-2022
05:38 AM
Hi @hanumanth You can also review this for spark tuning https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/tuning-spark/topics/spark-admin_spark_tuning.html
... View more
12-06-2022
05:34 AM
Hi @neters You can check component version for CDH 5.16.x here and compare them with apache versions. https://docs.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh_package_tarball_516.html
... View more
12-02-2022
09:44 AM
@sss123 Are you able to run spark commands via spark-shell spark-submit?
... View more
12-01-2022
02:18 AM
Hi @d_liu Could be you took restart of HS2 but didn't logged in again from Hue, could be it is using same session. But HS2 restart should solve your problem.
... View more
12-01-2022
02:17 AM
1 Kudo
Hi @HopVH , For impala queries you can check under Impala>Queries> Query profile For Hive it should have generated YARN application id you can trace its logs by navigating to JHS or with yarn logs -applicationId application_xxxx_xx Regards, Asim
... View more
11-30-2022
02:06 AM
Hi @SchmidtS Please try increasing session/ query timeout.
... View more
11-30-2022
02:03 AM
@d_liu Quickest way would be to restart HS2/Hue. Are you able to fetch results from impala?
... View more
11-28-2022
01:50 AM
@WuHua Can you please try to add this property and re-run query? Add --control_service_queue_mem_limit=200M to "Impala Daemon Command Line Argument Advanced Configuration Snippet (Safety Valve)" Thanks !
... View more
11-25-2022
06:09 AM
Hi @bulbcat - Spark version used in CDH 6.3.2 is Spark 2.4.0 [1] - Also spark.yarn.priority will only effect from spark-3.0 [2] [1]https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_63_packaging.html [2]https://github.com/apache/spark/pull/27856 If you want to update priority for application during runtime you can use below yarn application -updatePriority 10 -appId application_xxxx_xx Thanks!
... View more
11-21-2022
02:11 AM
2 Kudos
Hi @Tellyou6bu6 If you are installing trail version of CDP , It will use embedded Postgre for installation. Installing a Trial Cluster In this procedure, Cloudera Manager automates the installation of the Oracle JDK, Cloudera Manager Server, embedded PostgreSQL database, Cloudera Manager Agent, Runtime, and managed service software on cluster hosts. Cloudera Manager also configures databases for the Cloudera Manager Server and Hive Metastore and optionally for Cloudera Management Service roles." You can check below link for detail explanation. https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/installation/topics/cdpdc-trial-installation.html Regards, Asim
... View more
11-18-2022
05:46 AM
@shamly Can you share full stack trace?
... View more
11-18-2022
05:45 AM
@drgenious How you upgraded impala? You can check "strace -s 2000 impala-shell -i hostname -k --ca-cert.pem -q "show databases > impala_log to verify if is referring correct impala version. Thanks!
... View more
11-17-2022
02:11 AM
Hi @tencentemr w as your question answered ? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
11-17-2022
02:09 AM
@pankshiv1809 w as your question answered ? Make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.
... View more
11-16-2022
04:20 AM
@pankshiv1809 You can review below blogs for tuning spark applications based on your case you need to tune executer,driver memories and cores along with other parameters mentioned in below blog. https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-1/ https://blog.cloudera.com/how-to-tune-your-apache-spark-jobs-part-2/ Thanks!
... View more
11-02-2022
03:06 AM
Hi @tencentemr You can tune below mentioned script as per your requirement. https://community.cloudera.com/t5/Community-Articles/script-to-kill-yarn-application-if-it-is-running-more-than-x/ta-p/244521 Thanks!
... View more