About dice

dice · ‎04-21-2019

Hi Roberto, Thank you for asking! 1) Please kindly provide the name and date/time of creation of the cluster. Similarly, if this fails for termination, please provide the same for the time the termination was attempted. 2) Do you repeatedly see the same cluster creation failure over Azure? 3) For Azure-service related issues with Altus clusters, please gather the following info from the Azure Portal: - Login to https://portal.azure.com - Navigate to Resource Groups - Select the Resource Group of the cluster in question - On the left hand column under Overview, select Activity Log - Filter Event Severity to Error and Critical - Review the Operations Names that have Failed or Critical Statuses, click any related items, select JSON and copy the contents to the filed support case. Or if you prefer, we can coordinate a webex session to gather this information) Regards, Daisuke

dice · ‎10-31-2018

Thank you for uploading the files! Okay, let us investigate to identify what is going on. Daisuke

dice · ‎10-29-2018

ashish5ahu, Thank you for posting the question. I am assuming you are using Workload XM with Altus, correct? If true, look at the right-bottom side of the job's page, where you are trying to see the Analytics tab, copy the CRN, and reply back with it. With the CRN, we can diagnose what's going on. Regards, Daisuke

dice · ‎09-05-2017

Symptoms A Spark job fails with INTERNAL_FAILURE. In the WA (Workload Analytics) page of the job that failed, the following message is reported: org.apache.spark.SparkException: Application application_1503474791091_0002 finished with failed status Diagnosis As the Telemetry Publisher didn't retrieve the application log due to a known bug, we have to diagnose the application logs (application_1503474791091_0002) directly, which are stored in the user's S3 bucket. If the following exception is found, it indicates that the application failed to resolve a dependency in the Hadoop class path: 17/08/24 13:13:33 INFO ApplicationMaster: Preparing Local resources Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.tracing.TraceUtils.wrapHadoopConf(Ljava/lang/String;Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/htrace/core/HTraceConfiguration; at org.apache.hadoop.fs.FsTracer.get(FsTracer.java:42) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:671) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:155) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653) This most likely occurred because the jar may have been built using the another Hadoop distribution's repository, for example EMR (Amazon Elastic MapReduce) Solution To resolve the issue, rebuild the application using the CDH repository, https://repository.cloudera.com/artifactory/cloudera-repos/, using Maven or sbt. The example of using Maven is as follows. https://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_vd_cdh5_maven_repo.html

dice · ‎06-21-2017

Question Where does Workload Analytics ingest user workloads into and analyze it? Answer Workload Analytics runs as part of Cloudera Altus functionality on an environment operated by Cloudera. Telemetry Publisher, part of Cloudera Manager installation, sends user's workload to the environment as soon as a job ends and analyze it. Thereon the result shows up in the Cloudera Altus UI. https://www.cloudera.com/documentation/altus/topics/wa_overview.html

dice · ‎06-21-2017

Question Is it possible to tune the thresholds for Health Check which Workload Analytics does based on user's requirements? Answer No, it's not possible to tune the thresholds. Health Check uses predefined thresholds which are described in the document: https://www.cloudera.com/documentation/altus/topics/wa_analyze_jobs.html

dice · ‎09-04-2016

Hi, Are you sure that the blocks are still existing in the DataNode hosts even after rebooting the instances? By default, the location should be under /dfs/dn{1,.2..}.

dice · ‎09-01-2016

Such custom jar file that HBase uses cannot be distributed across the hosts by CM automatically. You have to locate by yourself. HTH. -- Sent from my mobile

dice · ‎08-24-2016

Do you find something related error in the postgres logs that's located under /var/lib/cloudera-scm-server-db/data/pg_log/?

dice · ‎08-24-2016

Hi, That WARN message was introduced by an improvement per HDFS-9260. However, as our backport missed sorting, the WARN appears. It already gets addressed in: https://github.com/cloudera/hadoop-common/commit/95c7d8fbe122de617d11b6e4ea7d101803d0bd12 and the fix is available in CDH releases of 5.7.2 onwards and also in the 5.8.x series.

dice · ‎08-23-2016

Even you hit a single host crash, the corresponding blocks are replicated in the other hosts. That is, the hfile in the HDFS level should be safe. Otherwise, are you running the cluster as standalone mode? Thanks, Dice. -- Sent from my mobile

dice · ‎04-13-2016

Are you able to see the host being listed in the 'Hosts' page? Is the latest hearbeat coming within 15s? Also what's the OS version and the name of the distribution?

dice · ‎04-12-2016

Both HBASE-14533 and HBASE-14196 are being included in CDH 5.6. Can you upload 1) the client log, 2) thriftserver log, and 3) the result of 'hbase version' command in your console? Dice.

dice · ‎04-12-2016

You can change the parcel directory from /opt/cloudera/parcels (by default) to given directory per the following guide: Before following the steps, please shutdown the cluster for a safe. http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_parcels.html?scroll=cmug_topic_7_11_5_unique_1__section_srx_xyx_bm_unique_1 Configuring the Host Parcel Directory Otherwise, you need to add more disk to /opt.

dice · ‎04-05-2016

Hi, Unfortunately, it's hard-coded in the CM server side. I'm wondering why it takes longer than the particular timeout limit (90 seconds). What do you see in the CM agent log in that target host?

dice · ‎12-08-2015

What is the i18n configuration in those targeted hosts?

dice · ‎12-07-2015

To enable Kerberos in your CM managed CDH cluster, please follow the document below: http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cm_sg_authentication.html As you've already noticed, files under /etc take effect only on the client programs. HTH. -- Sent from my mobile

dice · ‎12-03-2015

If you are still hitting the same stack trace, the indications won't be relevant. Can you upload the Reports Manager log please? -- Sent from my mobile

dice · ‎12-02-2015

Run 'hadoop version' as well.

dice · ‎12-02-2015

Also upload the contents under /var/log/cloudera-manager-installer/.

dice · ‎12-02-2015

Can you upload the CM server log as well?

dice · ‎12-02-2015

Hi, I'm unsure if your /etc/hosts is the real one, but ensure that you meet all the requirements under " Networking and Security Requirements " in the following guide. http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cm_ig_cm_requirements.html Also it looks you're enabling single user mode per "Because agent not running as root". Is this correct? Have you followed the guide below? http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/install_singleuser_reqts.html

dice · ‎12-01-2015

Hi, Could you please elaborate where you get stuck? What are you looking at now?

dice · ‎12-01-2015

Is the issue still occurring? Just in case, can you run 'rpm -qa | grep cloudera' in the CM server host?

dice · ‎12-01-2015

Is your CM version higher than 5.4? What is the CDH version, btw? Dice.

dice · ‎11-30-2015

Hi hawkphil, What did you actually do in the cluster before you face the error? Which version are you running on? Dice.

dice · ‎11-27-2015

1. No, snapshot is just for the metadata operation. 2. Once you make particular directory snapshottable, the blocks belonging the underlying files never be deleted.

dice · ‎10-14-2015

Hi, Please note that Cloudera Search is being included in CDH 5. As the CDH 5.4.3 parcel looks already being activated, you can simply use it via "Add services" from the CM home page.

dice · ‎06-21-2015

Are you using Cloudera Manager 5.4 and higher? If you're still on 5.2 or 5.3, Kafka CSD needs to be downloaded per http://www.cloudera.com/content/cloudera/en/documentation/cloudera-kafka/latest/topics/kafka_installing.html, thereon see the parcels being available.

dice · ‎06-07-2015

Hi, What did you make a change over the cluster before you see the message, "Space free in the cluster: 0 B"? How did you verify that the space is not the case? Can you also verify if the DataNodes are up? Are there actual blocks in DNs' local directories?

Online	Offline
Last Visited	‎02-26-2021 05:29 AM

Date Registered	‎08-01-2013 05:35 PM
Last Visited	‎02-26-2021 05:29 AM
Total Messages Posted	187
Total Kudos Received	10

Re: Adding a host-- HBase not copying over all Jar...

Re: Can't find the latest SOLR package in CM

Re: Kafka not in list of services to be added

Re: Failed to execute command Initialize Solr on s...

Re: CDH 5.3 Impala Llama Kerberos: yarn is not all...

Re: CREATION_GENERAL_ERROR

Re: Workload XM internal error keeps happening

Re: Workload XM internal error keeps happening

Jars Built for EMR Failed to Run on Cloudera Altus

Where do Workload Analytics analyze user workloads...

Is Health Check Threshold Tunable in Workload Anal...

Re: Problem with starting CDH cluster on AWS using...

Re: Adding a host-- HBase not copying over all Jar...

Re: embedded DB failed to start during installatio...

Re: name node log full of WARN Please update the D...

Re: Is hbase snapshot valid after hfiles get delet...

Re: Unable to deploy client configuration first ti...

Re: HBase 1.0.0-cdh5.6.0 Failed to get region loca...

Re: Free space error after restarting the cluster.

Re: CDH cluster - command timed-out after 90 secon...

Re: Cloudera Manager hangs when Installing Selecte...

Re: How to change hdfs-site.xml

Re: Reports Manager - Aggregate Data has not been ...

Re: Reports Manager - Aggregate Data has not been ...

Re: Installing Cloudera Cluster

Re: Installing Cloudera Cluster

Re: Failed to add new Host through Cloudera Manage...

Re: Installing Cloudera Cluster

Re: Reports Manager - Aggregate Data has not been ...

Re: Reports Manager - Aggregate Data has not been ...

Re: Cannot restart namenode: Failed to load an FSI...

Re: HDFS Snapshot

Re: Can't find the latest SOLR package in CM

Re: Kafka not in list of services to be added

Re: HDFS goes in bad health