Member since
01-24-2016
48
Posts
11
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2798 | 07-02-2018 01:22 PM | |
| 10424 | 07-22-2016 06:04 PM |
06-28-2021
11:14 PM
Dear CDH community, I just rebuilt my servers after a ubuntu OS fubar that knocked out my CDH and Cloudera. I could not locate the free versions of Cloudera Manager and CDH. I am hoping they are still available to starving developers like me. Thanks Sanjay
... View more
Labels:
07-02-2018
01:22 PM
I resolved this by installing an earlier version of Spark2 1. Deleted Spark2 Service 2. Deactivated, and removed distributed parcel 2.3.0.cloudera2-1.cdh5.13.3.p0.316101 3. Got earlier parcel 2.2.0.cloudera2-1.cdh5.12.0.p0.232957 4. Downloaded and distributed and activated this parcel 5. Got the CSD for this and put into /opt/cloudera/csd 6. Installed Spark2 from CM. Spark2 up and running on my cluster !
... View more
06-10-2018
09:49 PM
sudo grep ERROR /var/log/cloudera-scm-agent/cloudera-scm-agent.log [10/Jun/2018 21:46:46 +0000] 32685 MainThread util ERROR Lineage file not found, skipping Spark plugin creation: /etc/spark2/conf.cloudera.spark2_on_yarn/navigator.lineage.client.properties
... View more
06-10-2018
09:44 PM
Hi everyone I just upgraded by 3 node research cluster to 5.14.2-1.cdh5.14.2.p0.3. The HDFS, YARN, ZOOKEEPER, HIVE, IMPALA services got successfully installed and upgraded. I followed the steps in https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html to install 2.3.0.cloudera2-1.cdh5.13.3.p0.316101 install on the cluster Get the following errors. I am not sure where do I go to see any detail error logs. Nothing is shown in CM Deploying Client Configuration Failed to deploy client configuration to the cluster. Execute DeployClusterClientConfig for {hdfs,yarn,spark_on_yarn,hive,spark2_on_yarn} in parallel. Completed only 4/5 steps. First failure: Failed to execute command Deploy Client Configuration on service Spark 2 Execute command Deploy Client Configuration on service Spark 2 Deploy Client Configuration failed. Generate and deploy client configuration. Completed only 0/3 steps. First failure: Client configuration (id=12) on host hp8300one (id=1) exited with 1 and expected 0.
... View more
Labels:
- Labels:
-
Apache Spark
-
Cloudera Manager
01-31-2017
08:16 PM
Awesome Thanks Tim I did the mem checks specifically "memz detailed=true" And realized the mem_limit was somehow 6GB for node 1 and 2 but 256MB on node 3 😞 I changed all three to 6GB each and the query works now. Really appreciate your help and my belief in Cloudera only becomes 10 fold stronger ! warmly and appreciatively sanjay
... View more
01-31-2017
08:15 PM
1 Kudo
Awesome Thanks Tim I did the mem checks specifically http://hostname:25000/memz?detailed=true And realized the mem_limit was somehow 6GB for node 1 and 2 but 256MB on node 3 😞 I changed all three to 6GB each and the query works now. Really appreciate your help and my belief in Cloudera only becomes 10 fold stronger ! warmly and appreciatively sanjay
... View more
01-30-2017
08:02 AM
I am having same issues. I use CDH 5.8.0 CM 5.8.1 WARNINGS: Memory limit exceeded The memory limit is set too low to initialize spilling operator (id=3). The minimum required memory to spill this operator is 264.00 MB. Memory Limit Exceeded Query(60409f68f36d7b3d:301437049bd7bba0) Limit: Consumption=160.58 MB Fragment 60409f68f36d7b3d:301437049bd7bba2: Consumption=123.18 MB AGGREGATION_NODE (id=3): Consumption=122.02 MB EXCHANGE_NODE (id=2): Consumption=0 DataStreamRecvr: Consumption=1.16 MB Fragment 60409f68f36d7b3d:301437049bd7bba5: Consumption=37.40 MB AGGREGATION_NODE (id=1): Consumption=11.03 MB HDFS_SCAN_NODE (id=0): Consumption=26.23 MB DataStreamSender: Consumption=80.00 KB Block Manager: Limit=156.00 MB Consumption=114.00 MB Could not execute command: select isr, count(isr) as counts from aers.demo_drug_reac_combo_clean group by isr having counts > 1 Impala 2.6.0+cdh5.8.0+0 My query is ultra simple select isr, count(isr) as counts from aers.demo_drug_reac_combo_clean group by isr having counts > 1 aers.demo_drug_reac_combo_clean contains only 10 million records and 9 cols Metadata is as follows | isr | drugname | pt | year | age | age_cod | age_norm | age_group | | 3175747 | troglitazone | hepatotoxicity nos | 1999 | 68 | YR | 68 | 65-69 | Hadoop Cluster Setup ==================== 3 nodes (HP8300 Elite Desktops) , 32GB RAM each node
... View more
12-02-2016
02:26 PM
Hi guys I face this problem every day now on at least two of the hosts "This host has been out of contact with the Cloudera Manager Server for too long. This host is in contact with the Host Monitor. The host's Cloudera Manager Agent version matches the Host Monitor version (5.8.1)." Support admin Inspector Results Validations Inspector failed on the following hosts... ip-10-231-19-[23, 196]: Can only run host inspector when host is healthy. Inspector ran on 6 hosts. Individual hosts resolved their own hostnames correctly. No errors were found while looking for conflicting init scripts. No errors were found while checking /etc/hosts. All hosts resolved localhost to 127.0.0.1. All hosts checked resolved each other's hostnames correctly and in a timely manner. Host clocks are approximately in sync (within ten minutes). Host time zones are consistent across the cluster. No users or groups are missing. No conflicts detected between packages and parcels. No kernel versions that are known to be bad are running. No problems were found with /proc/sys/vm/swappiness on any of the hosts. No performance concerns with Transparent Huge Pages settings. CDH 5 Hue Python version dependency is satisfied. 0 hosts are running CDH 4 and 8 hosts are running CDH 5. There are mismatched versions across the system, which will cause failures. See below for details on which hosts are running what versions of components. Java versions are inconsistent across the managed hosts. Check the component version below to identify hosts with inconsistent Java versions. All checked Cloudera Management Daemons versions are consistent with the server. All checked Cloudera Management Agents versions are consistent with the server. Version Summary HostsComponent Version Release CDH Version ml-nonprod — Group 1 (CDH 5) ip-10-231-18-[68, 203]; ip-10-231-19-[23, 86-87, 196] Bigtop-Tomcat (CDH 5 only) 0.7.0+cdh5.8.0+0 1.cdh5.8.0.p0.73 CDH 5 Crunch (CDH 5 only) 0.11.0+cdh5.8.0+91 1.cdh5.8.0.p0.77 CDH 5 Flume NG 1.6.0+cdh5.8.0+50 1.cdh5.8.0.p0.75 CDH 5 MapReduce 1 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 Hadoop 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HDFS 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HttpFS 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 hadoop-kms 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 MapReduce 2 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 YARN 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HBase 1.2.0+cdh5.8.0+160 1.cdh5.8.0.p0.80 CDH 5 Lily HBase Indexer 1.5+cdh5.8.0+64 1.cdh5.8.0.p0.75 CDH 5 Hive 1.1.0+cdh5.8.0+610 1.cdh5.8.0.p0.77 CDH 5 HCatalog 1.1.0+cdh5.8.0+610 1.cdh5.8.0.p0.77 CDH 5 Hue 3.9.0+cdh5.8.0+2512 1.cdh5.8.0.p0.88 CDH 5 Impala 2.6.0+cdh5.8.0+0 1.cdh5.8.0.p0.111 CDH 5 Kite (CDH 5 only) 1.0.0+cdh5.8.0+136 1.cdh5.8.0.p0.73 CDH 5 Llama (CDH 5 only) 1.0.0+cdh5.8.0+0 1.cdh5.8.0.p0.73 CDH 5 Mahout 0.9+cdh5.8.0+27 1.cdh5.8.0.p0.71 CDH 5 Oozie 4.1.0+cdh5.8.0+291 1.cdh5.8.0.p0.83 CDH 5 Parquet 1.5.0+cdh5.8.0+174 1.cdh5.8.0.p0.71 CDH 5 Pig 0.12.0+cdh5.8.0+83 1.cdh5.8.0.p0.71 CDH 5 sentry 1.5.1+cdh5.8.0+244 1.cdh5.8.0.p0.83 CDH 5 Solr 4.10.3+cdh5.8.0+423 1.cdh5.8.0.p0.79 CDH 5 spark 1.6.0+cdh5.8.0+205 1.cdh5.8.0.p0.74 CDH 5 Sqoop2 1.99.5+cdh5.8.0+38 1.cdh5.8.0.p0.72 CDH 5 Sqoop 1.4.6+cdh5.8.0+65 1.cdh5.8.0.p0.69 CDH 5 Whirr 0.9.0+cdh5.8.0+17 1.cdh5.8.0.p0.68 CDH 5 Zookeeper 3.4.5+cdh5.8.0+94 1.cdh5.8.0.p0.76 CDH 5 Cloudera Manager Management Daemons 5.8.1 1.cm581.p0.7 Not applicable Supervisord 3.0-cm5.8.1 Unavailable Not applicable Java 7 JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) Unavailable Not applicable Java 6 JAVA_HOME=/usr/lib/jvm/j2sdk1.6-oracle java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) Unavailable Not applicable Cloudera Manager Agent 5.8.1 1.cm581.p0.7 Not applicable HostsComponent Version Release CDH Version ml-nonprod — Group 2 (CDH 5) ip-10-231-19-197 Bigtop-Tomcat (CDH 5 only) 0.7.0+cdh5.8.0+0 1.cdh5.8.0.p0.73 CDH 5 Crunch (CDH 5 only) 0.11.0+cdh5.8.0+91 1.cdh5.8.0.p0.77 CDH 5 Flume NG 1.6.0+cdh5.8.0+50 1.cdh5.8.0.p0.75 CDH 5 MapReduce 1 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 Hadoop 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HDFS 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HttpFS 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 hadoop-kms 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 MapReduce 2 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 YARN 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HBase 1.2.0+cdh5.8.0+160 1.cdh5.8.0.p0.80 CDH 5 Lily HBase Indexer 1.5+cdh5.8.0+64 1.cdh5.8.0.p0.75 CDH 5 Hive 1.1.0+cdh5.8.0+610 1.cdh5.8.0.p0.77 CDH 5 HCatalog 1.1.0+cdh5.8.0+610 1.cdh5.8.0.p0.77 CDH 5 Hue 3.9.0+cdh5.8.0+2512 1.cdh5.8.0.p0.88 CDH 5 Impala 2.6.0+cdh5.8.0+0 1.cdh5.8.0.p0.111 CDH 5 Kite (CDH 5 only) 1.0.0+cdh5.8.0+136 1.cdh5.8.0.p0.73 CDH 5 Llama (CDH 5 only) 1.0.0+cdh5.8.0+0 1.cdh5.8.0.p0.73 CDH 5 Mahout 0.9+cdh5.8.0+27 1.cdh5.8.0.p0.71 CDH 5 Oozie 4.1.0+cdh5.8.0+291 1.cdh5.8.0.p0.83 CDH 5 Parquet 1.5.0+cdh5.8.0+174 1.cdh5.8.0.p0.71 CDH 5 Pig 0.12.0+cdh5.8.0+83 1.cdh5.8.0.p0.71 CDH 5 sentry 1.5.1+cdh5.8.0+244 1.cdh5.8.0.p0.83 CDH 5 Solr 4.10.3+cdh5.8.0+423 1.cdh5.8.0.p0.79 CDH 5 spark 1.6.0+cdh5.8.0+205 1.cdh5.8.0.p0.74 CDH 5 Sqoop2 1.99.5+cdh5.8.0+38 1.cdh5.8.0.p0.72 CDH 5 Sqoop 1.4.6+cdh5.8.0+65 1.cdh5.8.0.p0.69 CDH 5 Whirr 0.9.0+cdh5.8.0+17 1.cdh5.8.0.p0.68 CDH 5 Zookeeper 3.4.5+cdh5.8.0+94 1.cdh5.8.0.p0.76 CDH 5 Cloudera Manager Management Daemons 5.8.1 1.cm581.p0.7 Not applicable Supervisord 3.0-cm5.8.1 Unavailable Not applicable Java 7 JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) Unavailable Not applicable Java 6 JAVA_HOME=/usr/lib/jvm/j2sdk1.6-oracle java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) Unavailable Not applicable Java 8 JAVA_HOME=/usr/lib/jvm/java-8-oracle java version "1.8.0_101" Java(TM) SE Runtime Environment (build 1.8.0_101-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode) Unavailable Not applicable Cloudera Manager Agent 5.8.1 1.cm581.p0.7 Not applicable HostsComponent Version Release CDH Version ml-nonprod — Group 3 (CDH 5) ip-10-231-18-181 Bigtop-Tomcat (CDH 5 only) 0.7.0+cdh5.8.0+0 1.cdh5.8.0.p0.73 CDH 5 Crunch (CDH 5 only) 0.11.0+cdh5.8.0+91 1.cdh5.8.0.p0.77 CDH 5 Flume NG 1.6.0+cdh5.8.0+50 1.cdh5.8.0.p0.75 CDH 5 MapReduce 1 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 Hadoop 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HDFS 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HttpFS 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 hadoop-kms 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 MapReduce 2 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 YARN 2.6.0+cdh5.8.0+1601 1.cdh5.8.0.p0.93 CDH 5 HBase 1.2.0+cdh5.8.0+160 1.cdh5.8.0.p0.80 CDH 5 Lily HBase Indexer 1.5+cdh5.8.0+64 1.cdh5.8.0.p0.75 CDH 5 Hive 1.1.0+cdh5.8.0+610 1.cdh5.8.0.p0.77 CDH 5 HCatalog 1.1.0+cdh5.8.0+610 1.cdh5.8.0.p0.77 CDH 5 Hue 3.9.0+cdh5.8.0+2512 1.cdh5.8.0.p0.88 CDH 5 Impala 2.6.0+cdh5.8.0+0 1.cdh5.8.0.p0.111 CDH 5 Kite (CDH 5 only) 1.0.0+cdh5.8.0+136 1.cdh5.8.0.p0.73 CDH 5 Llama (CDH 5 only) 1.0.0+cdh5.8.0+0 1.cdh5.8.0.p0.73 CDH 5 Mahout 0.9+cdh5.8.0+27 1.cdh5.8.0.p0.71 CDH 5 Oozie 4.1.0+cdh5.8.0+291 1.cdh5.8.0.p0.83 CDH 5 Parquet 1.5.0+cdh5.8.0+174 1.cdh5.8.0.p0.71 CDH 5 Pig 0.12.0+cdh5.8.0+83 1.cdh5.8.0.p0.71 CDH 5 sentry 1.5.1+cdh5.8.0+244 1.cdh5.8.0.p0.83 CDH 5 Solr 4.10.3+cdh5.8.0+423 1.cdh5.8.0.p0.79 CDH 5 spark 1.6.0+cdh5.8.0+205 1.cdh5.8.0.p0.74 CDH 5 Sqoop2 1.99.5+cdh5.8.0+38 1.cdh5.8.0.p0.72 CDH 5 Sqoop 1.4.6+cdh5.8.0+65 1.cdh5.8.0.p0.69 CDH 5 Whirr 0.9.0+cdh5.8.0+17 1.cdh5.8.0.p0.68 CDH 5 Zookeeper 3.4.5+cdh5.8.0+94 1.cdh5.8.0.p0.76 CDH 5 Cloudera Manager Management Daemons 5.8.1 1.cm581.p0.7 Not applicable Supervisord 3.0-cm5.8.1 Unavailable Not applicable Java 7 JAVA_HOME=/usr/lib/jvm/java-7-oracle-cloudera java version "1.7.0_67" Java(TM) SE Runtime Environment (build 1.7.0_67-b01) Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode) Unavailable Not applicable Java 6 JAVA_HOME=/usr/lib/jvm/j2sdk1.6-oracle java version "1.6.0_31" Java(TM) SE Runtime Environment (build 1.6.0_31-b04) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) Unavailable Not applicable Java 8 JAVA_HOME=/usr/lib/jvm/java-8-oracle java version "1.8.0_111" Java(TM) SE Runtime Environment (build 1.8.0_111-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode) Unavailable Not applicable Cloudera Manager Agent 5.8.1 1.cm581.p0.7 Not applicable
... View more
Labels:
- Labels:
-
Cloudera Manager
08-02-2016
10:33 AM
Thats awesome thanks a ton Mike. Early this morning before your mails came in - I grew impatient 🙂 as is my nature - and did give it a shot to the Cloudera director as-is-wher-is scripts. 1. Used the Cloud Formation template here https://s3.amazonaws.com/quickstart-reference/cloudera/latest/templates/Template2-Cloudera-AWS-ExistingVPC.template 2. Created a "ClusterLauncher Instance" on AWS 3. SSH to "ClusterLauncher Instance" cloudera-director bootstrap cloudera/setup-default/aws.reference.conf Process logs can be found at /home/ec2-user/.cloudera-director/logs/application.log Plugins will be loaded from /var/lib/cloudera-director-plugins OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0 Cloudera Director 2.1.0 initializing ... The configuration file aws.reference.conf is not present or cannot be read. [ec2-user@ip-10-219-178-74 ~]$ cloudera-director bootstrap cloudera/setup-default/aws.reference.conf Process logs can be found at /home/ec2-user/.cloudera-director/logs/application.log Plugins will be loaded from /var/lib/cloudera-director-plugins OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0 Cloudera Director 2.1.0 initializing ... Installing Cloudera Manager ... * Starting ..... done * Requesting an instance for Cloudera Manager .......................... done * Installing screen package (1/1) ....... done * Running custom bootstrap script on [10.219.177.189, ip-10-219-177-189.us-west-2.compute.internal, 52.43.22.181, ec2-52-43-22-181.us-west-2.compute.amazonaws.com] .......... done * Waiting for SSH access to [10.219.177.189, ip-10-219-177-189.us-west-2.compute.internal, 52.43.22.181, ec2-52-43-22-181.us-west-2.compute.amazonaws.com], default port 22 ..... done * Inspecting capabilities of 10.219.177.189 .......... done * Normalizing a3508870-3de7-4dc0-84a5-f69c77610c89 ..... done * Installing ntp package (1/4) ..... done * Installing curl package (2/4) ..... done * Installing nscd package (3/4) ..... done * Installing gdisk package (4/4) ..................... done * Resizing instance root partition ......... done * Mounting all instance disk drives ............ done * Waiting for new external database servers to start running ........ done * Installing repositories for Cloudera Manager ....... done * Installing oracle-j2sdk1.7 package (1/3) ..... done * Installing cloudera-manager-daemons package (2/3) ..... done * Installing cloudera-manager-server package (3/3) ...... done * Setting up embedded PostgreSQL database for Cloudera Manager ...... done * Installing cloudera-manager-server-db-2 package (1/1) ..... done * Starting embedded PostgreSQL database ...... done * Starting Cloudera Manager server ... done * Waiting for Cloudera Manager server to start ..... done * Setting Cloudera Manager License ... done * Enabling Enterprise Trial ... done * Configuring Cloudera Manager ..... done * Deploying Cloudera Manager agent ...... done * Waiting for Cloudera Manager to deploy agent on 10.219.177.189 ... done * Setting up Cloudera Management Services ............ done * Backing up Cloudera Manager Server configuration ...... done * Inspecting capabilities of 10.219.177.189 ...... done * Done ... Cloudera Manager ready. Creating cluster C5-Reference-AWS ... * Starting ..... done * Requesting 11 instance(s) in 3 group(s) ....................................... done * Preparing instances in parallel (20 at a time) .............................................................. done * Waiting for Cloudera Manager installation to complete ... done * Installing Cloudera Manager agents on all instances in parallel (20 at a time) ........ done * Waiting for new external database servers to start running ... done * Creating CDH5 cluster using the new instances ... done * Creating cluster: C5-Reference-AWS .... done * Downloading parcels: CDH-5.7.2-1.cdh5.7.2.p0.18,KAFKA-2.0.2-1.2.0.2.p0.5 ... done * Distributing parcels: KAFKA-2.0.2-1.2.0.2.p0.5,CDH-5.7.2-1.cdh5.7.2.p0.18 ... done * Activating parcels: KAFKA-2.0.2-1.2.0.2.p0.5,CDH-5.7.2-1.cdh5.7.2.p0.18 ...... done * Configuring Hive to use Sentry ... done * Creating Sentry Database ... done * Calling firstRun on cluster C5-Reference-AWS ... done * Waiting for firstRun on cluster C5-Reference-AWS .... done * Running cluster post creation scripts ...... done * Adjusting health thresholds to take into account optional instances. ... done * Done ...
... View more
08-01-2016
01:49 PM
http://docs.aws.amazon.com/quickstart/latest/cloudera/welcome.html Hey guys Will this work for the starving developers version as well ? I am using CDH 5.8.0 thanks sanjay
... View more
Labels: