Member since
09-12-2016
21
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2309 | 10-09-2016 05:42 AM | |
3024 | 09-23-2016 08:18 AM |
10-09-2016
05:42 AM
All, We can set the parameters if we do the following and it was a tough lesson to learn.. There is a config session for Service wide parameters and then for specific areas, do not list or mix which parameters exist for which session as it will throw an error, but not an intelligent one. Additionally, I would recommend before setting parameters to run the script to dump the entire configuration into a file and then to review the specific parameters and the naming of them. I was using the GUI and seeing items like HBASE.replication and thought that this could be used, which was not the case. Instead, it was HBASE_enable_replication. I also found what I believe was a naming issue in that the name of the parameter did not align with the value, it wasn't spelled correctly like "..ion" and was "..on". The problem was I was spelling according to english vs reviewing the parameter name.
... View more
10-04-2016
01:53 PM
thank you for the timely follow-up... still trying to figure our configs section though ;-(
... View more
10-04-2016
01:13 PM
From first glance, when running the bootstrap command it appears as though you can run in multiple times and that logic might exist for it take take in any changes and incorporate them. This is based on my personal observation of running it, as it prompts the user informing that an existing configuration already exists and do they want to rebuild or resume. I have been trying to debug configuring the "Configs" section, both at the System-wide and at the Role-level. Given that the script errors out, i presumed that I could edit or update and then re-run the file. My oberservation is that the re-run does not take into account any changes made to the file being submitted, as i ended up commenting out a whole section of configs and the error continued to show-up. I then terminated and re-ran the script and everything worked fine (aka, I had no parameters in the config section). So, beware...
... View more
10-04-2016
08:30 AM
fyi - there is a type in the regionserver, as it only requires a single s
... View more
10-04-2016
08:29 AM
Problem not resolved but now get the following error [2016-10-04 07:19:05] INFO [pipeline-thread-1] - c.c.launchpad.pipeline.AbstractJob: Applying custom configurations of services [2016-10-04 07:19:05] ERROR [pipeline-thread-1] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: ClouderaManagerException{message="API call to Cloudera Manager failed. Method=ServicesResource.updateServiceConfig",causeClass=class javax.ws.rs.BadRequestException, causeMessage="HTTP 400 Bad Request"} at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:321) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:100) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.job.Job5.runUnchecked(Job5.java:34) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.job.Job5$$FastClassBySpringCGLIB$$54178505.invoke(<generated>) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.codahale.metrics.Timer.time(Timer.java:101) ~[metrics-core-3.1.2.jar!/:3.1.2] at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0] at sun.reflect.GeneratedMethodAccessor86.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_101] at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_101] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at com.cloudera.launchpad.bootstrap.cluster.AddServices$$EnhancerBySpringCGLIB$$94a41497.runUnchecked(<generated>) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:159) [launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:130) [launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) [guava-retrying-1.0.6.jar!/:na] at com.github.rholder.retry.Retryer.call(Retryer.java:110) [guava-retrying-1.0.6.jar!/:na] at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:99) [launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:125) [launchpad-pipeline-database-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) [launchpad-common-2.1.0.jar!/:2.1.0] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_101] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_101] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_101] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101] Caused by: com.cloudera.api.ext.ClouderaManagerException: API call to Cloudera Manager failed. Method=ServicesResource.updateServiceConfig at com.cloudera.api.ext.ClouderaManagerClientProxy.invoke(ClouderaManagerClientProxy.java:97) ~[launchpad-cloudera-manager-api-ext-2.1.0.jar!/:2.1.0] at com.sun.proxy.$Proxy182.updateServiceConfig(Unknown Source) ~[na:na] at com.cloudera.launchpad.bootstrap.cluster.AddServices.customConfigureServices(AddServices.java:504) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.bootstrap.cluster.AddServices.customConfigureServices(AddServices.java:466) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:304) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] ... 33 common frames omitted [2016-10-04 07:19:05] ERROR [pipeline-thread-1] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error
... View more
10-03-2016
03:07 PM
I tried changing this to a Roletype of Master and it didn't work I think I found the problem and am moving this to the Cluster definition. I believe the error may be how I am setting the values and think they should be w/o the double quotes..
... View more
10-03-2016
12:36 PM
All, I have successfully configured an AWS cluster with 8 nodes up and running. I have posted questions on how to confgure services, which I understand can be done via the API but was seeking to do this via the configuration file that is shipped to CD to create the CM and the rest of the nodes. However, when I set the values I do not see the any change in the values (using the CM UI). Below is my example.. when I try to set the HBASE parameters I do the following... roles { HDFS: [ JOURNALNODE, BALANCER] ZOOKEEPER: [SERVER] HBASE: [MASTER] } # # Configure services per ML team # config { HBASE { REGIONSERVER { hbase_regionsserver_msginterval: "25" hbase_hstore_compactionThreshold: "15" hbase_hstore_blockStoreFiles: "35" hbase_hstore_blockingWaitTime: "10" hbase_enable_replication: "true" hbase_snapshot_enabled: "true" hbase_enable_indexing: "true" hbase_active_master_detection_window: "10" hbase_server_dfs_client_hedged_read_threadpool_size: "10" hbase_regionserver_regionmover_thread_count: "4" } } and I don't see a change in the values. I patterned this after how Cloudera shows how to do the HA Config, but can't seem to get this to work.. ideas?
... View more
10-02-2016
05:35 AM
Curious.. can't we set the values for Hbase, HDFS, etc using the configuration file with the "Configs {} section after we identify the services?
... View more
09-29-2016
02:48 PM
Having configured a base system on AWS with CM, I want to programtically update the environment - hoefully through the use of the configuration file, to update specific paramaters. I found the API relevant names, example, hbase_regionserver_msginterval and want to change this from the default to 25. How is this to be done?
... View more
09-23-2016
08:18 AM
Below is my script that I got running.. but sanitized AWS credentials, network, security.. etc In shows how to attach multiple EBS volumes (2), set delete on terminate so that they clean up, tag the volumes/resources ################################################################################ # begin # name: INSERT-NAME-OF-CLUSTER #Ultimately should have comments on each line for understanding. provider { type: aws region: us-west-2 subnetId: INSERT AWS Subnet ID securityGroupsIds: INSERT SG-ID or create IAM Profile instanceNamePrefix: INSERT-NAME-OF-CLUSTER rootVolumeSizeGB: 100 rootVolumeType: gp2 # associatePublicIpAddresses: true # spotBimi-e3d81583dUSDPerHr: 0.50 } ssh { username: ec2-user # for RHEL image privateKey: keypair.pem # with an absolute path to .pem file } instances { m42x { type: m4.2xlarge # requires an HVM AMI image: INSERT AMI rootVolumeSizeGB: 100 # iamProfileName: iam-profile-REPLACE-ME tags { owner: ${?USER} Project: "INSERT Project name" Vertical: "Insert Project vertical" } bootstrapScript: """#!/bin/sh # # This is an embedded bootstrap script that runs as root and can be used to customize # the instances immediately after boot and before any other Cloudera Director action export AWS_ACCESS_KEY_ID=INSERT AWS Access Key export AWS_SECRET_ACCESS_KEY=INSERT AWS Secret Key export INSTANCE_ID=$(curl http://instance-data/latest/meta-data/instance-id) export AVAILABILITY_ZONE=$(curl http://instance-data/latest/meta-data/placement/availability-zone) export VOL1_ID=$(aws ec2 create-volume --size 15 --region us-west-2 --availability-zone ${AVAILABILITY_ZONE} --volume-type gp2 --query "VolumeId" | tr -d '"') export VOL2_ID=$(aws ec2 create-volume --size 15 --region us-west-2 --availability-zone ${AVAILABILITY_ZONE} --volume-type gp2 --query "VolumeId" | tr -d '"') echo 'Sleeping 30' sleep 30 echo 'Create additional EBS volumes' aws ec2 attach-volume --region us-west-2 --volume-id ${VOL1_ID} --instance-id ${INSTANCE_ID} --device /dev/sdf aws ec2 attach-volume --region us-west-2 --volume-id ${VOL2_ID} --instance-id ${INSTANCE_ID} --device /dev/sdg # Create filesystems on the devices echo 'Create filesystems' mkfs.ext4 /dev/xvdf mkfs.ext4 /dev/xvdg # Create directories for the mount points mkdir /data1 mkdir /data2 # Add the mount points to /etc/fstab echo "/dev/xvdf /data1 ext4 defaults 0 0" >> /etc/fstab echo "/dev/xvdg /data2 ext4 defaults 0 0" >> /etc/fstab sleep 10 # Mount all the devices echo 'Mount drives' mount -a # # modify attributes of new volumes to delete on terminate echo 'Modify volume attributes' aws ec2 modify-instance-attribute --instance-id ${INSTANCE_ID} --region us-west-2 --block-device-mappings "[{\"DeviceName\": \"/dev/sdf\",\"Ebs\":{\"DeleteOnTermination\":true}}]" aws ec2 modify-instance-attribute --instance-id ${INSTANCE_ID} --region us-west-2 --block-device-mappings "[{\"DeviceName\": \"/dev/sdg\",\"Ebs\":{\"DeleteOnTermination\":true}}]" # # tag the volumes echo 'Tag resources' aws ec2 create-tags --region us-west-2 --resources ${INSTANCE_ID} --tags Key=Application,Value=BIGDATA Key=Vertical,Value=VerticalName Key=Project,Value="Project name" aws ec2 create-tags --region us-west-2 --resources ${VOL1_ID} --tags Key=Application,Value=BIGDATA Key=Vertical,Value=VerticalName Key=Project,Value="Project name" aws ec2 create-tags --region us-west-2 --resources ${VOL2_ID} --tags Key=Application,Value=BIGDATA Key=Vertical,Value=VerticalName Key=Project,Value="Project name" # # If the exit code is not zero Cloudera Director will automatically retry echo 'This is the last line of the script' exit 0 """ # # Flag indicating whether to normalize the instance. Not setting normalization here implies that your # bootstrap script will take care of normalization. This is an advanced configuration that will require # assistance from Cloudera support. # # Normalization includes: # downloading and installing packages # minimizing swappiness # increasing the maximun number of open files # mounting ephemeral disks # resizing the root partition. # # Defaults to true # normalizeInstance: true } m44x { type: m4.4xlarge image: ami tags { owner: ${?USER} } } c34x { type: c3.4xlarge image: ami tags { owner: ${?USER} } } i2x { type: i2.xlarge image: ami tags { owner: ${?USER} } } i22x { type: i2.2xlarge image: ami tags { owner: ${?USER} } } r32x { type: r3.2xlarge image: ami tags { owner: ${?USER} } } t2l { # only suitable as a gateway type: t2.large image: ami tags { owner: ${?USER} } } } cloudera-manager { instance: ${instances.m42x} { tags { # add any additional tags as needed Application: "Cloudera Manager 5" Project: "place project name here" Vertical: "place vertical name here" } } repository: "http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.8/" repositoryKeyUrl: "http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/RPM-GPG-KEY-cloudera" enableEnterpriseTrial: true } cluster { products { CDH: 5 } configs { # HDFS fencing should be set to true for HA configurations HDFS { dfs_ha_fencing_methods: "shell(true)" } } services: [HDFS, ZOOKEEPER, HBASE ] hdfsmaster-1 { count: 2 instance: ${instances.m44x} { tags { group: hdfsmaster-1 } } roles { HDFS: [NAMENODE, JOURNALNODE, FAILOVERCONTROLLER ] ZOOKEEPER: [SERVER] } # NameNode nameservice, autofailover, and quorum journal name must be configured for high availability configs { HDFS { NAMENODE { dfs_federation_namenode_nameservice: hanameservice autofailover_enabled: true dfs_namenode_quorum_journal_name: hanameservice } } } } hdfsmaster-2 { count: 1 instance: ${instances.m44x} { tags { group: hdfsmaster-2 } } roles { HDFS: [ JOURNALNODE, BALANCER] ZOOKEEPER: [SERVER] HBASE: [MASTER] } } master-1 { count: 1 instance: ${instances.m44x} { tags { group: master } } roles { HBASE: [MASTER] ZOOKEEPER: [SERVER] } } workers { count: 3 minCount: 1 instance: ${instances.m42x} { # placementGroup: REPLACE-ME tags { group: worker } } roles { HDFS: [DATANODE] HBASE: [REGIONSERVER] } } postCreateScripts: ["""#!/bin/sh echo 'Hello World!' exit 0 """, """#!/usr/bin/python print 'Hello again!' """] preTerminateScripts: ["""#!/bin/sh echo 'Goodbye World!' exit 0 """, """#!/usr/bin/python print 'Goodbye again!' """] }
... View more
09-21-2016
04:15 PM
I can successfully create a cluster using the Cloudera provided template and nodes following the AWS template profile. Am seeking to script how to configure parameters such as hbase.regionserver.msginterval hbase.hstore.blockingStoreFiles hbase.snapshot.enabled... I presume that these will go in the post creation script... how?
... View more
09-16-2016
02:41 PM
Using the following Configuration file, it dies... This is patterned off of the provided example from Cloudera.. it builds fine when hdfs-master count = 1, but when making it 2 to have more than a single NameNode, etc.. it fails.. cloudera-manager { instance: ${instances.m42x} { tags { # add any additional tags as needed application: "Cloudera Manager 5" } } repository: "http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.8/" repositoryKeyUrl: "http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/RPM-GPG-KEY-cloudera" enableEnterpriseTrial: true } cluster { products { CDH: 5 } configs { # HDFS fencing should be set to true for HA configurations HDFS { dfs_ha_fencing_methods: "shell(true)" } } services: [HDFS, ZOOKEEPER, HBASE, HIVE ] hdfsmaster-1 { count: 2 instance: ${instances.m42x} { tags { group: hdfsmaster-1 } } roles { HDFS: [NAMENODE, JOURNALNODE, FAILOVERCONTROLLER, BALANCER] ZOOKEEPER: [SERVER] HBASE: [MASTER] } # NameNode nameservice, autofailover, and quorum journal name must be configured for high availability configs { HDFS { NAMENODE { dfs_federation_namenode_nameservice: hanameservice autofailover_enabled: true dfs_namenode_quorum_journal_name: hanameservice } } } } hdfsmaster-2 { count: 1 instance: ${instances.m42x} { tags { group: hdfsmaster-2 } } roles { HDFS: [ JOURNALNODE] ZOOKEEPER: [SERVER] } } master-1 { count: 1 instance: ${instances.m42x} { tags { group: master } } roles { HBASE: [MASTER] ZOOKEEPER: [SERVER] } } workers { count: 3 minCount: 1 instance: ${instances.c34x} { # placementGroup: REPLACE-ME tags { group: worker } } roles { HDFS: [DATANODE] HBASE: [REGIONSERVER] } } gateways { count: 1 instance: ${instances.t2l} { tags { group: gateway } } roles { HIVE: [GATEWAY] } } postCreateScripts: ["""#!/bin/sh echo 'Hello World!' exit 0 """, """#!/usr/bin/python print 'Hello again!' """] preTerminateScripts: ["""#!/bin/sh echo 'Goodbye World!' exit 0 """, """#!/usr/bin/python print 'Goodbye again!' """]
... View more