About jnlkds

jnlkds · ‎10-09-2016

All, We can set the parameters if we do the following and it was a tough lesson to learn.. There is a config session for Service wide parameters and then for specific areas, do not list or mix which parameters exist for which session as it will throw an error, but not an intelligent one. Additionally, I would recommend before setting parameters to run the script to dump the entire configuration into a file and then to review the specific parameters and the naming of them. I was using the GUI and seeing items like HBASE.replication and thought that this could be used, which was not the case. Instead, it was HBASE_enable_replication. I also found what I believe was a naming issue in that the name of the parameter did not align with the value, it wasn't spelled correctly like "..ion" and was "..on". The problem was I was spelling according to english vs reviewing the parameter name.

jnlkds · ‎10-04-2016

thank you for the timely follow-up... still trying to figure our configs section though ;-(

jnlkds · ‎10-04-2016

From first glance, when running the bootstrap command it appears as though you can run in multiple times and that logic might exist for it take take in any changes and incorporate them. This is based on my personal observation of running it, as it prompts the user informing that an existing configuration already exists and do they want to rebuild or resume. I have been trying to debug configuring the "Configs" section, both at the System-wide and at the Role-level. Given that the script errors out, i presumed that I could edit or update and then re-run the file. My oberservation is that the re-run does not take into account any changes made to the file being submitted, as i ended up commenting out a whole section of configs and the error continued to show-up. I then terminated and re-ran the script and everything worked fine (aka, I had no parameters in the config section). So, beware...

jnlkds · ‎10-04-2016

fyi - there is a type in the regionserver, as it only requires a single s

jnlkds · ‎10-04-2016

Problem not resolved but now get the following error [2016-10-04 07:19:05] INFO [pipeline-thread-1] - c.c.launchpad.pipeline.AbstractJob: Applying custom configurations of services [2016-10-04 07:19:05] ERROR [pipeline-thread-1] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: ClouderaManagerException{message="API call to Cloudera Manager failed. Method=ServicesResource.updateServiceConfig",causeClass=class javax.ws.rs.BadRequestException, causeMessage="HTTP 400 Bad Request"} at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:321) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:100) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.job.Job5.runUnchecked(Job5.java:34) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.job.Job5$$FastClassBySpringCGLIB$$54178505.invoke(<generated>) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0] at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.codahale.metrics.Timer.time(Timer.java:101) ~[metrics-core-3.1.2.jar!/:3.1.2] at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0] at sun.reflect.GeneratedMethodAccessor86.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_101] at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_101] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] at com.cloudera.launchpad.bootstrap.cluster.AddServices$$EnhancerBySpringCGLIB$$94a41497.runUnchecked(<generated>) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:159) [launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:130) [launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) [guava-retrying-1.0.6.jar!/:na] at com.github.rholder.retry.Retryer.call(Retryer.java:110) [guava-retrying-1.0.6.jar!/:na] at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:99) [launchpad-pipeline-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:125) [launchpad-pipeline-database-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) [launchpad-common-2.1.0.jar!/:2.1.0] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_101] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_101] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_101] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_101] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101] Caused by: com.cloudera.api.ext.ClouderaManagerException: API call to Cloudera Manager failed. Method=ServicesResource.updateServiceConfig at com.cloudera.api.ext.ClouderaManagerClientProxy.invoke(ClouderaManagerClientProxy.java:97) ~[launchpad-cloudera-manager-api-ext-2.1.0.jar!/:2.1.0] at com.sun.proxy.$Proxy182.updateServiceConfig(Unknown Source) ~[na:na] at com.cloudera.launchpad.bootstrap.cluster.AddServices.customConfigureServices(AddServices.java:504) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.bootstrap.cluster.AddServices.customConfigureServices(AddServices.java:466) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:304) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0] ... 33 common frames omitted [2016-10-04 07:19:05] ERROR [pipeline-thread-1] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error

jnlkds · ‎10-03-2016

I tried changing this to a Roletype of Master and it didn't work I think I found the problem and am moving this to the Cluster definition. I believe the error may be how I am setting the values and think they should be w/o the double quotes..

jnlkds · ‎10-03-2016

All, I have successfully configured an AWS cluster with 8 nodes up and running. I have posted questions on how to confgure services, which I understand can be done via the API but was seeking to do this via the configuration file that is shipped to CD to create the CM and the rest of the nodes. However, when I set the values I do not see the any change in the values (using the CM UI). Below is my example.. when I try to set the HBASE parameters I do the following... roles { HDFS: [ JOURNALNODE, BALANCER] ZOOKEEPER: [SERVER] HBASE: [MASTER] } # # Configure services per ML team # config { HBASE { REGIONSERVER { hbase_regionsserver_msginterval: "25" hbase_hstore_compactionThreshold: "15" hbase_hstore_blockStoreFiles: "35" hbase_hstore_blockingWaitTime: "10" hbase_enable_replication: "true" hbase_snapshot_enabled: "true" hbase_enable_indexing: "true" hbase_active_master_detection_window: "10" hbase_server_dfs_client_hedged_read_threadpool_size: "10" hbase_regionserver_regionmover_thread_count: "4" } } and I don't see a change in the values. I patterned this after how Cloudera shows how to do the HA Config, but can't seem to get this to work.. ideas?

jnlkds · ‎10-02-2016

Curious.. can't we set the values for Hbase, HDFS, etc using the configuration file with the "Configs {} section after we identify the services?

jnlkds · ‎09-29-2016

Having configured a base system on AWS with CM, I want to programtically update the environment - hoefully through the use of the configuration file, to update specific paramaters. I found the API relevant names, example, hbase_regionserver_msginterval and want to change this from the default to 25. How is this to be done?

jnlkds · ‎09-29-2016

let me know.. as i know it works.. no worries!

Online	Offline
Last Visited	‎11-16-2016 02:09 PM

Member Since	‎09-12-2016 12:02 PM
Last Visited	‎11-16-2016 02:09 PM
Posts	21
Kudos received	1

Cloudera Community

Cloudera Community

Re: AWS Configuration File not using Configs secti...

Re: Unable to configure CM 5.8 on AWS

Re: AWS Configuration File not using Configs secti...

Re: re-run Bootstap command does not take use of u...

re-run Bootstap command does not take use of updat...

Re: AWS Configuration File not using Configs secti...

Re: AWS Configuration File not using Configs secti...

Re: AWS Configuration File not using Configs secti...

AWS Configuration File not using Configs section t...

Re: Howto use CM API to reset values from standard...

Howto use CM API to reset values from standard con...

Re: Unable to configure CM 5.8 on AWS