Cloudera Community
jnlkds
user rank
Contributor
My Accepted Solutions
Show All

Re: AWS Configuration File not using Configs secti...

by Contributor in Support Questions
‎10-09-2016 05:42 AM
‎10-09-2016 05:42 AM
All,   We can set the parameters if we do the following and it was a tough lesson to learn..   There is a config session for Service wide parameters and then for specific areas, do not list or mix which parameters exist for which session as it will throw an error, but not an intelligent one.    Additionally, I would recommend before setting parameters to run the script to dump the entire configuration into a file and then to review the specific parameters and the naming of them.  I was using the GUI and seeing items like HBASE.replication and thought that this could be used, which was not the case.  Instead, it was HBASE_enable_replication.   I also found what I believe was a naming issue in that the name of the parameter did not align with the value, it wasn't spelled correctly like "..ion" and was "..on".  The problem was I was spelling according to english vs reviewing the parameter name.     ... View more

Re: re-run Bootstap command does not take use of u...

by Contributor in Support Questions
‎10-04-2016 01:53 PM
‎10-04-2016 01:53 PM
thank you for the timely follow-up... still trying to figure our configs section though ;-( ... View more

re-run Bootstap command does not take use of updat...

by Contributor in Support Questions
‎10-04-2016 01:13 PM
‎10-04-2016 01:13 PM
From first glance, when running the bootstrap command it appears as though you can run in multiple times and that logic might exist for it take take in any changes and incorporate them.   This is based on my personal observation of running it, as it prompts the user informing that an existing configuration already exists and do they want to rebuild or resume.  I have been trying to debug configuring the "Configs" section, both at the System-wide and at the Role-level.  Given that the script errors out, i presumed that I could edit or update and then re-run the file.   My oberservation is that the re-run does not take into account any changes made to the file being submitted, as i ended up commenting out a whole section of configs and the error continued to show-up. I then terminated and re-ran the script and everything worked fine (aka, I had no parameters in the config section).   So, beware... ... View more

Re: AWS Configuration File not using Configs secti...

by Contributor in Support Questions
‎10-04-2016 08:30 AM
‎10-04-2016 08:30 AM
fyi - there is a type in the regionserver, as it only requires a single s ... View more

Re: AWS Configuration File not using Configs secti...

by Contributor in Support Questions
‎10-04-2016 08:29 AM
‎10-04-2016 08:29 AM
Problem not resolved but now get the following error [2016-10-04 07:19:05] INFO  [pipeline-thread-1] - c.c.launchpad.pipeline.AbstractJob: Applying custom configurations of services [2016-10-04 07:19:05] ERROR [pipeline-thread-1] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: ClouderaManagerException{message="API call to Cloudera Manager failed. Method=ServicesResource.updateServiceConfig",causeClass=class javax.ws.rs.BadRequestException, causeMessage="HTTP 400 Bad Request"}         at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:321) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:100) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.pipeline.job.Job5.runUnchecked(Job5.java:34) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.pipeline.job.Job5$$FastClassBySpringCGLIB$$54178505.invoke(<generated>) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0]         at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ~[spring-core-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at com.cloudera.launchpad.pipeline.PipelineJobProfiler$1.call(PipelineJobProfiler.java:67) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0]         at com.codahale.metrics.Timer.time(Timer.java:101) ~[metrics-core-3.1.2.jar!/:3.1.2]         at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:63) ~[launchpad-pipeline-2.1.0.jar!/:2.1.0]         at sun.reflect.GeneratedMethodAccessor86.invoke(Unknown Source) ~[na:na]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_101]         at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_101]         at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]         at com.cloudera.launchpad.bootstrap.cluster.AddServices$$EnhancerBySpringCGLIB$$94a41497.runUnchecked(<generated>) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:159) [launchpad-pipeline-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:130) [launchpad-pipeline-2.1.0.jar!/:2.1.0]         at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78) [guava-retrying-1.0.6.jar!/:na]         at com.github.rholder.retry.Retryer.call(Retryer.java:110) [guava-retrying-1.0.6.jar!/:na]         at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:99) [launchpad-pipeline-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:125) [launchpad-pipeline-database-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57) [launchpad-common-2.1.0.jar!/:2.1.0]         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_101]         at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_101]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_101]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_101]         at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101] Caused by: com.cloudera.api.ext.ClouderaManagerException: API call to Cloudera Manager failed. Method=ServicesResource.updateServiceConfig         at com.cloudera.api.ext.ClouderaManagerClientProxy.invoke(ClouderaManagerClientProxy.java:97) ~[launchpad-cloudera-manager-api-ext-2.1.0.jar!/:2.1.0]         at com.sun.proxy.$Proxy182.updateServiceConfig(Unknown Source) ~[na:na]         at com.cloudera.launchpad.bootstrap.cluster.AddServices.customConfigureServices(AddServices.java:504) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.bootstrap.cluster.AddServices.customConfigureServices(AddServices.java:466) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0]         at com.cloudera.launchpad.bootstrap.cluster.AddServices.run(AddServices.java:304) ~[launchpad-bootstrap-2.1.0.jar!/:2.1.0]         ... 33 common frames omitted [2016-10-04 07:19:05] ERROR [pipeline-thread-1] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error   ... View more

Re: AWS Configuration File not using Configs secti...

by Contributor in Support Questions
‎10-03-2016 03:07 PM
‎10-03-2016 03:07 PM
I tried changing this to a Roletype of Master and it didn't work   I think I found the problem and am moving this to the Cluster definition. I believe the error may be how I am setting the values and think they should be w/o the double quotes.. ... View more

AWS Configuration File not using Configs section t...

by Contributor in Support Questions
‎10-03-2016 12:36 PM
‎10-03-2016 12:36 PM
All,   I have successfully configured an AWS cluster with 8 nodes up and running.  I have posted questions on how to confgure services, which I understand can be done via the API but was seeking to do this via the configuration file that is shipped to CD to create the CM and the rest of the nodes.   However, when I set the values I do not see the any change in the values (using the CM UI).   Below is my example.. when I try to set the HBASE parameters I do the following...         roles {         HDFS: [ JOURNALNODE, BALANCER]         ZOOKEEPER: [SERVER]         HBASE: [MASTER]       } # # Configure services per ML team #         config {                 HBASE {                 REGIONSERVER {                         hbase_regionsserver_msginterval: "25"                         hbase_hstore_compactionThreshold: "15"                         hbase_hstore_blockStoreFiles: "35"                         hbase_hstore_blockingWaitTime: "10"                         hbase_enable_replication: "true"                         hbase_snapshot_enabled: "true"                         hbase_enable_indexing: "true"                         hbase_active_master_detection_window: "10"                         hbase_server_dfs_client_hedged_read_threadpool_size: "10"                         hbase_regionserver_regionmover_thread_count: "4"                         }                 }   and I don't see a change in the values.   I patterned this after how Cloudera shows how to do the HA Config, but can't seem to get this to work..   ideas? ... View more

Re: Howto use CM API to reset values from standard...

by Contributor in Support Questions
‎10-02-2016 05:35 AM
‎10-02-2016 05:35 AM
Curious.. can't we set the values for Hbase, HDFS, etc using the configuration file with the "Configs {} section after we identify the services? ... View more

Howto use CM API to reset values from standard con...

by Contributor in Support Questions
‎09-29-2016 02:48 PM
‎09-29-2016 02:48 PM
Having configured a base system on AWS with CM, I want to programtically update the environment - hoefully through the use of the configuration file, to update specific paramaters.   I found the API relevant names, example, hbase_regionserver_msginterval and want to change this from the default to 25.   How is this to be done?     ... View more
Labels:

Re: Unable to configure CM 5.8 on AWS

by Contributor in Support Questions
‎09-29-2016 01:29 PM
‎09-29-2016 01:29 PM
let me know.. as i know it works.. no worries! ... View more
Contact Me
Online
Offline
Last Visited
‎11-16-2016 02:09 PM
Community Statistics
Member Since ‎09-12-2016 12:02 PM
Last Visited ‎11-16-2016 02:09 PM
Posts 21
Kudos received 1