Member since
01-12-2017
15
Posts
0
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1373 | 05-11-2017 05:13 PM | |
268 | 07-01-2016 06:11 PM |
01-18-2018
11:12 AM
Thanks for the reply. I have tried something similar, but when I run the installer binary (./cloudera-manager-installer.bin), it gives an interactive prompt: Is there any way to get this installer to run without user input?
... View more
01-17-2018
03:07 PM
Context I'm working on setting up a Hadoop cluster consisting of multiple VMs (for testing and development purposes) on a local desktop machine. I have a proof of concept working, where I installed Cloudera Manager manually and then setup the cluster. But now I'd like to automate it. Problem To automate this process, I need to be able to script the installation of Cloudera Manager, so that it works without manual intervention. Using Cloudera Director doesn't seem to be an option for this use case, because we're not running the cluster on any cloud provider (AWS, Azure, etc.). Is there any way to automate the Cloudera Manager installation for a local desktop?
... View more
Labels:
05-11-2017
05:13 PM
I've figured out a way to do this. I used peerConfigGenerators in the SDL file. Peer configs create a properties file with key/value pairs--the key/value pairs themselves weren't useful to me, but each line of the file is prepended by hostname that the chose role is running on. I then parsed that file for the hostname in the control script.
... View more
05-09-2017
01:47 PM
Thanks for the reply @csguna. I have seen those links before, and haven't found the answer I'm looking for yet.
... View more
05-08-2017
04:51 PM
I'm creating a CSD for a custom service with multiple roles. Some of the roles need to have knowledge of which host(s) other roles are running on, similar to how a NameNode needs to know which hosts have DataNodes. I think this information should be accessible somehow after a user has chosen the role hosts in the CM UI, but I can't figure out how. So, how can I access a role's hostname from the service.sdl or control.sh files?
... View more
- Tags:
- csd
Labels:
02-03-2017
10:47 PM
Ok, thanks. Is there any property that will make HDFS/SECONDARY_NAMENODE the default node my service will be installed on? (even if the user is able to override manually)
... View more
02-03-2017
10:17 PM
When installing a service via an Ambari custom stack, I would like to restrict one component to only be installed on a node that has a specific component already installed on it. For example, I might want the component to only get installed on the same node as the secondary namenode. Is there a way to force or guarantee that?
I have tried adding something like the following to my service's metainfo file within the component level: <dependencies>
<dependency>
<name>HDFS/SECONDARY_NAMENODE</name>
<scope>host</scope>
<auto-deploy>
<enabled>true</enabled>
<co-locate>MY_SERVICE/MY_COMPONENT</co-locate>
</auto-deploy>
</dependency>
</dependencies>
But when I install through the Ambari UI, it still lets me install on any node.
... View more
Labels:
01-19-2017
02:12 PM
The service being installed needs to use another service's keytab. I have read the portion of the wiki about externalKerberosPrincipals, but it is not clear how that property should be used, or what changes other properties need to make. I have tried the following: 1) Adding this to the roles section "externalKerberosPrincipals" : [
{
"name" : "principal1",
"primary" : "${principal}",
"instance" : "${host}"
},
{
"name" : "principal2",
"primary" : "alice",
"instance" : "${host}"
}
] 2) Adding this to the roles.configWriter.generators "kerberosPrincipals" : [
{
"principalName" : "bluetalon_principal",
"propertyName" : "bluetalon.kerberos.principal",
"external" : "true",
"instanceWildcard" : "_HOST"
},
{
"principalName" : "bluetalon_principal2",
"propertyName" : "bluetalon.other.kerberos.principal",
"external" : "true",
"instanceWildcard" : "_HOST"
}
], The key things that need to happen for this to work are that the principal names and keytab locations need to get added to the service's site file. The reason they can't be added statically is that the principal names depend on the host and the realm e.g. alice/hostname1@EXAMPLE.COM, and the keytab location of the external service changes every time that service is restarted. Is there a way to use the sdl file to handle all of the restrictions I've mentioned?
... View more
Labels:
01-19-2017
11:42 AM
Hi Darren, I think that will functionally work, but when a user looks at properties in the Cloudera Manager UI, won't they see the old values? I think that will be confusing to a user. Is there a way to update the values in the UI as well? Thanks
... View more
01-12-2017
11:41 AM
I'm working on creating a CSD to integrate with Cloudera Manager. We have a configuation file (in hadoop xml format) that is being managed by the CSD, and there are certain properties in that file which need to be added (or changed), if and only if, the cluster is kerberized. For example, if installing through Cloudera Manager on a non-kerberized cluster, the following properties might be set: a.b.c = 1 d.e.f = 2 And if the cluster is kerberized, the properties might look like this: a.b.c = 1 d.e.f = 1000 g.h.i = 7 The changes can be done manually, of course, but I'm hoping to be able to automate it. There is a way to do this in Ambari, and I'm looking for something similar. Is there a way to do this in the SDL file, or anywhere else? I've gone through the documentation here: https://github.com/cloudera/cm_ext/wiki/Service-Descriptor-Language-Reference, and the only kerberos references I see are with respect to adding principals.
... View more
Labels:
07-21-2016
11:50 PM
Yes, I think that'll work. Can you point me to the relevant documentation?
... View more
07-21-2016
11:42 PM
I am trying to setup a new custom service, and I would like to be able to update Ambari's view of the configuration from the python script. However, when I try to set a value (e.g. Script.get_config()['configurations']['my-config-file']['my-property'] = "some data"), I see an error telling me I cannot update the immutable config dict. I see some methods in the Ambari python resource_management.core source such as "update_config", "set_params", etc. that seem like they might be what I need, but they are not well documented. The Ambari REST API is not an option either, unless I can determine the Ambari username/password from inside the script. Is there a good way for me to set a property from within the python script?
... View more
Labels:
07-01-2016
06:11 PM
I have figured out the issue. There was a typo; a directory was named ..../packages instead of ..../package
... View more
07-01-2016
04:47 PM
From the UI, I'm accessing as the admin user, but this exception is appearing when I restart ambari-server.
... View more
06-30-2016
10:50 PM
I have created a custom service in Ambari, I tested it in an HDP 2.4 sandbox and found that it worked correctly. Later I tried to install it on a new HDP 2.4 sandbox, and I've found that it is failing immediately because <SERVICE>/packages/archive.zip, and the <SERVICE>/packages/.hash are not being created. I have restarted both the Ambari server and Ambari agent several times with no luck. I have also turned on DEBUG mode in the logging and I'm seeing this message in ambari-server.log: 30 Jun 2016 22:20:50,373 DEBUG [qtp-ambari-client-26 - /api/v1/clusters/Sandbox/requests?to=end&page_size=10&fields=Requests&_=1467325359411] ExceptionTranslationFilter:165 - Access is denied (user is an\
onymous); redirecting to authentication entry point
org.springframework.security.access.AccessDeniedException: Access is denied
at org.springframework.security.access.vote.AffirmativeBased.decide(AffirmativeBased.java:83)
at org.springframework.security.access.intercept.AbstractSecurityInterceptor.beforeInvocation(AbstractSecurityInterceptor.java:206)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.apache.ambari.server.security.authorization.AmbariAuthorizationFilter.doFilter(AmbariAuthorizationFilter.java:196)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
at org.apache.ambari.server.api.MethodOverrideFilter.doFilter(MethodOverrideFilter.java:72)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
at org.apache.ambari.server.security.AbstractSecurityHeaderFilter.doFilter(AbstractSecurityHeaderFilter.java:109)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467)
at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82)
I have not enabled any security policies, so I'm not sure why this is happening. Any help would be appreciated!
... View more
Labels: