Member since
01-06-2017
10
Posts
0
Kudos Received
0
Solutions
10-26-2017
02:04 AM
Yup. We got it resolved earlier this evening and that's what we did. Is this fact mentioned somewhere in the yarn installation/configuration portion? It's not like people setup kerberos right when they setup all the components before they are Kerberized, right? Anyhew, thanks for the response.
... View more
10-25-2017
05:17 PM
Hello, We just enabled FreeIPA integration in our Hortonworks cluster (HDP 2.5.3). We understand that part of the kerberization of Yarn, Ambari/yarn will try and delete the yarn.nodemanager.local-dirs and yarn.nodemanager.log-dirs. The directories defined for these configs in our environment are actual mount points, so as expected Yarn throws this error message "OSError: [Errno 16] Device or resource busy: '/hadoop/yarn/local/01". Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py", line 161, in <module>
Nodemanager().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py", line 51, in start
self.configure(env) # FOR SECURITY
File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py", line 57, in configure
yarn(name="nodemanager")
File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
return fn(*args, **kwargs)
File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py", line 168, in yarn
action='delete'
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 114, in __new__
cls(names_list.pop(0), env, provider, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 208, in action_delete
sudo.rmtree(path)
File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", line 102, in rmtree
shutil.rmtree(path)
File "/usr/lib64/python2.7/shutil.py", line 256, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/usr/lib64/python2.7/shutil.py", line 254, in rmtree
os.rmdir(path)
OSError: [Errno 16] Device or resource busy: '/hadoop/yarn/local/01' 2017-10-25 18:19:36,019 - checked_call returned (0, '')
2017-10-25 18:19:36,019 - Ensuring that hadoop has the correct symlink structure
2017-10-25 18:19:36,019 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2017-10-25 18:19:36,025 - Directory['/hadoop/yarn/local/01'] {'action': ['delete']}
2017-10-25 18:19:36,026 - Removing directory Directory['/hadoop/yarn/local/01'] and all its content
Command failed after 1 tries Any idea how do we get around this? Is there some script/flag we can modify for it to avoid attempting to delete these directories? Just an FYI, we can start the NM service on our DN manually. Any help would be appreciated. Thanks! K
... View more
Labels:
- Labels:
-
Apache YARN
01-09-2017
05:26 PM
Thanks! I shall work with our ops guys and test this out.
... View more
01-06-2017
11:36 PM
Hello, I am trying to create an ansible deployment script for HDP using Ambari blueprints. So far I can deploy clusters on an environment where I have internet access. However, I am unable to do so on environments that have their own local cached repo and no internet access for the servers. I get the following error when I am trying to modify the base repo url "org.apache.ambari.server.controller.spi.NoSuchResourceException: The specified resource doesn't exist: Stack data, Stack HDP 2.4 is not found in Ambari metainfo" This is when I set the repositories base url to our own internal repo: http://<internal_URL>:8080/repos/hortonworks/HDP/centos7/2.x/updates/2.4.2.0 FYI, I have confirmed that the above repo works. I understand I am getting the above exception because the info isn't there in the metainfo file. When I log on to the Ambari UI, I see the following stacks: Stacks
HDP 2.3
HDP 2.2
HDP 2.1 So I am trying to figure out if I am missing any steps after I do the ambari-server setup? Is my base URL wrong or have I missed changing any config file? I don't know where is it getting the previous versions of HDP. I can see them in the metainfo file under /var/lib/ but why are there previous versions of HDP present there. This is the repo file setup for me. #VERSION_NUMBER=2.2.0.0-1310
[Updates-ambari-2.2.0.0]
name=ambari-2.2.0.0 - Updates
baseurl=http://<repo_url>/repos/hortonworks/ambari-2.2.0.0/centos7/2.2.0.0-1310
gpgcheck=1
gpgkey=http://<repo_url>/repos/keys/RPM-GPG-KEY-ambari
enabled=1
priority=1
#VERSION_NUMBER=2.4.2.0-258
[HDP-2.4.2.0]
name=HDP Version - HDP-2.4.2.0
baseurl=http://<repo_url>/repos/hortonworks/HDP/centos7/2.x/updates/2.4.2.0
gpgcheck=1
gpgkey=http://<repo_url>/repos/keys/RPM-GPG-KEY-HDP
enabled=1
priority=1
[HDP-UTILS-1.1.0.20]
name=HDP Utils Version - HDP-UTILS-1.1.0.20
baseurl=http://<repo_url>/repos/hortonworks/HDP-UTILS-1.1.0.20/repos/centos7gpgcheck=1
gpgkey=http://<repo_url>/repos/keys/RPM-GPG-KEY-HDP-UTILSenabled=1
priority=1 I would be really grateful If anyone can point me to documentation which explains how I can setup a stack after I complete the Ambari server setup. I have seen the wiki (https://cwiki.apache.org/confluence/display/AMBARI/How-To+Define+Stacks+and+Services). There is nothing there that explains on how I can setup an HDP 2.4 Stack. Thank you!
... View more
- Tags:
- stackaccessexception