Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

hive installation error via ambari

avatar
Contributor

Hi experts,

I am trying to install hive using ambari which i downloaded the bigtop repo.

 

Here is the error:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py", line 79, in <module>
    HCatClient().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py", line 35, in install
    self.install_packages(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 821, in install_packages
    retry_count=agent_stack_retry_count)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 53, in action_install
    self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 264, in install_package
    self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput())
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 266, in checked_call_with_retries
    return self._call_with_retries(cmd, is_checked=True, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 283, in _call_with_retries
    code, out = func(cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install webhcat-tar-hive' returned 1. Error: Nothing to do

 Any help is very much appreciated.

 

Thanks,

 

1 ACCEPTED SOLUTION

avatar
Contributor

Here is an update.

So I finally was able to use ambari to get Hive installed and services started using the apache bigtop repo. Also was able to connect via hiveCLI as well as beeline(hiveserver2) and ran a simple "show databases;" which ran successfully.

 

So after symlinking the following directories, hiveserver2 finally was able to start successfully.

 

[root@test ~]# ll /usr/bgtp/current/
total 32
lrwxrwxrwx 1 root root 13 Apr 23 19:38 hive-client -> /usr/lib/hive
lrwxrwxrwx 1 root root 13 Apr 23 19:37 hive-metastore -> /usr/lib/hive
lrwxrwxrwx 1 root root 13 Apr 27 16:28 hive-server2 -> /usr/lib/hive
lrwxrwxrwx 1 root root 22 Apr 27 16:29 hive-webhcat -> /usr/lib/hive-hcatalog

 

I did not find any documentation on how to install hive with the apache bigtop packages using ambari. There was some documentation on how to install hive with the apache bigtop package using the command line.

 

If anyone finds any documentation on how to install the different components in the apache bigtop package, please let me know.

 

Thanks,

View solution in original post

8 REPLIES 8

avatar
Expert Contributor

Hi @ryu ,

 

It seems that your repo is not correctly configured.

 

Can you paste the output of "yum list webhcat-tar-hive*"?

 

Also, share the contents of /etc/yum.repos.d/<your repo file>.repo

 

Thanks,

Megh

avatar
Contributor

@vidanimegh  Thanks for the response.

Please find the below info...

 

[root@test ~]# yum list webhcat-tar-hive*
Loaded plugins: fastestmirror, rhnplugin
This system is receiving updates from RHN Classic or Red Hat Satellite.
Loading mirror speeds from cached hostfile
* base: bay.uchicago.edu
* epel: dl.fedoraproject.org
* extras: mirror.mobap.edu
* updates: ftp.ussg.iu.edu
Error: No matching Packages to list

 

[root@test ~]# cat /etc/yum.repos.d/bigtop.repo
[bigtop]
name=Bigtop
enabled=1
gpgcheck=1
baseurl=http://repos.bigtop.apache.org/releases/1.5.0/centos/7/$basearch
gpgkey=https://dist.apache.org/repos/dist/release/bigtop/KEYS
[root@test ~]# cat /etc/yum.repos.d/BGTP.repo
[BGTP-1.0]
name=BGTP-1.0
baseurl=http://repos.bigtop.apache.org/releases/1.5.0/centos/7/x86_64/

path=/
enabled=1
gpgcheck=0[root@test ~]#

 

avatar
Expert Contributor

I think the repo itself is broken.

 

navigating to http://repos.bigtop.apache.org/releases/1.5.0/centos/7/$basearch, takes to this:

<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>releases/1.5.0/centos/7/$basearch</Key>
<RequestId>K3BYRJWH03RKGFVK</RequestId>
<HostId>l+++bHT4W/dNczJrC5KWH7vPQ4dydVA6kLD69001OO8XQKpj6ziHZxLIsVCEIBsoZZT7o+1ILEQ=</HostId>
</Error>

 

Can you point to the documentation you're referring for this installation?

 

Thanks,

Megh

avatar
Contributor

@vidanimegh 

Thanks for the response.

However, I was able to install other components of hadoop such as zookeeper, hdfs, yarn, mapreduce using the bigtop mpack via ambari.

 

Also earlier ambari was doing a yum install hcatalog which it could not find the package.

But I modified the file "/var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml" so that it did a yum install hive-hcatalog.

 

If I modify that file again to ignore these packages, I am able to successfully install hive metastore and successfully accessed the hive database, however, am having issues starting hiveserver2 and webhcat as it seems to be looking for those files.

avatar
Contributor

@vidanimegh 

This is where I got the repos

https://bigtop.apache.org/download.html#releases

 

Also I have not found any documentation on how to deploy a hadoop cluster using ambari from the apache bigtop package.

All I found was that in the newest version they added a mpack which makes it easier for users to implement through ambari.

However, I have implemented the mpack, but it only consists of basic components such as hdfs, yarn, mapreduce, and zookeeper.

I was able to implement other components such as tez or sqoop successfully, but having issues with hive.

avatar
Contributor

When modifying the file "/var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml" to bypass the above errors, I am able to successfully install the different hive components(metastore, hiveserver2, and webhcat). I am also able to successfully start the metastore.

 

I commented out the following lines from the file "/var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml" to bypass the error.
<osSpecifics>
<osSpecific>
<osFamily>any</osFamily>
<packages>
<package>
<name>hive</name>
</package>
<package>
<name>hive-hcatalog</name>
</package>
<!--
<package>
<name>webhcat-tar-hive</name>
</package>
<package>
<name>webhcat-tar-pig</name>
</package>
<package>
<name>mysql-connector-java</name>
<skipUpgrade>true</skipUpgrade>
<condition>should_install_mysql_connector</condition>
</package>
-->
</packages>
</osSpecific>
<osSpecific>
<osFamily>amazon2015,redhat6,suse11,suse12</osFamily>
<packages>
<package>
<name>mysql</name>
<skipUpgrade>true</skipUpgrade>
</package>
</packages>
</osSpecific>

However, I get the following errors when trying to start hiveserver2 and webhcat.

2021-04-27 13:18:33,875 - WARNING. Cannot copy pig tarball because file does not exist: /usr/bgtp/1.0/pig/pig.tar.gz . It is possible that this component is not installed on this host.
2021-04-27 13:18:33,877 - WARNING. Cannot copy hive tarball because file does not exist: /usr/bgtp/1.0/hive/hive.tar.gz . It is possible that this component is not installed on this host.
2021-04-27 13:18:33,878 - WARNING. Cannot copy sqoop tarball because file does not exist: /usr/bgtp/1.0/sqoop/sqoop.tar.gz . It is possible that this component is not installed on this host.
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py", line 161, in <module>
    HiveServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py", line 77, in start
    self.configure(env) # FOR SECURITY
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 120, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py", line 51, in configure
    hive(name='hiveserver2')
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
    return fn(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py", line 269, in hive
    mode=0600)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/xml_config.py", line 66, in action_create
    encoding = self.resource.encoding
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 123, in action_create
    content = self._get_content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 160, in _get_content
    return content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 52, in __call__
    return self.get_content()
  File "/usr/lib/python2.6/site-packages/resource_management/core/source.py", line 144, in get_content
    rendered = self.template.render(self.context)
  File "/usr/lib/python2.6/site-packages/ambari_jinja2/environment.py", line 891, in render
    return self.environment.handle_exception(exc_info, True)
  File "<template>", line 2, in top-level template code
  File "/usr/lib/python2.6/site-packages/ambari_jinja2/filters.py", line 176, in do_dictsort
    return sorted(value.items(), key=sort_func)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py", line 73, in __getattr__
    raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'hiveserver2-site' was not found in configurations dictionary!

avatar
Contributor

UPDATE:
So I tried to reinstall hive again and to comment out those packages as i had mentioned earlier on this thread.
I just noticed the error on the bottom in ambari

resource_management.core.exceptions.Fail: Configuration parameter 'hiveserver2-site' was not found in configurations dictionary!

 So I ran the command 
cp /var/lib/ambari-server/resources/stacks/BigInsights/4.2.5/services/HIVE/configuration/hiveserver2-site.xml /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/configuration

 

restarted ambari and now it seems like I dont get any errors in ambari anymore.

Per ambari, it is able to start all the hive components successfully, however my hiveserver2 seems to be crashing after ambari starts it. All the other hive components remain running.

avatar
Contributor

Here is an update.

So I finally was able to use ambari to get Hive installed and services started using the apache bigtop repo. Also was able to connect via hiveCLI as well as beeline(hiveserver2) and ran a simple "show databases;" which ran successfully.

 

So after symlinking the following directories, hiveserver2 finally was able to start successfully.

 

[root@test ~]# ll /usr/bgtp/current/
total 32
lrwxrwxrwx 1 root root 13 Apr 23 19:38 hive-client -> /usr/lib/hive
lrwxrwxrwx 1 root root 13 Apr 23 19:37 hive-metastore -> /usr/lib/hive
lrwxrwxrwx 1 root root 13 Apr 27 16:28 hive-server2 -> /usr/lib/hive
lrwxrwxrwx 1 root root 22 Apr 27 16:29 hive-webhcat -> /usr/lib/hive-hcatalog

 

I did not find any documentation on how to install hive with the apache bigtop packages using ambari. There was some documentation on how to install hive with the apache bigtop package using the command line.

 

If anyone finds any documentation on how to install the different components in the apache bigtop package, please let me know.

 

Thanks,