Member since
04-08-2018
64
Posts
2
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 8975 | 05-04-2018 05:01 PM | |
| 12814 | 04-16-2018 10:13 AM |
04-16-2018
10:13 AM
I solved this problem by opening the ports 2888 and 3888 that are used by ZooKeeper nodes for communicating between each other.
... View more
04-16-2018
09:54 AM
Sorry, yes, there is more content in LOG file. Please see the error messages below. There seem to be a problem with ZooKeeper, but I do not have any alerts for ZooKeeper. Zookeeper seems to run well: # jps -l | grep -i zookeeper 5043 org.apache.zookeeper.server.quorum.QuorumPeerMain # netstat -anp | grep 2181 tcp6 0 0 :::2181 :::* LISTEN 5043/java Errors log: 2018-04-16 09:16:23,821 ERROR resourcemanager.ResourceManager (LogAdapter.java:error(69)) - RECEIVED SIGNAL 15: SIGTERM 2018-04-16 09:16:25,315 INFO zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1019)) - Opening socket connection to server eureambarislave1.local.eurecat.org/192.168.0.10:2181. Will not attempt to authenticate using SASL (unknown error) 2018-04-16 09:16:25,316 INFO zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(864)) - Socket connection established, initiating session, client: /192.168.0.15:53808, server: eureambarislave1.local.eurecat.org/192.168.0.10:2181 2018-04-16 09:16:25,316 INFO zookeeper.ClientCnxn (ClientCnxn.java:run(1142)) - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect 2018-04-16 09:16:25,417 INFO recovery.ZKRMStateStore (ZKRMStateStore.java:runWithRetries(1227)) - Exception while executing a ZK operation.
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /rmstore
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$1.run(ZKRMStateStore.java:326)
... View more
04-16-2018
09:41 AM
No, I do not have ResourceManager HA enabled. This is the content of "yarn-yarn-resourcemanager-eureambarislave2.local.eurecat.org.log": 2018-04-15 23:13:33,362 INFO resourcemanager.ResourceManager (LogAdapter.java:info(45)) - STARTUP_MSG:
/************************************************************ STARTUP_MSG: Starting ResourceManager STARTUP_MSG: user = yarn STARTUP_MSG: host = eureambarislave2.local.eurecat.org/192.168.0.15
STARTUP_MSG: args = [] STARTUP_MSG: version = 2.7.3.2.6.4.0-91 And this is the content of "yarn-yarn-resourcemanager-eureambarislave2.local.eurecat.org.out": SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
(-i) 64019 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 32768 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 65536 virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
... View more
04-16-2018
09:27 AM
This is what I get: # jps -l | grep -i resourcemanager 4274 org.apache.hadoop.yarn.server.resourcemanager.ResourceManager # netstat -anp | grep 8088
[root@eureambarislave2 sbin] # The netstat returns nothing.
... View more
04-16-2018
09:13 AM
By the way, I opened the port 8088 as follows: iptables -I INPUT 1 -p tcp --dport 8088 -j ACCEPT Is it correct?
... View more
04-16-2018
08:54 AM
I get the following output after running this command: root 811 31804 0 08:51 pts/0 00:00:00 grep --color=auto lsof -i :8088 | grep -i listen | awk '{print }' But the command "netstat -tunalp | grep LISTEN" does not show the port 8088.
... View more
04-15-2018
11:21 PM
I want to add Hive service to my cluster. In the Add Service Wizard I see the attached screen. It looks like MySQL is the default option. So, what is a recommended way to add Hive? I do not have any special requirement for selecting MySQL. My final goal is to install Spark2 which requires Hive.
... View more
04-15-2018
09:32 PM
By the way, this is how I opened the port 8088: iptables -I INPUT 1 -p tcp --dport 8088 -j ACCEPT Also, I created a TCP security rule for 8088. What do I miss?
... View more
04-15-2018
09:19 PM
The "jps" command shows that ResourceManager is running. However, there is an alert next to ResourceManager that says "Connection failed to http://eureambarislave2.local.eurecat.org:8088 (<urlopen error [Errno 111] Connection refused>). I restarted everything several times and checked that the port 8088 is opened. But the error still exists. I cannot access ResourceManager in a browser.
... View more
04-15-2018
09:14 PM
When I add Hive service, MySQL installation fails with the following error: Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_server.py", line 64, in <module>
MysqlServer().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 375, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/mysql_server.py", line 33, in install
self.install_packages(env)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 821, in install_packages
retry_count=agent_stack_retry_count)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 53, in action_install
self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 264, in install_package
self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput())
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 266, in checked_call_with_retries
return self._call_with_retries(cmd, is_checked=True, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 283, in _call_with_retries
code, out = func(cmd, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install mysql-community-release' returned 1. Error: Nothing to do stdout: /var/lib/ambari-agent/data/output-129.txt 2018-04-15 21:03:50,120 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=None -> 2.6
2018-04-15 21:03:50,126 - Using hadoop conf dir: /usr/hdp/2.6.4.0-91/hadoop/conf
2018-04-15 21:03:50,127 - Group['livy'] {}
2018-04-15 21:03:50,128 - Group['spark'] {}
2018-04-15 21:03:50,129 - Group['hdfs'] {}
2018-04-15 21:03:50,129 - Group['hadoop'] {}
2018-04-15 21:03:50,129 - Group['users'] {}
2018-04-15 21:03:50,130 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-04-15 21:03:50,130 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-04-15 21:03:50,131 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-04-15 21:03:50,132 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-04-15 21:03:50,133 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-04-15 21:03:50,134 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None}
2018-04-15 21:03:50,134 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users'], 'uid': None}
2018-04-15 21:03:50,135 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hdfs'], 'uid': None}
2018-04-15 21:03:50,136 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-04-15 21:03:50,137 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-04-15 21:03:50,138 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop'], 'uid': None}
2018-04-15 21:03:50,138 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2018-04-15 21:03:50,140 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2018-04-15 21:03:50,145 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa 0'] due to not_if
2018-04-15 21:03:50,145 - Group['hdfs'] {}
2018-04-15 21:03:50,145 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hdfs', u'hdfs']}
2018-04-15 21:03:50,146 - FS Type:
2018-04-15 21:03:50,146 - Directory['/etc/hadoop'] {'mode': 0755}
2018-04-15 21:03:50,161 - File['/usr/hdp/2.6.4.0-91/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2018-04-15 21:03:50,162 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2018-04-15 21:03:50,176 - Repository['HDP-2.6-repo-1'] {'append_to_file': False, 'base_url': 'http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0', 'action': ['create'], 'components': [u'HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2018-04-15 21:03:50,183 - File['/etc/yum.repos.d/ambari-hdp-1.repo'] {'content': '[HDP-2.6-repo-1]\nname=HDP-2.6-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0\n\npath=/\nenabled=1\ngpgcheck=0'}
2018-04-15 21:03:50,184 - Writing File['/etc/yum.repos.d/ambari-hdp-1.repo'] because contents don't match
2018-04-15 21:03:50,184 - Repository['HDP-2.6-GPL-repo-1'] {'append_to_file': True, 'base_url': 'http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.4.0', 'action': ['create'], 'components': [u'HDP-GPL', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2018-04-15 21:03:50,188 - File['/etc/yum.repos.d/ambari-hdp-1.repo'] {'content': '[HDP-2.6-repo-1]\nname=HDP-2.6-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-2.6-GPL-repo-1]\nname=HDP-2.6-GPL-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.4.0\n\npath=/\nenabled=1\ngpgcheck=0'}
2018-04-15 21:03:50,188 - Writing File['/etc/yum.repos.d/ambari-hdp-1.repo'] because contents don't match
2018-04-15 21:03:50,188 - Repository['HDP-UTILS-1.1.0.22-repo-1'] {'append_to_file': True, 'base_url': 'http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7', 'action': ['create'], 'components': [u'HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'ambari-hdp-1', 'mirror_list': None}
2018-04-15 21:03:50,192 - File['/etc/yum.repos.d/ambari-hdp-1.repo'] {'content': '[HDP-2.6-repo-1]\nname=HDP-2.6-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.4.0\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-2.6-GPL-repo-1]\nname=HDP-2.6-GPL-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP-GPL/centos7/2.x/updates/2.6.4.0\n\npath=/\nenabled=1\ngpgcheck=0\n[HDP-UTILS-1.1.0.22-repo-1]\nname=HDP-UTILS-1.1.0.22-repo-1\nbaseurl=http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7\n\npath=/\nenabled=1\ngpgcheck=0'}
2018-04-15 21:03:50,192 - Writing File['/etc/yum.repos.d/ambari-hdp-1.repo'] because contents don't match
2018-04-15 21:03:50,192 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-04-15 21:03:50,271 - Skipping installation of existing package unzip
2018-04-15 21:03:50,271 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-04-15 21:03:50,280 - Skipping installation of existing package curl
2018-04-15 21:03:50,280 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-04-15 21:03:50,288 - Skipping installation of existing package hdp-select
2018-04-15 21:03:50,293 - The repository with version 2.6.4.0-91 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2018-04-15 21:03:50,299 - Skipping stack-select on MYSQL_SERVER because it does not exist in the stack-select package structure.
2018-04-15 21:03:50,489 - MariaDB RedHat Support: false
2018-04-15 21:03:50,493 - Using hadoop conf dir: /usr/hdp/2.6.4.0-91/hadoop/conf
2018-04-15 21:03:50,507 - call['ambari-python-wrap /usr/bin/hdp-select status hive-server2'] {'timeout': 20}
2018-04-15 21:03:50,529 - call returned (0, 'hive-server2 - 2.6.4.0-91')
2018-04-15 21:03:50,530 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=None -> 2.6
2018-04-15 21:03:50,565 - File['/var/lib/ambari-agent/cred/lib/CredentialUtil.jar'] {'content': DownloadSource('http://eureambarimaster1.local.eurecat.org:8080/resources/CredentialUtil.jar'), 'mode': 0755}
2018-04-15 21:03:50,566 - Not downloading the file from http://eureambarimaster1.local.eurecat.org:8080/resources/CredentialUtil.jar, because /var/lib/ambari-agent/tmp/CredentialUtil.jar already exists
2018-04-15 21:03:50,566 - checked_call[('/usr/jdk64/jdk1.8.0_112/bin/java', '-cp', u'/var/lib/ambari-agent/cred/lib/*', 'org.apache.ambari.server.credentialapi.CredentialUtil', 'get', 'javax.jdo.option.ConnectionPassword', '-provider', u'jceks://file/var/lib/ambari-agent/cred/conf/mysql_server/hive-site.jceks')] {}
2018-04-15 21:03:51,189 - checked_call returned (0, 'SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".\nSLF4J: Defaulting to no-operation (NOP) logger implementation\nSLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.\nApr 15, 2018 9:03:50 PM org.apache.hadoop.util.NativeCodeLoader <clinit>\nWARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\neurecat123')
2018-04-15 21:03:51,197 - Package['mysql-community-release'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2018-04-15 21:03:51,278 - Installing package mysql-community-release ('/usr/bin/yum -d 0 -e 0 -y install mysql-community-release')
2018-04-15 21:03:52,017 - Execution of '/usr/bin/yum -d 0 -e 0 -y install mysql-community-release' returned 1. Error: Nothing to do
2018-04-15 21:03:52,017 - Failed to install package mysql-community-release. Executing '/usr/bin/yum clean metadata'
2018-04-15 21:03:52,211 - Retrying to install package mysql-community-release after 30 seconds
2018-04-15 21:04:27,630 - The repository with version 2.6.4.0-91 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2018-04-15 21:04:27,636 - Skipping stack-select on MYSQL_SERVER because it does not exist in the stack-select package structure.
Command failed after 1 tries
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hive