Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

Solved Go to solution
Highlighted

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

Mentor

@sun pepper please paste the ambari agent logs.

Highlighted

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

ambari-agentlog.txt

Sequence: Cleared Previous Agent Logs --> Ambari Login -> Nodes -> Node manager -> Start

Waited for few mins

Node Manager went down automatic.

Highlighted

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

#2: Sequence: Truncate Ambari Server and Agent logs

Start NodeManager, Then start HiveServer2

Logs Attached.

ambari-serverloghive.txt

ambari-agentloghive.txt

Highlighted

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

Mentor
ERROR 2016-10-26 08:09:38,619 script_alert.py:119 - [Alert][yarn_nodemanager_health] Failed with result CRITICAL: ['Connection failed to http://xxxxx.example.com:8042


Can you make sure firewall ports are open to this machine and network connectivity is functional.
Highlighted

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

Mentor

@sun pepper port 8042 belongs to YARN nodemanager API. Please see this https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.2/bk_installing_manually_book/content/configu...

also make sure ip4firewall is off as well as SELinux. If all else fails try using telnet to that port to make sure it is open.

Highlighted

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

Please see my Firewall and other info below. No luck still. Cannot get my NodeManager, Hive UP. Those are must haves for me.

Highlighted

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

@Artem Ervits may I know which service runs on 8042? I looked at this to see which one it is. can't fine 8042.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_HDP_Reference_Guide/content/accumulo-por...

ip6tables: was running. I just turned if off now.

iptables: was off since couple of days.

[root@~]# service ip6tables status

ip6tables: Firewall is not running.

[root@~]# service iptables status

iptables: Firewall is not running.

Tried Restarting NodeManager. Still Same

ERROR 2016-10-26 09:45:38,614 script_alert.py:119 - [Alert][yarn_nodemanager_health] Failed with result CRITICAL: ['Connection failed to http://node09.example.com:8042/ws/v1/node/info (Traceback (most recent call last):\n File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/alerts/alert_nodemanager_health.py", line 171, in execute\n url_response = urllib2.urlopen(query, timeout=connection_timeout)\n File "/usr/lib64/python2.6/urllib2.py", line 126, in urlopen\n return _opener.open(url, data, timeout)\n File "/usr/lib64/python2.6/urllib2.py", line 391, in open\n response = self._open(req, data)\n File "/usr/lib64/python2.6/urllib2.py", line 409, in _open\n \'_open\', req)\n File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain\n result = func(*args)\n File "/usr/lib64/python2.6/urllib2.py", line 1190, in http_open\n return self.do_open(httplib.HTTPConnection, req)\n File "/usr/lib64/python2.6/urllib2.py", line 1165, in do_open\n raise URLError(err)\nURLError: <urlopen error [Errno 111] Connection refused>\n)']

Highlighted

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

One more Puzzle. SafeMode turns ON automatic some times.

hdfs dfsadmin -fs hdfs://node09.example.com:8020 -safemode get

Safe mode is ON

I manually turned off:

[hdfs@node09 ~]$ hdfs dfsadmin -fs hdfs://node09.example.com:8020 -safemode leave

Then:

-- Restarted NameNode

-- Started NodeManager: Still turns off after some time

-- Started HiveServer2: Still doesnt come up. Never did so far.

Re: HDP2.4.3::Ambari 2.4.1.0 ::HiveServer2, History Server and NodeManager Not Starting

*WORKED*

1: su - hdfs

hdfs dfs -put /usr/hdp/2.4.3.0-227/hadoop/mapreduce.tar.gz /hdp/apps/2.4.3.0-227/mapreduce/

3: su - atlas

cp /usr/hdp/2.4.3.0-227/etc/atlas/conf.dist/client.properties /etc/atlas/conf/

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here