Support Questions
Find answers, ask questions, and share your expertise

all HDP service of ambari failed to start , help please !!!!!

all HDP service of ambari failed to start , help please !!!!!

Expert Contributor

all HDP service of ambari failed to start , yesterday i have install ranger then restart all service and it's work , today nothing wanna start

5 REPLIES 5

Re: all HDP service of ambari failed to start , help please !!!!!

@Mourad Chahri Can you please pass more details about the error you see.

What error you get in ambari UI?

PLease check service logs also.

Re: all HDP service of ambari failed to start , help please !!!!!

Expert Contributor
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py", line 58, in <module>
    AmsMonitor().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_monitor.py", line 40, in start
    action = 'start'
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
    return fn(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/ams_service.py", line 79, in ams_service
    user=params.ams_user
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 258, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
    raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/sbin/ambari-metrics-monitor --config /etc/ambari-metrics-monitor/conf/ start' returned 255. psutil build directory is not empty, continuing...
Verifying Python version compatibility...
Using python  /usr/bin/python2.6
Checking for previously running Metric Monitor...
tput: No value for $TERM and no -T specified
ERROR: ambari-metrics-monitor already running
tput: No value for $TERM and no -T specified
Check /var/run/ambari-metrics-monitor/ambari-metrics-monitor.pid for PID.

					
				
			
			
				
			
			
			
			
			
			
			
		

Re: all HDP service of ambari failed to start , help please !!!!!

Expert Contributor
stdout: /var/lib/ambari-agent/data/output-1454.txt
2016-08-25 10:15:35,539 - Group['hadoop'] {'ignore_failures': False}
2016-08-25 10:15:35,552 - Group['users'] {'ignore_failures': False}
2016-08-25 10:15:35,553 - Group['knox'] {'ignore_failures': False}
2016-08-25 10:15:35,558 - Group['ranger'] {'ignore_failures': False}
2016-08-25 10:15:35,558 - Group['spark'] {'ignore_failures': False}
2016-08-25 10:15:35,572 - User['hdfs'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,578 - User['knox'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,579 - User['ranger'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['ranger']}
2016-08-25 10:15:35,580 - User['storm'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,586 - User['spark'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,588 - User['mapred'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,589 - User['accumulo'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,594 - User['hbase'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,599 - User['tez'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['users']}
2016-08-25 10:15:35,602 - User['zookeeper'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,605 - User['mahout'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,606 - User['kafka'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,611 - User['falcon'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['users']}
2016-08-25 10:15:35,612 - User['sqoop'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,617 - User['yarn'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,618 - User['hcat'] {'gid': 'hadoop', 'ignore_failures': False, 'groups': ['hadoop']}
2016-08-25 10:15:35,675 - Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2016-08-25 10:15:35,702 - Skipping Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if
2016-08-25 10:15:35,703 - Group['hdfs'] {'ignore_failures': False}
2016-08-25 10:15:35,704 - User['hdfs'] {'ignore_failures': False, 'groups': ['hadoop', 'hdfs']}
2016-08-25 10:15:35,705 - Directory['/etc/hadoop'] {'mode': 0755}
2016-08-25 10:15:35,767 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2016-08-25 10:15:35,816 - Execute['('setenforce', '0')'] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2016-08-25 10:15:37,375 - File['/etc/ambari-metrics-monitor/conf//ams-env.sh'] {'content': InlineTemplate(...), 'owner': 'ams'}
2016-08-25 10:15:37,380 - Execute['/usr/sbin/ambari-metrics-monitor --config /etc/ambari-metrics-monitor/conf/ start'] {'user': 'ams'}

Re: all HDP service of ambari failed to start , help please !!!!!

Explorer

I could see this message in your snippet ERROR: ambari-metrics-monitor already running can you check the value in /var/run/ambari-metrics-monitor/ambari-metrics-monitor.pid and do a ps -ef | grep on that pid value to see if that is running still. Kill it if it still running and move /var/run/ambari-metrics-monitor/ambari-metrics-monitor.pid to /var/run/ambari-metrics-monitor/ambari-metrics-monitor.pid_old and try restarting the ams again. @Mourad Chahri

Re: all HDP service of ambari failed to start , help please !!!!!

@Mourad Chahri

On the ambari metrics server can you try below -

1. Login to the node. Stop ambari-server and agent on the node.

2. move the file - /var/lib/ambari-agent/data/structured-out-status.json to other location

3.Try restarting ambari server and agent.

4. Check if that works.