question Re: Metron Indexing stopped right after started in Support Questions

Metron Indexing stopped right after started

anggriawanrezab — Tue, 27 Mar 2018 13:42:57 GMT

When i start metron indexing service via ambari, the service will start successfully but stopped after few second.

i look into /var/log/metron/ directory but only found metron-rest.log. where is the log file for other metron component?

Thanks.

Re: Metron Indexing stopped right after started

asubramanian — Tue, 27 Mar 2018 13:57:21 GMT

@Bramantya Anggriawan

You need to look for the indexing logs under the storm worker logs. There are two topologies that run as a part of the Metron Indexing service--random_access_indexing and batch_indexing. You can view the respective logs under /var/log/storm/worker-logs/<indexing-topo-name>/6700/worker.log.

Btw, I would also suggest that you look at the Ambari agent log for any errors. I have seen earlier that the indexing service appearing stopped could possibly be an issue with python-requests package not installed as well (see https://issues.apache.org/jira/browse/METRON-1451).

Re: Metron Indexing stopped right after started

anggriawanrezab — Wed, 28 Mar 2018 10:25:40 GMT

@asubramanian

I look at ambari-agent.log and find this :

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/PythonReflectiveExecutor.py", line 59, in run_file
    imp.load_source('__main__', script)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.1.0/package/scripts/indexing_master.py", line 18, in <module>
    import requests
  File "/usr/lib/python2.7/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/usr/lib/python2.7/site-packages/urllib3/__init__.py", line 8, in <module>
    from .connectionpool import (
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 11, in <module>
    from .exceptions import (
  File "/usr/lib/python2.7/site-packages/urllib3/exceptions.py", line 2, in <module>
    from .packages.six.moves.http_client import (
  File "/usr/lib/python2.7/site-packages/urllib3/packages/six.py", line 198, in load_module
    return sys.modules[fullname]
AttributeError: 'NoneType' object has no attribute 'modules'

I already update the python-request but it looks like the problem is python-urllib3 now.

Re: Metron Indexing stopped right after started

asubramanian — Wed, 28 Mar 2018 13:25:38 GMT

Can you post the version of python-requests you have installed?

pip list | grep requests

Re: Metron Indexing stopped right after started

asubramanian — Wed, 28 Mar 2018 13:29:58 GMT

Also, can you describe your test environment more...

* full dev or bare metal

* version of Metron

Re: Metron Indexing stopped right after started

anggriawanrezab — Wed, 28 Mar 2018 14:03:29 GMT

@asubramanian

here is the result :

[root@workernode-01 ~]# pip list | grep requests
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
requests (2.18.4)
[root@workernode-01 ~]#

my cluster build on vm with RHIF bare metal, with metron version 0.4.1.1.4.1.0

Re: Metron Indexing stopped right after started

asubramanian — Wed, 28 Mar 2018 14:05:46 GMT

ok.. can you install the following version of requests and check if things work?

pip install requests==2.6.1

Re: Metron Indexing stopped right after started

logominesusa — Wed, 28 Mar 2018 14:38:37 GMT

Such a great post to know different useful information.

Logomines

Re: Metron Indexing stopped right after started

anggriawanrezab — Wed, 28 Mar 2018 15:23:18 GMT

@asubramanian

after i install request 2.6.1 like your advice, this error still persist :

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/ambari_agent/PythonReflectiveExecutor.py", line 59, in run_file
    imp.load_source('__main__', script)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.1.0/package/scripts/indexing_master.py", line 18, in <module>
    import requests
  File "/usr/lib/python2.7/site-packages/requests/__init__.py", line 53, in <module>
    from .packages.urllib3.contrib import pyopenssl
  File "/usr/lib/python2.7/site-packages/requests/packages/__init__.py", line 3, in <module>
    from . import urllib3
  File "/usr/lib/python2.7/site-packages/requests/packages/__init__.py", line 61, in load_module
AttributeError: 'NoneType' object has no attribute 'modules'

Re: Metron Indexing stopped right after started

asubramanian — Wed, 28 Mar 2018 15:55:22 GMT

Hm.. this error seems different than the earlier one. Can you list the version of urllib3 ? (pip list | grep url)

Re: Metron Indexing stopped right after started

anggriawanrezab — Wed, 28 Mar 2018 16:25:02 GMT

@asubramanian

below are urllib3 :

[root@workernode-01 ~]# pip list | grep url
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
pycurl (7.19.0)
urlgrabber (3.10)
urllib3 (1.22)

Re: Metron Indexing stopped right after started

anggriawanrezab — Fri, 13 Apr 2018 18:32:00 GMT

It actually running after i restart the server

Re: Metron Indexing stopped right after started

hacofayik — Mon, 17 Dec 2018 16:42:06 GMT

can you find solution for this problem ?

Re: Metron Indexing stopped right after started

asubramanian — Mon, 17 Dec 2018 17:04:56 GMT

Have you checked this issue with python-requests package not installed: https://issues.apache.org/jira/browse/METRON-1451 ?

Re: Metron Indexing stopped right after started

hacofayik — Mon, 17 Dec 2018 17:46:48 GMT

Hi @asubramanian ,

Yes , I installed python-requests but nothing changed.

Python module version that installed on server :

urllib3==1.24.1
pyOpenSSL==18.0.0
requests==2.6.1

I have this problem with HCP 1.7.1 On CentOS 7.