Superset of Druid can't be started in HDP 2.6.0.3

[#1]

Hi guys,

I have encountered a problem about Superset of Druid. I have installed HDP2.6.0.3 and Ambari 2.5.0.3, and added the service Druid ,but the sub-service Superset of Druid is not able to be started .

the error log is blow:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 169, in <module>
    Superset().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 108, in start
    self.configure(env, upgrade_type=upgrade_type)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 117, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 84, in configure
    user=params.druid_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'source /etc/superset/conf/superset-env.sh ; /usr/hdp/current/druid-superset/bin/superset db upgrade' returned 1. /usr/hdp/2.6.0.3-8/superset/lib/python3.4/importlib/_bootstrap.py:1161: ExtDeprecationWarning: Importing flask.ext.sqlalchemy is deprecated, use flask_sqlalchemy instead.
  spec.loader.load_module(spec.name)
/usr/hdp/2.6.0.3-8/superset/lib/python3.4/importlib/_bootstrap.py:1161: ExtDeprecationWarning: Importing flask.ext.script is deprecated, use flask_script instead.
  spec.loader.load_module(spec.name)
Loaded your LOCAL configuration
Traceback (most recent call last):
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 337, in dbapi
    from sqlite3 import dbapi2 as sqlite  # try 2.5+ stdlib name.
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/sqlite3/__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: No module named '_sqlite3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/hdp/current/druid-superset/bin/superset", line 84, in <module>
    from superset.cli import manager
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/superset/__init__.py", line 36, in <module>
    utils.pessimistic_connection_handling(db.engine.pool)
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/flask_sqlalchemy/__init__.py", line 816, in engine
    return self.get_engine(self.get_app())
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/flask_sqlalchemy/__init__.py", line 833, in get_engine
    return connector.get_engine()
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/flask_sqlalchemy/__init__.py", line 499, in get_engine
    self._engine = rv = sqlalchemy.create_engine(info, **options)
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/sqlalchemy/engine/__init__.py", line 387, in create_engine
    return strategy.create(*args, **kwargs)
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/sqlalchemy/engine/strategies.py", line 80, in create
    dbapi = dialect_cls.dbapi(**dbapi_args)
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 339, in dbapi
    raise e
  File "/usr/hdp/2.6.0.3-8/superset/lib/python3.4/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 334, in dbapi
    from pysqlite2 import dbapi2 as sqlite
ImportError: No module named 'pysqlite2'

my os is centos 7.X

Please help me check this problem, and give me some suggestions!

Thx

[#2]

Team, i am tracking this error separately, see below. If anybody finds the solution please mark me. Thanks in Advance!!!

https://www.linkedin.com/pulse/hdp-26-issue-druid-superset-start-fail-importerror-module-kumar

[#3]

It worked for me to change the database to mysql.

1.you need to create user and databases named 'superset' using mysql shell

GRANT ALL PRIVILEGES ON *.* TO 'superset'@'%' IDENTIFIED BY '*****' WITH GRANT OPTION;

create database superset;

2. change the config for Druid , set Mysql's hostname port user password .etc

3. start the druid, and it worked!

[#4]

@Mukesh Kumar

Not sure why the article is protected, but it essentially says to switch the superset database to mysql/postgreSQL instead of sqlite for a temporary work around. You can adjust this in Ambari by going to the druid service and modifying the config for the "Superset Database name".

[#5]

No luck, please see above my reply to Deepesh! Thanks!!!

[#6]

Thanks Devin, I have tried changing database for superset but having similar issue:-

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 169, in <module>
    Superset().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 108, in start
    self.configure(env, upgrade_type=upgrade_type)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 117, in locking_configure
    original_configure(obj, *args, **kw)
  File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 84, in configure
    user=params.druid_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'source /etc/superset/conf/superset-env.sh ; /usr/hdp/current/druid-superset/bin/superset db upgrade' returned 1. Traceback (most recent call last):
  File "/usr/hdp/current/druid-superset/bin/superset", line 10, in <module>
    import gunicorn.app.base
ImportError: No module named gunicorn.app.base

[#7 · Accepted solution]

this is a copy past of response

Answer by Nishant Bangarwa Apr 14 at 09:30 PM

Thanks for reporting this. seems like an issue with the packaging on Centos7. We are working on fixing it.

In the meantime, can you try with changing the superset database type to 'mysql' or 'postgresql' ?

I verified that it works with mysql.

[#8]

Thx, it works when I change the database type to mysql.