Thanks @Akhil S Naik for reply. I think its not database inconsistency issue. But i have tried with your suggestion. Its not working. Even i cant see individual service stop drop-down at far right corner. Its just showing busy. ... View more

Even after changing the database to 'mysql' or 'postgresql' have not start Druid Superset service successfully and error message is below:- Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 169, in <module> Superset().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 108, in start self.configure(env, upgrade_type=upgrade_type) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 117, in locking_configure original_configure(obj, *args, **kw) File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 84, in configure user=params.druid_user) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'source /etc/superset/conf/superset-env.sh ; /usr/hdp/current/druid-superset/bin/superset db upgrade' returned 1. /usr/hdp/current/druid-superset/lib/python3.4/importlib/_bootstrap.py:1161: ExtDeprecationWarning: Importing flask.ext.sqlalchemy is deprecated, use flask_sqlalchemy instead. spec.loader.load_module(spec.name) /usr/hdp/current/druid-superset/lib/python3.4/importlib/_bootstrap.py:1161: ExtDeprecationWarning: Importing flask.ext.script is deprecated, use flask_script instead. spec.loader.load_module(spec.name) Loaded your LOCAL configuration Traceback (most recent call last): File "/usr/hdp/current/druid-superset/bin/superset", line 84, in <module> from superset.cli import manager File "/usr/hdp/current/druid-superset/lib/python3.4/site-packages/superset/__init__.py", line 36, in <module> utils.pessimistic_connection_handling(db.engine.pool) File "/usr/hdp/current/druid-superset/lib/python3.4/site-packages/flask_sqlalchemy/__init__.py", line 816, in engine return self.get_engine(self.get_app()) File "/usr/hdp/current/druid-superset/lib/python3.4/site-packages/flask_sqlalchemy/__init__.py", line 833, in get_engine return connector.get_engine() File "/usr/hdp/current/druid-superset/lib/python3.4/site-packages/flask_sqlalchemy/__init__.py", line 493, in get_engine info = make_url(uri) File "/usr/hdp/current/druid-superset/lib/python3.4/site-packages/sqlalchemy/engine/url.py", line 194, in make_url return _parse_rfc1738_args(name_or_url) File "/usr/hdp/current/druid-superset/lib/python3.4/site-packages/sqlalchemy/engine/url.py", line 240, in _parse_rfc1738_args return URL(name, **components) File "/usr/hdp/current/druid-superset/lib/python3.4/site-packages/sqlalchemy/engine/url.py", line 60, in __init__ self.port = int(port) ValueError: invalid literal for int() with base 10: '' ... View more

Thanks Devin, I have tried changing database for superset but having similar issue:- Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 169, in <module> Superset().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute method(env) File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 108, in start self.configure(env, upgrade_type=upgrade_type) File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 117, in locking_configure original_configure(obj, *args, **kw) File "/var/lib/ambari-agent/cache/common-services/DRUID/0.9.2/package/scripts/superset.py", line 84, in configure user=params.druid_user) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err) resource_management.core.exceptions.ExecutionFailed: Execution of 'source /etc/superset/conf/superset-env.sh ; /usr/hdp/current/druid-superset/bin/superset db upgrade' returned 1. Traceback (most recent call last): File "/usr/hdp/current/druid-superset/bin/superset", line 10, in <module> import gunicorn.app.base ImportError: No module named gunicorn.app.base ... View more

No luck, please see above my reply to Deepesh! Thanks!!! ... View more

Team, i am tracking this error separately, see below. If anybody finds the solution please mark me. Thanks in Advance!!! https://www.linkedin.com/pulse/hdp-26-issue-druid-superset-start-fail-importerror-module-kumar ... View more

Me too unable to open given link for solution!!!!! https://community.hortonworks.com/questions/96619/druid-superset-wont-start-on-hdp-26.html Access Denied We're sorry, but you do not have permission to do the activity you attempted. If you believe this to be in error, please contact the site administrator(s). ... View more

Hi @Himanshu Rawat There are two approach for this problem. 1. Create partition wise separate files using unix or any other tool and load them on individually in static partitions like below:- ALTER TABLE Unm_Parti ADD PARTITION (Department='A') location '/user/mukesh/HIVE/HiveTrailFolder/A'; ALTER TABLE Unm_Parti ADD PARTITION (Department='B') location '/user/mukesh/HIVE/HiveTrailFolder/B'; ALTER TABLE Unm_Parti ADD PARTITION (Department='C') location '/user/mukesh/HIVE/HiveTrailFolder/C'; 2. Create external table and put file into external table HDFS location, we can call it as staging table. Now create final partition table and load it using dynamic partition enable:- 1. set hive.exec.dynamic.partition=true This enable dynamic partitions, by default it is false. 2. set hive.exec.dynamic.partition.mode=nonstrict We are using the dynamic partition without a static partition (A table can be partitioned based on multiple columns in hive) in such case we have to enable the non strict mode. In strict mode we can use dynamic partition only with a Static Partition. Now use below statement to load data:- INSERT OVERWRITE TABLE Final_Table PARTITION(c2) SELECT c1, c4,c3,c2 FROM stage_table; ... View more

Yes, Because it is required to map schema, the last column in file is partitioned column. But if you are loading from another table then in select statement keep your partitioned column last. ... View more

You can create external table and map schema and move file to HDFS, CREATE EXTERNAL TABLE IF NOT EXISTS Cars( Name STRING, Miles_per_Gallon INT, Cylinders INT, Displacement INT, Horsepower INT, Weight_in_lbs INT, Acceleration DECIMAL, Year DATE, Origin CHAR(1)) COMMENT 'Data about cars from a public database' ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE location '/user/<username>/visdata'; hdfs dfs -copyFromLocal cars.csv /user/<username>/visdata Now create ORC table :- CREATE TABLE IF NOT EXISTS mycars( Name STRING, Miles_per_Gallon INT, Cylinders INT, Displacement INT, Horsepower INT, Weight_in_lbs INT, Acceleration DECIMAL, Year DATE, Origin CHAR(1)) COMMENT 'Data about cars from a public database' ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS ORC; Insert the data from the external table to the Hive ORC table. INSERT OVERWRITE TABLE mycars SELECT * FROM cars; ... View more

Hi @Arun A K, it has been observed that most of the time is consumed when we write data for downstream in Cassandra as single node is serving to Cassandra cluster. Now we are planning to increase create multiple Cassandra nodes inside the Hadoop cluster for fast writing. I'll keep you update on progress. ... View more

Thanks @Arun A K, i'll verify suggestions on my test case let you know progress if get. ... View more

Just received feedback from developers that using above approach there are able to utilize 61 virtual cores out of 64. But performance is still the bottleneck means file still taking same time. Anybody have idea whats wrong going on? ... View more