Member since
05-13-2016
34
Posts
5
Kudos Received
0
Solutions
09-17-2018
08:48 AM
Trying to get ambari repository file using following command. Getting HTTP 404 Not found error. wget -O /etc/apt/sources.list.d/ambari.list http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates//ambari.list This is as per Ambari installation documentation available here
... View more
Labels:
- Labels:
-
Apache Ambari
08-29-2018
12:17 PM
Exception went away after changing the method sqlalchemy_uri_decrypted() in file /usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/models.py as following. Not sure what side effects this will create, so far working okay. @property
def sqlalchemy_uri_decrypted(self):
conn = sqla.engine.url.make_url(self.sqlalchemy_uri)
conn.password = self.password
#return str(conn)
return conn
... View more
08-29-2018
08:52 AM
Using HDP-2.6.4.0 - Superset. Getting internal error when I select source - databases from menu bar. Error is as following. File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/security/decorators.py", line 26, in wraps
return f(self, *args, **kwargs)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/views.py", line 453, in list
widgets=widgets)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/baseviews.py", line 158, in render_template
return render_template(template, **dict(list(kwargs.items()) + list(self.extra_args.items())))
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/templating.py", line 134, in render_template
context, ctx.app)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask/templating.py", line 116, in _render
rv = template.render(context)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 2, in top-level template code
{% import 'appbuilder/general/lib.html' as lib %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/templates/appbuilder/base.html", line 1, in top-level template code
{% extends base_template %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/templates/superset/base.html", line 1, in top-level template code
{% extends "appbuilder/baselayout.html" %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/templates/appbuilder/baselayout.html", line 2, in top-level template code
{% import 'appbuilder/baselib.html' as baselib %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/templates/appbuilder/init.html", line 46, in top-level template code
{% block body %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/templates/appbuilder/baselayout.html", line 21, in block "body"
{% block content %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 13, in block "content"
{% block list_list scoped %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 15, in block "list_list"
{{ widgets.get('list')()|safe }}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/widgets.py", line 34, in __call__
return template.render(args)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/jinja2/_compat.py", line 37, in reraise
raise value.with_traceback(tb)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/list.html", line 2, in top-level template code
{% extends 'appbuilder/general/widgets/base_list.html' %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/base_list.html", line 23, in top-level template code
{% block begin_loop_values %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/list.html", line 45, in block "begin_loop_values"
{% for item in value_columns %}
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/jinja2/runtime.py", line 435, in __init__
self._after = self._safe_next()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/jinja2/runtime.py", line 455, in _safe_next
return next(self._iterator)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/models/base.py", line 87, in get_values
retdict[col] = self._get_attr_value(item, col)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_appbuilder/models/base.py", line 36, in _get_attr_value
if not hasattr(item, col):
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/models.py", line 712, in backend
url = make_url(self.sqlalchemy_uri_decrypted)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/models.py", line 859, in sqlalchemy_uri_decrypted
return str(conn)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/engine/url.py", line 90, in __str__
return self.__to_string__(hide_password=False)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/engine/url.py", line 72, in __to_string__
else _rfc_1738_quote(self.password))
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/engine/url.py", line 247, in _rfc_1738_quote
return re.sub(r'[:@/]', lambda m: "%%%X" % ord(m.group(0)), text)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/re.py", line 179, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: can't use a string pattern on a bytes-like object
... View more
Labels:
08-29-2018
08:40 AM
Modified as suggested. Service is starting now. thanks
... View more
08-29-2018
05:05 AM
I have added some print statement in the code where the exception is happening as i thought the issue was with python base64 encoding problem with padding. But looks like the the string passed to base64.b64decode() itself is malformed. See following log 2018-08-28 12:50:18,780:INFO:root:Syncing role definition
2018-08-28 12:50:18,800:INFO:root:Creating database reference
Loaded your LOCAL configuration
value: b'x393565452f683845463837555032666d5941577164673d3d'
Traceback (most recent call last):
File "/usr/hdp/current/superset/bin/superset", line 85, in <module>
manager.run()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_script/__init__.py", line 412, in run
result = self.handle(sys.argv[0], sys.argv[1:])
... View more
08-29-2018
04:51 AM
See postgres command output as following superset=# SHOW SERVER_ENCODING;
server_encoding
-----------------
UTF8
(1 row) ambari-agent log has following INFO 2018-08-28 12:50:19,237 PythonExecutor.py:130 - Command ['/usr/bin/python',
'/var/lib/ambari-agent/cache/common-services/SUPERSET/0.15.0/package/scripts/superset.py',
'START',
'/var/lib/ambari-agent/data/command-1688.json',
'/var/lib/ambari-agent/cache/common-services/SUPERSET/0.15.0/package',
'/var/lib/ambari-agent/data/structured-out-1688.json',
'INFO',
'/var/lib/ambari-agent/tmp',
'PROTOCOL_TLSv1',
''] failed with exitcode=1
... View more
08-28-2018
06:23 PM
Using HDP-2.6.4.0. Enabled apache superset service. Service start is failing with following error. I am using postgresql as database for storing superset metadata. 2018-08-28 10:05:16,587:INFO:root:Creating database reference
Loaded your LOCAL configuration
Traceback (most recent call last):
File "/usr/hdp/current/superset/bin/superset", line 85, in <module>
manager.run()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_script/__init__.py", line 412, in run
result = self.handle(sys.argv[0], sys.argv[1:])
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_script/__init__.py", line 383, in handle
res = handle(*args, **config)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_script/commands.py", line 216, in __call__
return self.run(*args, **kwargs)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/cli.py", line 27, in init
security.sync_role_definitions()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/security.py", line 122, in sync_role_definitions
get_or_create_main_db()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/security.py", line 96, in get_or_create_main_db
.filter_by(database_name='main')
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/query.py", line 2690, in first
ret = list(self[0:1])
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/query.py", line 2482, in __getitem__
return list(res)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 90, in instances
util.raise_from_cause(err)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 75, in instances
rows = [proc(row) for row in fetch]
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 75, in <listcomp>
rows = [proc(row) for row in fetch]
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 437, in _instance
loaded_instance, populate_existing, populators)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 498, in _populate_full
dict_[key] = getter(row)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/sql/type_api.py", line 1122, in process
return process_value(impl_processor(value), dialect)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy_utils/types/encrypted.py", line 266, in process_result_value
decrypted_value = self.engine.decrypt(value)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy_utils/types/encrypted.py", line 90, in decrypt
decrypted = base64.b64decode(value)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/base64.py", line 90, in b64decode
return binascii.a2b_base64(s)
binascii.Error: Incorrect padding Looks like a malformed string getting passed to base64.b64decode() function
... View more
- Tags:
- superset
Labels:
08-28-2018
06:16 PM
Using HDP-2.6.4.0. Enabled apache superset service. The service is not starting. Startup failes with error following error. Traceback (most recent call last):
File "/usr/hdp/current/superset/bin/superset", line 85, in <module>
manager.run()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_script/__init__.py", line 412, in run
result = self.handle(sys.argv[0], sys.argv[1:])
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_script/__init__.py", line 383, in handle
res = handle(*args, **config)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/flask_script/commands.py", line 216, in __call__
return self.run(*args, **kwargs)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/cli.py", line 27, in init
security.sync_role_definitions()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/security.py", line 122, in sync_role_definitions
get_or_create_main_db()
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/superset/security.py", line 96, in get_or_create_main_db
.filter_by(database_name='main')
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/query.py", line 2690, in first
ret = list(self[0:1])
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/query.py", line 2482, in __getitem__
return list(res)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 90, in instances
util.raise_from_cause(err)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 75, in instances
rows = [proc(row) for row in fetch]
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 75, in <listcomp>
rows = [proc(row) for row in fetch]
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 437, in _instance
loaded_instance, populate_existing, populators)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/orm/loading.py", line 498, in _populate_full
dict_[key] = getter(row)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy/sql/type_api.py", line 1122, in process
return process_value(impl_processor(value), dialect)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy_utils/types/encrypted.py", line 266, in process_result_value
decrypted_value = self.engine.decrypt(value)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/site-packages/sqlalchemy_utils/types/encrypted.py", line 90, in decrypt
decrypted = base64.b64decode(value)
File "/usr/hdp/2.6.4.0-91/superset/lib/python3.4/base64.py", line 90, in b64decode
return binascii.a2b_base64(s)
binascii.Error: Incorrect padding Looks like a malformed string is getting passed to base64.b64decode. Any pointers on how to workaround the issue. I am using postgresql database for superset meta data.
... View more
Labels:
05-17-2018
05:04 AM
Share ambari-server.out
... View more
05-16-2018
09:35 AM
@anarasimham, thanks for the info, Any reference to documentation stating this ?
... View more
05-15-2018
11:45 AM
I am using HDP 2.6.4 and Ambari 2.6.1.5. In Ambari HDFS summary page there is a metric called "Disk Usage (DFS Used)" which in case is showing 19GB. If I do a hdfs dfs -du -h / it is giving a total of 6GB. Shouldn't these two results be the same, or I am missing something here?
... View more
Labels:
- Labels:
-
Apache Ambari
06-29-2017
05:26 AM
I am running HDP 2.6.0 with zeppelin configured with Active Directory for authentication. Is there a way to pass on the currently logged-in zeppelin user to JDBC data sources such as a hive instance. (This hive instance is also configured with the same Active Directory instance via ranger)
... View more
Labels:
- Labels:
-
Apache Zeppelin
06-13-2017
01:35 PM
Users are back after following the steps suggested. Thanks for the quick response
... View more
06-13-2017
12:48 PM
Getting this exception. 13 Jun 2017 18:15:40,081 ERROR [ambari-client-thread-26] ReadHandler:102 - Caught a runtime exception executing a query
com.google.common.cache.CacheLoader$InvalidCacheLoadException: CacheLoader returned null for key 10.
at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2348)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2318)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
... View more
06-13-2017
12:34 PM
Just upgraded ambari from 2.4.x to 2.5.x. Also upgraded HDP from 2.5.0.x to 2.6.0.3. Ambari user logins are not working except for admin user. All are local users. On login as admin, in user management view no users are displayed, but when tried to create user which was existing earlier, getting error that "User already exists". What could be wrong. I have followed steps as in [1] [1] - https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.0.3/bk_ambari-upgrade/content/post_ambari_upgrade_tasks.html
... View more
Labels:
- Labels:
-
Apache Ambari
05-26-2017
09:04 AM
I also faced the same issue. Had to remove one note to make the the service. Is there a way to figure out the the notes that are corrupted. Log file doesn't tell anything in version 0.6.0.2.5
... View more
12-07-2016
12:27 PM
Using version 0.5.x. Had to modify as following. delete from x_group_users where added_by_id in (1,2);
delete from x_policy_item_user_perm where user_id in (select id from x_user where added_by_id in (1,2));
delete from x_user where added_by_id in (1,2);
delete from x_policy_item_group_perm where group_id in (select id from x_group where added_by_id in (1,2));
delete from x_group where added_by_id in (1,2);
delete from x_user_module_perm where added_by_id in (1,2);
delete from x_portal_user_role where added_by_id in (1,2);
delete from x_portal_user where added_by_id in (1,2);
... View more
11-03-2016
09:07 AM
I am using HDF-2.0.1.0-12 with HDP 2.4 (Hive 1.2.1.2.4. HDFS 2.7.1.2.4) . jdk1.8.0_71
... View more
11-03-2016
09:04 AM
Thanks for the response. Tried with ConvertAvroSchema and is working fine.
... View more
11-02-2016
12:04 PM
In JdbcCommon.java, see lines 318 - 322 // Did not find direct suitable type, need to be clarified!!!!
case DECIMAL:
case NUMERIC: builder.name(columnName).type().unionOf().nullBuilder().endNull().and().stringType().endUnion().noDefault();
break;
... View more
11-02-2016
11:56 AM
1 Kudo
I am trying to import data from a postgresql table to Hive using Apache Nifi using a simple flow QueryDatabaseTable -> PutHiveStreamig and getting following error. WARN [put-hive-streaming-0] org.apache.hive.hcatalog.data.JsonSerDe Error [org.codehaus.jackson.JsonParseException: Current token (VALUE_STRING) not numeric, can not use numeric value accessors Source table is having some columns of type Numeric which I think is not handled well in the Nifi code causing error while interpreting data types at Hive side. In JSON record, numeric values are appearing as strings with quotes like "33.750". Is there a work around for the problem ?
... View more
Labels:
- Labels:
-
Apache NiFi
10-28-2016
09:57 AM
2 Kudos
I am evaluating Apache Nifi for moving data to Hive instance in HDP. While moving the data to Hive, I am having a requirement to mask/transform some of the data attributes using a lookup table similar to what can be done in traditional ETL lookup transformations. How can I achieve the same in Nifi ?
... View more
Labels:
- Labels:
-
Apache NiFi