Member since
10-15-2014
126
Posts
2
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1558 | 10-29-2018 12:45 PM | |
3767 | 10-29-2018 11:42 AM | |
1577 | 04-05-2017 06:57 PM | |
4596 | 02-05-2016 05:13 PM | |
26734 | 10-31-2014 11:25 AM |
10-29-2018
12:45 PM
I had to customize Hue Server Advanced Configuration Snippet (Safety Valve) for hue_safety_valve_server.ini to add back the base functionality [notebook] [[interpreters]] [[[hive]]] name=Hive interface=hiveserver2 [[[impala]]] name=Impala interface=hiveserver2 [[[presto]]] name=Presto interface=jdbc options='{"url": "jdbc:presto://ta-pci-prod-presto-master01.tripactions.local:8889/ta", "driver": "com.facebook.presto.jdbc.PrestoDriver"}'
... View more
10-29-2018
11:42 AM
I had to customize Hue Server Advanced Configuration Snippet (Safety Valve) for hue_safety_valve_server.ini to add back the base functionality [notebook] [[interpreters]] [[[hive]]] name=Hive interface=hiveserver2 [[[impala]]] name=Impala interface=hiveserver2 [[[presto]]] name=Presto interface=jdbc options='{"url": "jdbc:presto://ta-pci-prod-presto-master01.tripactions.local:8889/ta", "driver": "com.facebook.presto.jdbc.PrestoDriver"}'
... View more
10-26-2018
03:19 PM
sure I should be able to see both Hive and HUE on this drop down see attached and yes it is activated
... View more
10-26-2018
01:25 PM
I added Impala as a service after the installation of CDH6 Impala via commandline on the box works but it doesn't work in HUE I followed the instructions per the documents and and from here https://community.cloudera.com/t5/Web-UI-Hue-Beeswax/Do-not-see-impala-app-in-hue-after-installation-of-impala/m-p/63267/highlight/true#M2560 According to @Romainr I need to add the following to the HUE.ini [[[impala]]]
name=Impala
interface=hiveserver2 Every time I add something to Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini and try to restart the service the service refuses to start am I not putting the condition in the right place or in the right format? It suppose to be a simple change from None to Impala and it should work
... View more
Labels:
09-18-2018
03:38 PM
On CMS 6 (playing around with it ) and I configured HUE but Hive is not available The only option is Hive Service Hue (Service-Wide) Hive am I missing something?
... View more
Labels:
- Labels:
-
Apache Hive
-
Cloudera Hue
07-18-2018
11:11 AM
Is there a Chart or way to see all the tables used by Impala for the last 6 months? I know that in CM the Queries tab shows only 30 days of prior queries is there a chart I can build out?
... View more
Labels:
- Labels:
-
Apache Impala
-
Cloudera Manager
02-06-2018
10:32 AM
We are on Impala Shell v2.8.0-cdh5.11.1 community edition SSL is enabled but no sentry Executed a command refresh schema.table 17:15:14 Query: refresh hbasestage.raw_transactions
17:15:14 Query submitted at: 2018-02-06 01:15:14 (Coordinator: http://hadoop4-private.wdc01.infra.ripple.com:25000)
03:21:40 Query progress can be monitored at: http://hadoop4-private.wdc01.infra.ripple.com:25000/query_plan?query_id=984250bf5d880d33:fddc42d100000000
03:21:40
03:21:40 Fetched 0 row(s) in 36385.13s There has to be a better way Table is normally populated by Hive, so refresh is required for impala to recognize new partitions Table has 1861 partitions - total 1.28 TB of data, each partition is no bigger than 3GB (partition is by date) Files are avro but that shouldn't impact it (should it?) Yarn does NOT manage resources
... View more
Labels:
- Labels:
-
Apache Impala
08-17-2017
09:20 AM
Another update left master alone to see if it can resolve Starting namespace manager (since 11hrs, 55mins, 3sec ago) unfortunately I cannot find any logs on what exactly its trying to do
... View more
08-16-2017
04:23 PM
add on https://issues.apache.org/jira/browse/HBASE-16488 and master is still starting Starting namespace manager (since 1hrs, 55mins, 57sec ago)
... View more
08-16-2017
03:54 PM
My primary master died due to scheduled maintenance and my backup master failed to kick in CM Agent has tried to start it up but could not initialize the namespace table After several manual efforts where I followed https://community.cloudera.com/t5/Storage-Random-Access-HDFS/HBase-Master-Failed-to-become-active-master/m-p/27186#M1225 I did the following only rmr /hbase/meta-region-server
rmr /hbase/rs
rmr /hbase/splitWAL
rmr /hbase/backup-masters
rmr /hbase/table-lock
rmr /hbase/flush-table-proc
rmr /hbase/region-in-transition
rmr /hbase/running
rmr /hbase/balancer
rmr /hbase/recovering-regions
rmr /hbase/draining
rmr /hbase/namespace
rmr /hbase/hbaseid
rmr /hbase/table I got the a master to come up after setting hbase.master.namespace.init.timeout to some absurd value I see the master registering dead region servers (though I cannot find where it pick them up, not in the WAL, Archive or data) and I see the master registering the following Starting namespace manager (since 1hrs, 20mins, 5sec ago) even though cloudera manager shows healthy list the catalog in hbase shell gives me the following error hbase(main):004:0> list
TABLE
ERROR: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing
at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2373)
at org.apache.hadoop.hbase.master.MasterRpcServices.getTableNames(MasterRpcServices.java:907)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55650)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2182)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:185)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:165) fsck /hbase -files -blocks shows healthy hbck shows zero inconsistencies I am on version Hadoop 2.6.0+cdh5.11.1+2400 HBase 1.2.0+cdh5.11.1+319 I did have a master colocated with a region server and was wondering if I ran into this https://issues.apache.org/jira/browse/HBASE-14861 and then this https://issues.apache.org/jira/browse/HBASE-14664 as the cause of the failed backup kicking in But I cannot determine why I am getting namespace manager would take so long to initialize
... View more
Labels:
- Labels:
-
Apache HBase
08-10-2017
11:39 AM
Recently upgraded Cloudera from 5.4 to 5.12 but running 5.11 components so Impala 2.8.0+cdh5.11.1+0 Also recently moved Hive Metastore database from postgres to MySQL Currently I have found that Impala is reacting slowly to executing queiries and executing them at all I have a 7 node cluster and only 5 real users using Impala In searching the logs I am getting the following error E0810 18:01:54.606694 25271 ShortCircuitCache.java:215] ShortCircuitCache(0xba3834e): failed to release short-circuit shared memory slot Slot(slotIdx=17, shm=DfsClientShm(5db0b20a1ec13fd9af815db2d22cd894)) by sending ReleaseShortCircuitAccessRequestProto to /var/run/hdfs-sockets/dn. Closing shared memory segment.
Java exception follows:
java.io.IOException: ERROR_INVALID: there is no shared memory segment registered with shmId 5db0b20a1ec13fd9af815db2d22cd894
at org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache$SlotReleaser.run(ShortCircuitCache.java:208)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) I also see the following as well cc:278
channel send status:
Sender timed out waiting for receiver fragment instance: 3d4e8ea794a486e7:a74d612000000010
... View more
Labels:
- Labels:
-
Apache Impala
08-10-2017
11:31 AM
I recently migrated the HUE database to MySQL as well as the hive metastore database and then upgraded Cloudera manager to 5.12 but using a 5.11 components (so HUE 3.9.0+cdh5.11.1+5073) My users have noticed that HUE takes a tremendous amount of time to render its pages Note cross post on google group hue users I extended the timeout settings and under an incognito window (to ensure nothing is cached) I was able to get this message Lock wait timeout exceeded Traceback: File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/handlers/base.py" in get_response 112. response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/transaction.py" in inner 371. return func(*args, **kwargs) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/django_axes-1.5.0-py2.7.egg/axes/decorators.py" in decorated_login 304. response = func(request, *args, **kwargs) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/desktop/core/src/desktop/auth/views.py" in dt_login 121. login(request, user) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/contrib/auth/__init__.py" in login 89. user_logged_in.send(sender=user.__class__, request=request, user=user) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/dispatch/dispatcher.py" in send 185. response = receiver(signal=self, sender=sender, **named) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/contrib/auth/models.py" in update_last_login 30. user.save(update_fields=['last_login']) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/base.py" in save 545. force_update=force_update, update_fields=update_fields) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/base.py" in save_base 573. updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/base.py" in _save_table 635. forced_update) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/base.py" in _do_update 679. return filtered._update(values) > 0 File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/query.py" in _update 510. return query.get_compiler(self.db).execute_sql(None) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/sql/compiler.py" in execute_sql 980. cursor = super(SQLUpdateCompiler, self).execute_sql(result_type) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/sql/compiler.py" in execute_sql 786. cursor.execute(sql, params) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/backends/util.py" in execute 53. return self.cursor.execute(sql, params) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/utils.py" in __exit__ 99. six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/backends/util.py" in execute 53. return self.cursor.execute(sql, params) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/backends/mysql/base.py" in execute 124. return self.cursor.execute(query, args) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/MySQL_python-1.2.5-py2.7-linux-x86_64.egg/MySQLdb/cursors.py" in execute 205. self.errorhandler(self, exc, value) File "/opt/cloudera/parcels/CDH-5.11.1-1.cdh5.11.1.p0.4/lib/hue/build/env/lib/python2.7/site-packages/MySQL_python-1.2.5-py2.7-linux-x86_64.egg/MySQLdb/connections.py" in defaulterrorhandler 36. raise errorclass, errorvalue Exception Type: OperationalError at /accounts/login/ Exception Value: (1205, 'Lock wait timeout exceeded; try restarting transaction') On the runscpserver log I get [04/Aug/2017 07:53:13 -0700] wsgiserver ERROR WSGI (<WorkerThread(CP WSGIServer Thread-33, started 140223207950080)>) error: [('SSL routines', 'SSL23_WRITE', 'ssl handshake failure')] Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1304, in run conn.communicate() File "/opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1214, in communicate req.simple_response("408 Request Timeout") File "/opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 618, in simple_response self.wfile.sendall("".join(buf)) File "/opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1139, in sendall return self._safe_call(False, super(SSL_fileobject, self).sendall, *args, **kwargs) File "/opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1091, in _safe_call return call(*args, **kwargs) File "/opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 753, in sendall bytes_sent = self.send(data) File "/opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1142, in send return self._safe_call(False, super(SSL_fileobject, self).send, *args, **kwargs) File "/opt/cloudera/parcels/CDH-5.4.11-1.cdh5.4.11.p0.5/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1121, in _safe_call raise FatalSSLAlert(*e.args) but I am using self signed certs Any thoughts or help would be appreciated
... View more
Labels:
- Labels:
-
Cloudera Hue
04-05-2017
06:57 PM
Fixed Romain posted on Hue user group that I had a conflict in python libraries and indeed I did I had Impyla and Airflow installed as part of the main lib/distro instead of a controlled "virtualized" environment This caused the conflict. Unfortunately both Impyla and Airflow could not be uninstalled with a simple pip uninstall command Had to manually compare directories from my staging environment and remove several of the directories that could be potentially conflicting These include: hive_metastore hive_thrift_py pyhive hive_serde impala PyHive TCLIService hive_service impyla thrift Once the directories where removed from /usr/local/lib/python2.7/dist-packages and restarted HUE, it was querying again. Many kudos to Romain
... View more
04-03-2017
06:39 AM
I had Hive and Impala working on HUE before Versions HUE 3.7 Clouderan Manager 5.4.11 I recently restarted HUE and HUE no longer sees Impala or hive Configuration files located in /run/cloudera-scm-agent/process/7850-hue-HUE_SERVER
Potential misconfiguration detected. Fix and restart Hue.
Hive Editor The application won't work without a running HiveServer2.
Impala Editor No available Impalad to send queries to. When I run a query I get cannot import name TGetLogReq Error log shows the following [03/Apr/2017 00:16:53 -0700] wsgiserver ERROR WSGI (<WorkerThread(CP WSGIServer Thread-19, started 140018618201856)>) error: [('SSL routines', 'SSL23_WRITE', 'ssl handshake failure')]
Traceback (most recent call last):
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1304, in run
conn.communicate()
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1214, in communicate
req.simple_response("408 Request Timeout")
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 618, in simple_response
self.wfile.sendall("".join(buf))
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1139, in sendall
return self._safe_call(False, super(SSL_fileobject, self).sendall, *args, **kwargs)
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1091, in _safe_call
return call(*args, **kwargs)
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 753, in sendall
bytes_sent = self.send(data)
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1142, in send
return self._safe_call(False, super(SSL_fileobject, self).send, *args, **kwargs)
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py", line 1121, in _safe_call
raise FatalSSLAlert(*e.args)
FatalSSLAlert: [('SSL routines', 'SSL23_WRITE', 'ssl handshake failure')]
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Impala
-
Cloudera Hue
03-30-2017
06:18 PM
Additional information if at all possible - I would like to only use the users in HUE and not build the users on the local server, or activate LDAP. Unless absolutely required
... View more
03-30-2017
05:03 PM
1 Kudo
do you mind expanding on the answer a bit I have the same problem is the group setting in CM or on hdfs or os files system?
... View more
03-30-2017
04:15 PM
So after 5 days of following https://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_sentry_service.html http://gethue.com/apache-sentry-made-easy-with-the-new-hue-security-app/#howto testing out with policy files and not policy files, hdfs ownership, and finally after followinghttp://www.yourtechchick.com/hadoop/no-databases-available-permissions-missing-error-hive-sentry/ I can create roles in HUE - I can see them, manage them and I see the Sentry logs updating something Admin users can see query and work with everything. And error messages are no longer comming in on HUE; small victories After applying roles to groups to limit access to certain db (the only thing I needed Sentry to do) The users belonging to the limited set cannot see any database and hive is throwing the following 2017-03-30 22:48:09,619 ERROR org.apache.hadoop.hive.ql.Driver: FAILED: SemanticException No valid privileges
Required privileges for this query: Server=server1->Db=*->Table=+->action=insert;Server=server1->Db=*->Table=+->action=select;
org.apache.hadoop.hive.ql.parse.SemanticException: No valid privileges
Required privileges for this query: Server=server1->Db=*->Table=+->action=insert;Server=server1->Db=*->Table=+->action=select;
at org.apache.sentry.binding.hive.HiveAuthzBindingHook.postAnalyze(HiveAuthzBindingHook.java:356)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:436)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:306)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1120)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1113)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:99)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:170)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:398)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:379)
at org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:245)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:487)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hive.ql.metadata.AuthorizationException: User vchetty does not have privileges for SWITCHDATABASE
at org.apache.sentry.binding.hive.authz.HiveAuthzBinding.authorize(HiveAuthzBinding.java:320)
at org.apache.sentry.binding.hive.HiveAuthzBindingHook.authorizeWithHiveBindings(HiveAuthzBindingHook.java:540)
at org.apache.sentry.binding.hive.HiveAuthzBindingHook.postAnalyze(HiveAuthzBindingHook.java:346)
... 20 more Technically my goal is to have sentry manage users in HUE on what they can see via Impala and BeesWax(hive) versions Cloudera Express 5.4.7 Hue™ 3.7.0 Sentry installed correctly and running on same server as HUE config LDAP and Kerberos not enabled On Hive Hive sentry is enabled sentry-site.xml has <property> <name>sentry.hive.testing.mode</name> <value>true</value></property> hive.server2.enable.impersonation, hive.server2.enable.doAs is off On HUE Hue sentry is enabled user_augmentor desktop.auth.backend.DefaultUserAugmentor desktop.auth.backend.AllowFirstUserDjangoBackend sentry-site.xml has <property> <name>sentry.hive.testing.mode</name> <value>true</value></property> beeline CLI shows the roles and I can show a specific role on which databases it can use the user is mapped to that specific role in HUE Haven't tried any settings on Impala, I hope it just adopts hive settings once enabled Any ideas on what I may be missing ?? LDAP and kerberos is not something I am willing to deal with at this time
... View more
Labels:
03-30-2017
01:24 PM
2 things to update 1 - all under /user/hive/warehouse set to hive (probably not the problem) 2 - user is case sensative - I had Hue in the portal, CM has HUE, but linux has hue in groups Correcting the username in the portal to lower case allowed me to create roles
... View more
03-29-2017
11:01 AM
My goal is to enable sentry on HUE only to protect some databases via hive and Impala But both Impala/hive CLI should not be impacted. CLI tools are isolated to an edge node and can only be directly accessed via ssh sessions by a select set of users LDAP and Kerberos not enabled on HUE (maybe later?) versions Cloudera Express 5.4.7 Hue™ 3.7.0 Sentry installed correctly and running on same server as HUE Hive and Impala do NOT have sentry enabled admin groups and allowed connecting users hive, impala, hue, hdfs and 1 custom service account all other settings are default HUE is configured through CM Sentry service checked - no snippet invoked Authentication Backend = desktop.auth.backend.AllowFirstUserDjangoBackend create_users_on_login checked no LDAP settings not kerberized Synced user and now have the Hue user and promoted account to be admin When I try to add policy I get the following Sentry Log 2017-03-29 17:37:25,030 ERROR org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: Access denied to Hue
org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to Hue
at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.list_sentry_roles_by_group(SentryPolicyStoreProcessor.java:450)
at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:953)
at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$list_sentry_roles_by_group.getResult(SentryPolicyService.java:938)
at sentry.org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at sentry.org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:48)
at sentry.org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
at sentry.org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
^@2017-03-29 17:38:56,565 ERROR org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor: Access denied to Hue
org.apache.sentry.provider.db.SentryAccessDeniedException: Access denied to Hue
at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.authorize(SentryPolicyStoreProcessor.java:205)
at org.apache.sentry.provider.db.service.thrift.SentryPolicyStoreProcessor.create_sentry_role(SentryPolicyStoreProcessor.java:215)
at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$create_sentry_role.getResult(SentryPolicyService.java:833)
at org.apache.sentry.provider.db.service.thrift.SentryPolicyService$Processor$create_sentry_role.getResult(SentryPolicyService.java:818)
at sentry.org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at sentry.org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.sentry.provider.db.service.thrift.SentryProcessorWrapper.process(SentryProcessorWrapper.java:48)
at sentry.org.apache.thrift.TMultiplexedProcessor.process(TMultiplexedProcessor.java:123)
at sentry.org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745) HUE Error log ERROR could not retrieve roles
^@[29/Mar/2017 10:38:56 -0700] hive ERROR could not create role
Traceback (most recent call last):
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/apps/security/src/security/api/hive.py", line 156, in create_role
api.create_sentry_role(role['name'])
File "/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/lib/hue/desktop/libs/libsentry/src/libsentry/api.py", line 49, in decorator
raise e
SentryException: Access denied to Hue
... View more
Labels:
06-21-2016
10:21 AM
Thanks very helpful but the ASCII function only translates the first character and the decode function (in hive) is in conflict with Impala interpretation of decode UDF to be similar to case (a poor decision IMHO since case is much more readable than decode) Anyway opinions aside How do I convert a large hex value into a large number?
select ascii(unhex('130F9C6D6A2972')) largenumber
, ascii(unhex('13')) firstpart both yeild the value of 19 when the first one suppose to yeild 5365189082491250
... View more
06-20-2016
01:32 PM
Running in Impala I try the following select unhex('4B') and I get a string result of K If I go to Hex to decimal converter to convert hex value '4B' I get the number 75 which is what I expect Is unhex not a hexadecimal to decimal converter? according to http://www.cloudera.com/documentation/enterprise/5-5-x/topics/impala_math_functions.html unhex(string a) Purpose: Returns a string of characters with ASCII values corresponding to pairs of hexadecimal digits in the argument. Return type: string I don't even know where to begin to start troubleshooting this
... View more
Labels:
- Labels:
-
Apache Impala
02-05-2016
05:13 PM
Per support in another thread The last time I saw this issue, it was due to something cached in the browser used to access Hue. Can you try using another browser to see if the problem happens there? If so, you might try logging out of Hue, clearing your cache, then logging into Hue to see if that helps.
... View more
02-05-2016
04:22 PM
typing in the command show databases in HUE under the Impala GUI yeilds the following error message AnalysisException: Syntax error in line 1: USE `` ^ Encountered: EMPTY IDENTIFIER Expected: IDENTIFIER CAUSED BY: Exception: Syntax error works in hive, works on Impala Command line
... View more
02-05-2016
04:18 PM
I restarted HUE - - Impala via HUE does not see databases then I went to Cloudera Manager disabled Impala restarted HUE enabled Impala restarted HUE - Impala via HUE does not see databases then I went to Cloudera Manager restarted Impala, Hive and HUE - Impala via HUE does not see databases
... View more
02-05-2016
02:56 PM
My Impala catalog server decided to not restart (different problem) so I removed the service from one node and started in another in the same cluster Now HUE does not show any databases and cannot execute any sql in Impala through HUE I have tried restarted HUE through Cloudera manager and even changed the configuration to not have impala restart and bring back impala via hue and restart I am able to run via command line in Impala-shell fine So I guess my question is in Cloudera Manager - How do I tell HUE or the server HUE is on there is a new catalog server? All I see is Impala Service Hue (Service-Wide) the choices of Impala or none Not very configurable
... View more
Labels:
02-05-2016
02:37 PM
My Impala catalog server decided to not restart (different problem) so I removed the service from one node and started in another Now HUE does not show any databases and cannot execute any sql in Impala through HUE I am able to run via command line in Impala-shell fine So I guess my question is in Cloudera Manager - How do I tell HUE or the server HUE is on there is a new catalog server? All I see is Impala Service Hue (Service-Wide) the choices of Impala or none Not very configurable
... View more
Labels:
12-04-2015
12:26 PM
did all of the above in the query editor also checked firewalls (no changes) restarted Impala and HUE have not restarted Hive / Beeswax yet
... View more
12-04-2015
11:24 AM
Weird anomoly on HUE I have a instance using chrome hitting the same HUE instance on the same cluster in 1 tab I can run Impala queries but newer tabs cannot run any queries Chrome Version 47.0.2526.73 (64-bit) CDH 5.3.3 Sentry is not enabled
... View more
Labels:
11-11-2015
11:58 AM
Thanks for the input I am suspecting that it may be the region settings I am using that prematurely splits regions contributing to the latency Originally I set the regions at default to 10 gb, shrank it to 4 gb did a compaction and flush, the first writes where trying to reorg the regions which is stemming from my first problem http://community.cloudera.com/t5/Storage-Random-Access-HDFS/hbase-closed-upon-write/m-p/34001#U34001 I then increased the regions to 4.9 gb seems to be fine now The reason I am staying under 5 gb, is so that I can transfer to S3 which has a file size limit
... View more