Reply
Contributor
Posts: 40
Registered: ‎07-06-2018

Hue table browser limit - 5000

Hi All,

 

 

Is there a way to list more than 5000 tables in a database. By default Hue shows first 5000 tables from a database, is there a configuration change supplied through snippet by which we can override this. Found a related article not sure if this can be applied in CDH 5.14 as well?

 

https://community.hortonworks.com/articles/75938/hue-does-not-list-the-tables-post-5000-in-number.ht...

 

Regards

Contributor
Posts: 40
Registered: ‎07-06-2018

Re: Hue table browser limit - 5000

Hi can anyone confirm if the post from HDP support can be applied in CDH environment as well:

 

dbms.py is available under: /opt/cloudera/parcels/CDH/lib/hue/apps/beeswax/src/beeswax/server and has:

 


def get_indexes(self, db_name, table_name):
hql = 'SHOW FORMATTED INDEXES ON `%(table)s` IN `%(database)s`' % {'table': table_name, 'database': db_name}

query = hql_query(hql)
handle = self.execute_and_wait(query, timeout_sec=15.0)

if handle:
result = self.fetch(handle, rows=5000)
self.close(handle)

return result

 


def get_functions(self, prefix=None):
filter = '"%s.*"' % prefix if prefix else '".*"'
hql = 'SHOW FUNCTIONS %s' % filter

query = hql_query(hql)
handle = self.execute_and_wait(query, timeout_sec=15.0)

if handle:
result = self.fetch(handle, rows=5000)
self.close(handle)

return result

Posts: 1,004
Kudos: 31
Solutions: 20
Registered: ‎05-27-2014

Re: Hue table browser limit - 5000

[ Edited ]

Hi @Prav,

 

Unfortunately, there is no officially supported way to increase the number of tables loaded in Hue. However, we do currently have a feature request to improve on this behavior.

 

In the meantime, you can workaround this by:

  • distribute the tables in multiple DBs (recommended)
  • manually adjust the 'max_rows' limit in hive_server2_lib.py as shown in below. However, keep these implications in mind before you do that:
  1. The more the limit is increased, the more it will impact the performance of Hue.
  2. If something goes wrong with Hue, this change would potentially make troubleshooting difficult.
  3. The next time CDH is upgraded (even to a maintenance release), a new copy of hive_server2_lib.py will be installed, and change will have to be made again.
  4. Before making the change, hive_server2_lib.py should be backed up.

 Here is the sample code reference from /opt/cloudera/parcels/CDH/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py:

-----------------

def get_tables(self, database, table_names, table_types=None):
if not table_types:
table_types = self.DEFAULT_TABLE_TYPES
req = TGetTablesReq(schemaName=database, tableName=table_names, tableTypes=table_types)
res = self.call(self._client.GetTables, req)

results, schema = self.fetch_result(res.operationHandle, orientation=TFetchOrientation.FETCH_NEXT, max_rows=5000)
self.close_operation(res.operationHandle)

return HiveServerTRowSet(results.results, schema.schema).cols(('TABLE_NAME',))

-----------------

Hope this helps,

Li

Cloudera Employee

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum