Hue table browser limit - 5000

[#1]

Hi All,

Is there a way to list more than 5000 tables in a database. By default Hue shows first 5000 tables from a database, is there a configuration change supplied through snippet by which we can override this. Found a related article not sure if this can be applied in CDH 5.14 as well?

https://community.hortonworks.com/articles/75938/hue-does-not-list-the-tables-post-5000-in-number.ht...

Regards

[#2]

Hi can anyone confirm if the post from HDP support can be applied in CDH environment as well:

dbms.py is available under: /opt/cloudera/parcels/CDH/lib/hue/apps/beeswax/src/beeswax/server and has:

def get_indexes(self, db_name, table_name):
hql = 'SHOW FORMATTED INDEXES ON `%(table)s` IN `%(database)s`' % {'table': table_name, 'database': db_name}

query = hql_query(hql)
handle = self.execute_and_wait(query, timeout_sec=15.0)

if handle:
result = self.fetch(handle, rows=5000)
self.close(handle)

return result

def get_functions(self, prefix=None):
filter = '"%s.*"' % prefix if prefix else '".*"'
hql = 'SHOW FUNCTIONS %s' % filter

query = hql_query(hql)
handle = self.execute_and_wait(query, timeout_sec=15.0)

if handle:
result = self.fetch(handle, rows=5000)
self.close(handle)

return result

[#3]

Hi @Prav,

Unfortunately, there is no officially supported way to increase the number of tables loaded in Hue. However, we do currently have a feature request to improve on this behavior.

In the meantime, you can workaround this by:

• distribute the tables in multiple DBs (recommended)
• manually adjust the 'max_rows' limit in hive_server2_lib.py as shown in below. However, keep these implications in mind before you do that:

1. The more the limit is increased, the more it will impact the performance of Hue.
2. If something goes wrong with Hue, this change would potentially make troubleshooting difficult.
3. The next time CDH is upgraded (even to a maintenance release), a new copy of hive_server2_lib.py will be installed, and change will have to be made again.
4. Before making the change, hive_server2_lib.py should be backed up.

 Here is the sample code reference from /opt/cloudera/parcels/CDH/lib/hue/apps/beeswax/src/beeswax/server/hive_server2_lib.py:

-----------------

def get_tables(self, database, table_names, table_types=None):
if not table_types:
table_types = self.DEFAULT_TABLE_TYPES
req = TGetTablesReq(schemaName=database, tableName=table_names, tableTypes=table_types)
res = self.call(self._client.GetTables, req)

results, schema = self.fetch_result(res.operationHandle, orientation=TFetchOrientation.FETCH_NEXT, max_rows=5000)
self.close_operation(res.operationHandle)

return HiveServerTRowSet(results.results, schema.schema).cols(('TABLE_NAME',))

-----------------

Hope this helps,

Li

Cloudera Employee