The limitation of display of 5000 tables is due to dbms.py script under Hue, below is the snippet of the script:
def get_tables(self, database='default', table_names='*'):
hql = "SHOW TABLES IN %s '%s'" % (database, table_names) # self.client.get_tables(database, table_names) is too slow
query = hql_query(hql)
handle = self.execute_and_wait(query, timeout_sec=15.0)
if handle:
result = self.fetch(handle, rows=5000)
self.close(handle)
return [name for table in result.rows() for name in table]
else:
return []
To increase the number of the tables displayed in Hue. Please do the following:
3. Edit /usr/lib/hue/apps/beeswax/src/beeswax/server/dbms.py to change the rows value to 8000.
if handle:
result = self.fetch(handle, rows=8000) --> This value needs to be changed.
self.close(handle)
return [name for table in result.rows() for name in table]
else:
return []
I believe there is a better way to do it from hue config than changing the python code:
> Navigate to HUE > Configuration > Hue Service Advanced Configuration Snippet (Safety Valve) for hue_safety_valve.ini > add
==
[beeswax]
max_catalog_sql_entries=15000
==
If you need to list 15000 entries at once instead of 5k which is by default.
> Save and restart affected
Note that, you can't expect optimal performance out of HUE UI to load table list since no is too high and this is not a hue limitation rather being put on purpose so that table list load can be faster.