Support Questions

Find answers, ask questions, and share your expertise

Centralized DB for multiple hue servers

Hello,

I wish to load balance multiple hue servers under a load balancer. Each hue server keeps the record in its own prostgre database. For running a multiple hue servers, I need to keep these records in a centralized DB. Is it possible to maintain the entire hue servers in my cluster in a centralized db?

Thanks in advance.

1 ACCEPTED SOLUTION

Scheme should be created automatically, Follow below steps:

To set up Hue to use a MySQL database:

  1. Create a new user in MySQL, and grant privileges to it to manage the database using the MySQL database admin utility:
    # mysql -u root -p<
    CREATE USER $HUEUSER IDENTIFIED BY '$HUEPASSWORD';
    GRANT ALL PRIVILEGES on *.* to ‘$HUEUSER’@’localhost’ WITH GRANT OPTION;
    GRANT ALL on $HUEUSER.* to ‘$HUEUSER’@’localhost’ IDENTIFIED BY $HUEPASSWORD;
    FLUSH PRIVILEGES;

    where $HUEUSER is the Hue user name and $HUEPASSWORD is the Hue user password.

  2. Create the MySQL database for Hue:

    # mysql -u root -p

    CREATE DATABASE $DBNAME;

  3. Open the /etc/hue/conf/hue.ini file and edit the [[database]] section (modify for your MySQL setup).
    [[database]] 
    engine=mysql
    host=$DATABASEIPADDRESSORHOSTNAME
    port=$PORT
    user=$HUEUSER
    password=$HUEPASSWORD
    name=$DBNAME
  4. Synchronize Hue with the external database to create the schema and load the data.

    /usr/lib/hue/build/env/bin/hue syncdb --noinput

  5. Start Hue.

    /etc/init.d/hue start

View solution in original post

11 REPLIES 11

That should work I think. Just connect all your Hue instances to your central DB. It won't work with the embedded SQLite, but you have already switched to Postgres.

Yes, It works, I have made single centralized external ORACLE DB for my 3 instances of hue.

@Radhakrishnan Rk Yes, you should be able to point multiple Hue Servers to same database unless it is default SQLite i.e. MySQL/Postgres/Oracle. And then configure your loadbalancer i.e. haproxy/httpd/nginx etc. for the same.

Note: I am assuming multiple Hue Servers are for load balancing purpose i.e. One Hue server supports max 50 concurrent transactions. And in this case, ensure that you have same config setup i.e. ldap etc. for all Hue Server instances.

Yes, perfect Radha

Do I need to create a table and schema if I prefer to use mysql as my db engine? Will schema be created by hue itself when I restart the hue service?

Scheme should be created automatically, Follow below steps:

To set up Hue to use a MySQL database:

  1. Create a new user in MySQL, and grant privileges to it to manage the database using the MySQL database admin utility:
    # mysql -u root -p<
    CREATE USER $HUEUSER IDENTIFIED BY '$HUEPASSWORD';
    GRANT ALL PRIVILEGES on *.* to ‘$HUEUSER’@’localhost’ WITH GRANT OPTION;
    GRANT ALL on $HUEUSER.* to ‘$HUEUSER’@’localhost’ IDENTIFIED BY $HUEPASSWORD;
    FLUSH PRIVILEGES;

    where $HUEUSER is the Hue user name and $HUEPASSWORD is the Hue user password.

  2. Create the MySQL database for Hue:

    # mysql -u root -p

    CREATE DATABASE $DBNAME;

  3. Open the /etc/hue/conf/hue.ini file and edit the [[database]] section (modify for your MySQL setup).
    [[database]] 
    engine=mysql
    host=$DATABASEIPADDRESSORHOSTNAME
    port=$PORT
    user=$HUEUSER
    password=$HUEPASSWORD
    name=$DBNAME
  4. Synchronize Hue with the external database to create the schema and load the data.

    /usr/lib/hue/build/env/bin/hue syncdb --noinput

  5. Start Hue.

    /etc/init.d/hue start

Thank you so much, I try with this configs.

@Radhakrishnan Rk : I have tried below steps and it worked, this might help more:

1. If Hue is not installed, pick a node and with root permissions run the following command yum install hue*

2. Make appropriate changes to /etc/hue/conf/hue.ini from Kerberos and LDAP perspectives in hue.ini

3. Pick a node where MySQL need to be installed, with root permissions, run the following command: yum install MySQL

4. MySQL service will be stopped, by default.

5. Edit /etc/my.cnf and change the max_allowed_packet setting, set to the following: max_allowed_packet = 512M

6. Stop the Hue instances, if any.

7. Copy hue_db_dump.sql to a temporary location on the node where MySQL is installed

8. /etc/init.d/MySQL start

9. On the node where MySQL instance is installed, as root user, execute the following commands at the Linux prompt mysql -e "drop database hue; create database hue; use hue; source /tmp/hue_db_dump.sql;" mysql -e "GRANT ALL ON *.* to hue@'<node_with_hue_instance>' IDENTIFIED BY 'hue';" mysql -e "DELETE FROM hue.django_content_type; ALTER TABLE hue.beeswax_session ALTER proprietary SET DEFAULT 0;"

10. On the node where Hue is installed take a backup of hue.ini cp /etc/hue/conf/hue.ini /etc/hue/conf/hue.ini.bkup 1

1. On all the Hue instances edit /etc/hue/conf/hue.ini, to point the database to MySQL

12. Start all Hue instances, login to test.

New Contributor

We have tried the same thing on HDP clusters but it's not working.

 

Below is what we did.

Install Hue2.3  instances on two servers.

Installed Mysql on one server and created database and required user as per the instructions.

Modified both hue.ini files to point the same database.

Ran syncb commands on Both hue instances.

One hue is working fine, the other hue is throwing below error while running any queries on beeswax.

except beeswax remaining(Hcatlog,Hdfs browser are working fine.)

 

Error occurred executing hive query: Field 'session_id' doesn't have a default value

 

Please advise.

Community Manager

Hi @Rajuambala as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.



Regards,

Vidya Sargur,
Community Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

Thank you for all your responses. It works fine. But I got some error while restarting hue service in all hosts.

[centos@ip-172-31-56-224 ~]$ sudo -i [root@ip-172-31-56-224 ~]# /etc/init.d/hue restart

Shutting down hue: [ OK ] Traceback (most recent call last):

File "/usr/lib/hue/build/env/bin//hue", line 9, in <module> load_entry_point('desktop==2.6.1', 'console_scripts', 'hue')()

File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line 41, in entry from desktop import settings, appmanager File "/usr/lib/hue/desktop/core/src/desktop/settings.py", line 52, in <module> desktop.log.basic_logging(os.environ[ENV_HUE_PROCESS_NAME])

File "/usr/lib/hue/desktop/core/src/desktop/log/__init__.py", line 125, in basic_logging logging.config.fileConfig(log_conf) File "/usr/lib64/python2.6/logging/config.py", line 84, in fileConfig handlers = _install_handlers(cp, formatters)

File "/usr/lib64/python2.6/logging/config.py", line 162, in _install_handlers h = klass(*args)

File "/usr/lib64/python2.6/logging/handlers.py", line 112, in __init__ BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)

File "/usr/lib64/python2.6/logging/handlers.py", line 64, in __init__ logging.FileHandler.__init__(self, filename, mode, encoding, delay)

File "/usr/lib64/python2.6/logging/__init__.py", line 835, in __init__ StreamHandler.__init__(self, self._open())

File "/usr/lib64/python2.6/logging/__init__.py", line 854, in _open stream = open(self.baseFilename, self.mode)

IOError: [Errno 13] Permission denied: '/usr/lib/hue/logs/access.log'

Starting hue: [ OK ]

[root@ip-172-31-56-224 ~]# /etc/init.d/hue status

supervisor (pid 12586) is running...

[root@ip-172-31-56-224 ~]#

Will it impact my environment?

Thanks and Regards,

Radhakrishnan.R