Centralized DB for multiple hue servers

[#1]

Hello,

I wish to load balance multiple hue servers under a load balancer. Each hue server keeps the record in its own prostgre database. For running a multiple hue servers, I need to keep these records in a centralized DB. Is it possible to maintain the entire hue servers in my cluster in a centralized db?

Thanks in advance.

[#2]

That should work I think. Just connect all your Hue instances to your central DB. It won't work with the embedded SQLite, but you have already switched to Postgres.

[#3]

Yes, It works, I have made single centralized external ORACLE DB for my 3 instances of hue.

[#4]

@Radhakrishnan Rk Yes, you should be able to point multiple Hue Servers to same database unless it is default SQLite i.e. MySQL/Postgres/Oracle. And then configure your loadbalancer i.e. haproxy/httpd/nginx etc. for the same.

Note: I am assuming multiple Hue Servers are for load balancing purpose i.e. One Hue server supports max 50 concurrent transactions. And in this case, ensure that you have same config setup i.e. ldap etc. for all Hue Server instances.

[#5]

Yes, perfect Radha

[#6]

Do I need to create a table and schema if I prefer to use mysql as my db engine? Will schema be created by hue itself when I restart the hue service?

[#7 · Accepted solution]

Scheme should be created automatically, Follow below steps:

To set up Hue to use a MySQL database:

1. Create a new user in MySQL, and grant privileges to it to manage the database using the MySQL database admin utility:

   # mysql -u root -p<
   CREATE USER $HUEUSER IDENTIFIED BY '$HUEPASSWORD';
   GRANT ALL PRIVILEGES on *.* to ‘$HUEUSER’@’localhost’ WITH GRANT OPTION;
   GRANT ALL on $HUEUSER.* to ‘$HUEUSER’@’localhost’ IDENTIFIED BY $HUEPASSWORD;
   FLUSH PRIVILEGES;

   where $HUEUSER is the Hue user name and $HUEPASSWORD is the Hue user password.

2. Create the MySQL database for Hue:

   # mysql -u root -p

   CREATE DATABASE $DBNAME;

3. Open the /etc/hue/conf/hue.ini file and edit the [[database]] section (modify for your MySQL setup).

   [[database]] 
   engine=mysql
   host=$DATABASEIPADDRESSORHOSTNAME
   port=$PORT
   user=$HUEUSER
   password=$HUEPASSWORD
   name=$DBNAME

4. Synchronize Hue with the external database to create the schema and load the data.

   /usr/lib/hue/build/env/bin/hue syncdb --noinput

5. Start Hue.

   /etc/init.d/hue start

[#8]

Thank you so much, I try with this configs.

[#9]

@Radhakrishnan Rk : I have tried below steps and it worked, this might help more:

1. If Hue is not installed, pick a node and with root permissions run the following command yum install hue*

2. Make appropriate changes to /etc/hue/conf/hue.ini from Kerberos and LDAP perspectives in hue.ini

3. Pick a node where MySQL need to be installed, with root permissions, run the following command: yum install MySQL

4. MySQL service will be stopped, by default.

5. Edit /etc/my.cnf and change the max_allowed_packet setting, set to the following: max_allowed_packet = 512M

6. Stop the Hue instances, if any.

7. Copy hue_db_dump.sql to a temporary location on the node where MySQL is installed

8. /etc/init.d/MySQL start

9. On the node where MySQL instance is installed, as root user, execute the following commands at the Linux prompt mysql -e "drop database hue; create database hue; use hue; source /tmp/hue_db_dump.sql;" mysql -e "GRANT ALL ON *.* to hue@'<node_with_hue_instance>' IDENTIFIED BY 'hue';" mysql -e "DELETE FROM hue.django_content_type; ALTER TABLE hue.beeswax_session ALTER proprietary SET DEFAULT 0;"

10. On the node where Hue is installed take a backup of hue.ini cp /etc/hue/conf/hue.ini /etc/hue/conf/hue.ini.bkup 1

1. On all the Hue instances edit /etc/hue/conf/hue.ini, to point the database to MySQL

12. Start all Hue instances, login to test.

[#10]

We have tried the same thing on HDP clusters but it's not working.

Below is what we did.

Install Hue2.3  instances on two servers.

Installed Mysql on one server and created database and required user as per the instructions.

Modified both hue.ini files to point the same database.

Ran syncb commands on Both hue instances.

One hue is working fine, the other hue is throwing below error while running any queries on beeswax.

except beeswax remaining(Hcatlog,Hdfs browser are working fine.)

Error occurred executing hive query: Field 'session_id' doesn't have a default value

Please advise.