Support Questions

Find answers, ask questions, and share your expertise

Ambari with mysql 5.6

avatar
Super Collaborator

I had a tough time with Ambari 1.7 pointing to Mysql 5.6 on remote host. AS is using all the available ports on the source to make a connection to remote host to port 3306. At one time i saw 28000 ports being used by ambari to connect. this is killing us because it even used 50070 at one time, and i had tough time to figure it out. Finally i stopped ambari-server and all is well. Did anyone face this issue? if so any idea on fix? apart from downgrading Mysql or upgrade ambari to 2.1.2?

1 ACCEPTED SOLUTION

avatar
Super Collaborator

@bsaini i think i figured out the issue...it is a bug when we upgrade ambari database from Mysql 5.1 to later...as of now only option is to upgrade ambari to 2.2 or rollback the changes. thanks for taking time in answering my question...appreciate it!!!

https://na9.salesforce.com/articles/en_US/Issue/Ambari-log-reports-You-have-an-error-in-your-SQL-syn...

View solution in original post

3 REPLIES 3

avatar

@Raja Sekhar Chintalapati - Sounds like the OS level TCP IP params are not tuned correctly.

What kind of OS are you using? For e.g. on Centos and RHEL, you can use these command to check the values -

sysctl net.ipv4.tcp_keepalive_time

sysctl net.ipv4.tcp_keepalive_probes

sysctl net.ipv4.tcp_keepalive_intvl 

There are other params that can be tuned as well but these 3 are kind of critical. For e.g. if the tcp_keepalive_time is set to a high value then the OS holds onto that port even after the transaction is done (SQL is processed and results are obtained).. This is not Ambari specific, as you can see.

If this was Ambari issue, there should be a much wider impact.

I recommend checking the TCP WAITTIME and other params on both, the source and destination servers.

Here are the recommended values -

net.ipv4.tcp_keepalive_time = 600

net.ipv4.tcp_keepalive_probes = 5

net.ipv4.tcp_keepalive_intvl = 15

avatar

Note: Question moved to the Operations track.

avatar
Super Collaborator

@bsaini i think i figured out the issue...it is a bug when we upgrade ambari database from Mysql 5.1 to later...as of now only option is to upgrade ambari to 2.2 or rollback the changes. thanks for taking time in answering my question...appreciate it!!!

https://na9.salesforce.com/articles/en_US/Issue/Ambari-log-reports-You-have-an-error-in-your-SQL-syn...