Issue with iptables and SCM Agent

[#1]

Alright, here's the deal.

I have a 15 machine cluster running CDH4.4 using CM 4.6.3 on CentOS 6.2.

2 "master" servers running namenode and such (nn01 and nn02)

12 "slave" servers running datanode and such (dn01 to dn12)

1 manager machine running CM. (dmg01)

Everything has been running flawlessly for the past couple of weeks.

Now we need to secure the machines using the firewall.  The 2 masters and 12 slaves have this config.  The manager machine does not run the firewall.

Here's my iptables file (replaced the actual IPs by **)

*filter
:INPUT ACCEPT [3989:259610]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [47999:9127516]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "***my machine***"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "nn01"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "nn02"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn01"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn02"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn03"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn04"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn05"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn06"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn07"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn08"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn09"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn10"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn11"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dn12"
-A INPUT -s **/32 -m conntrack --ctstate NEW -j ACCEPT  -m comment --comment "dmg01"
-A INPUT -j DROP
COMMIT

The second I start the IPTables service, all the hosts turn to RED in CM and the "last heartbeat" column is getting bigger and bigger.

The actual services are up and running fine but it looks like the agent is having difficulties connecting.

Again, the machine running the CM server is not firewalled, only the datanodes and namenodes and on those, all nodes, including the CM machine, are there.

I have no clue what's going on here, any idea?  Any specific config needed for CM Agent to run?

Thanks

[#2 · Accepted solution]

Found the problem, turns out that the agent uses the localhost connection to do its stuff.

I added a simple rule :

-A INPUT -s 127.0.0.1/32 -m conntrack --ctstate NEW -j ACCEPT

And it fixed the problem.