I recently went through the exercise of adding an additional JBoss application server to a production CentOS 6.4 server. The two applications were to be hosted on the same machine using virtual name servers to distinguish requests. I have covered virtual name servers before in my post on install Trac on CentOS 6. Multiple instances of JBoss can be made to play nice on the same servers by shifting the ports via the switch ‑Djboss.service.binding.set=ports‑01.

So, with everything configured we should be done, right? Not so fast. Unfortunately I thought I had everything done but I got a 503 error, Service Temporarily Unavailable, when accessing the server via Apache. Directly accessing the JBoss server worked, so the problem had to be with Apache accessing the AJP connectors.

The connectors were configured thusly:

    ProxyPassReverseCookiePath /xyzzy /

    # Reverse proxy everything under /xyzzy

    ProxyPass /xyzzy/ ajp://prodas01:8109/xyzzy/

    ProxyPassReverse /xyzzy/ ajp://prodas01:8109/xyzzy/

Note that we are using port 8109 instead of 8009 since all the JBoss ports have been shifted by 100. Checking the Apache error log I saw the following:

[Wed Jun 05 13:11:55 2013] [error] (13)Permission denied: proxy: AJP: attempt to connect to 10.0.0.2:8109 (prodas01) failed

The problem did not appear to be firewall related since no firewall changes were necessary to get Apache talking to the first JBoss server on port 8009. Further probing revealed the following entry in the audit.log:

type=AVC msg=audit(1370452315.660:2053): avc: denied { name_connect } for pid=27314 comm="httpd" dest=8109 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=system_u:object_r:port_t:s0 tclass=tcp_socket

So the problem turned out to be SELinux. Apparently SELinux puts restrictions on what ports the httpd server can contact. The following shows the configuration out of the box:

[root@prodas01 /] semanage port -l | grep -w http_port_t

http_port_t                    tcp      80, 443, 488, 8008, 8009, 8443

So port 8009 is allowed, but port 8109 is not. We need to add it to the list:

[root@prodas01 /] semanage port -a -t http_port_t -p tcp 8109

[root@prodas01 /] semanage port -l | grep http_port_t

http_port_t                    tcp      8109, 80, 443, 488, 8008, 8009, 8443

Hat tip to krow oak for this helpful forum posting.