8 Replies Latest reply on Sep 19, 2013 8:57 AM by tim.skopnik

    MWG7 - HTTP Listener address with Proxy HA setup

    marcospenn

      Dear all,

       

      my  setup is based on 3 MWG7 proxies, configured in HA and working in explicit mode.

      Each proxy has 2 NIC configured:

      -ETH0 --> Production network where clients connects to and proxy gets the internet

      -ETH1 --> Management Network where GUI access is permitted and Central management ports are configured

       

      The IP segmentation between the 2 NIC works fine about GUI access and Central Management Cofiguration.

      And now the problem :-)

      Customer wants the proxies listening fro HTTP service only on production network (ETH0), and not on management network (ETH1)

       

      If i change the "listener address" (in Configuration/Appliances/Proxies tab) from the default 0.0.0.0:9090 to 10.x.y.z:9090 the clients stops working, retrieving the error "Connection rejected by proxy"

      The issue occur also after rebooting the MWG, and in addition, even if i point the client to the phisical MGW ip@ (instead of the VRRP one) the issue is the same.

       

      When i roll-back the listener setup to 0.0.0.0:9090, everything back to work normally.

       

      Any Idea if the VRRP if the root cause? If i test this setup in lab on a standalon machine (same production's release/hardware) the issue does NOT appear.

       

       

      Thank you for your help

       

      Marco

        • 1. Re: MWG7 - HTTP Listener address with Proxy HA setup
          asabban

          Hi Marco,

           

          I would expect your configuration to work. Sounds very strange to me. Unfortunately I do not have a Proxy HA test lab at hand right now, so I cannot test. When you tried to access the physical IP addresses, did you also attempt to do this on MWG itself? Like "ssh 10.x.y.z", login and do "telnet 10.x.y.z:9090"? Just to ensure your packets are not dropped somewhere on their way :-)

           

          If your only requirement is to prevent users from accessing port 9090 on eth1, you should also be able to use the "Network Protection" feature, which came with 7.1.5. It sets up iptables rules, so you can easily set up the box to drop packets which come in on eth1:9090, and leave the listener untouched.

           

          I know that this won´t resolve the original problem, but maybe it is suitable to address your requirements.

           

          Best,

          Andre

          • 2. Re: MWG7 - HTTP Listener address with Proxy HA setup
            marcospenn

            Hi Andre,

             

            thank you for the input! I have a flat-network lab environment, with no filtering devices in between proxies and clients...so nothing weird in communication path.

            This behaviour only happens in an HA layout...works like a charm on a standalone proxy.

            At state i've raised a support ticket to get in deep of this issue...and dealt with customer about iptables apply to narrow down proxy access ;-)

             

            Cheers,

            Marco

            • 3. Re: MWG7 - HTTP Listener address with Proxy HA setup

              Hi Marco

               

              I have seen simillar issue in my network. We have 2 MWG's configured as Proxy HA. When the listenrs are defined as 0.0.0.0:port then it works, but trying to limit it to one interface by setting IP address renders some problems.

              Try to add besides your 10.x.y.z:9090 a listener on 127.0.0.1:9090 this seems to help here. But I'm still testing this.

               

              @Andre

              I think you should try by McAfee to dig into this Proxy HA issues

               

               

              With Proxy HA and Network Protection you have to be aware that it in Network Protection you cannot define rule allowing VRRP traffic (https://community.mcafee.com/message/218630)

              • 4. Re: MWG7 - HTTP Listener address with Proxy HA setup
                asabban

                Hi,

                 

                the mwg-mon script which monitors if the MWG process is alive seems to utilize 127.0.0.1. If the configured proxy port is not accessble via 127.0.0.1 the mwg-mon script will tell the network driver that the node is offline.

                 

                You can check this by running

                 

                mwg-mon -v

                 

                It should give you something like:

                 

                current state: ok

                checking: port=9090

                ports looks good. no state change

                 

                If there is no listener on 127.0.0.1 you get:

                 

                current state: ok

                checking: port=9090

                state change: offline

                 

                Adding 127.0.0.1 should help.

                 

                Best,

                Andre

                • 5. Re: MWG7 - HTTP Listener address with Proxy HA setup
                  asabban

                  I have filed an FMR to change the hardcoded 127.0.0.1 to a dynamic lookup of the proxy listeners IP address.

                   

                  Best,

                  Andre

                  • 6. Re: MWG7 - HTTP Listener address with Proxy HA setup

                    In my case there are still some issues. It did worked yesterday, but fails (partially) today.

                    The setup is like this:

                     

                    subnet A:    pc1    pc2

                         |

                      gw

                       |

                    subnet B: proxyA  proxyB

                     

                    PC are in one subnet. Proxies are in second subnet. I am able to ping node IP and vrrp ip from both pc.

                    Proxies are set as Proxy HA with redirections (ports: 80, 8080, 2121) and proxies set on 127.0.0.1:port and 10.x.b.z:port for each redirection. ProxyA director priority was 95, ProxyB 90.

                     

                    Now when I try to access the proxies by node IP everything works fine. When I try to access vrrp ip on ports 8080, 2121, 80 from pc1 it works. But when I try to access it by vrrp IP then from pc2 8080 and 80 (http redirections) it fails. What is strange the 2121 (ftp redirection) works fine. The connection to 8080 is reset, see this wireshark caputure on pc2:

                    63    10.341062    10.x.a.57    10.36.32.19    TCP    jediserver > http-alt [SYN] Seq=0 Win=65535 Len=0 MSS=1460 SACK_PERM=1

                    64    10.342042    10.x.b.19    10.36.17.57    TCP    http-alt > jediserver [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

                     

                    On both proxies mwg-mon -v

                    current state: ok

                    checking: port=8080

                    checking: port=2121

                    checking: port=2122

                    checking: port=8086

                    checking: port=80

                    ports looks good. no state change

                     

                     

                     

                    The mfend command returns on proxy A:

                    mfend-lb -s

                         device: proxyA

                    statechange:

                             ip: 10.x.b.102

                            ip6: ::

                      protocols: 00000003

                            mac: 842b2b5b5a3b

                          state: NETWORK

                          stats: 0 0 263 0 0

                    statusvalid: 1

                           type: director

                     

                         device: __SELF__

                    statechange:

                             ip: 0.0.0.0

                            ip6: ::

                      protocols: 00000003

                            mac: 842b2b5b5a3b

                          state: OK

                          stats: 0 0 129 1 1

                    statusvalid: 1

                           type: scanning

                     

                         device: proxyB

                    statechange: 1327917188 (Mon Jan 30 10:53:08 2012)

                             ip: 10.x.b.201

                            ip6: ::

                      protocols: 00000003

                            mac: 842b2b5b5587

                          state: REDUNDANT

                          stats: 0 0 134 0 0

                    statusvalid: 1

                           type: redundant

                     

                         device: proxyB

                    statechange: 1327917188 (Mon Jan 30 10:53:08 2012)

                             ip: 10.x.b.201

                            ip6: ::

                      protocols: 00000003

                            mac: 842b2b5b5587

                          state: OK

                          stats: 0 0 134 0 0

                    statusvalid: 1

                           type: scanning

                     

                     

                     

                     

                    #### and proxy B:

                     

                    mfend-lb -s

                         device: proxyB

                    statechange:

                             ip: 10.x.b.201

                            ip6: ::

                      protocols: 00000003

                            mac: 842b2b5b5587

                          state: REDUNDANT

                    statusvalid: 1

                           type: redundant

                     

                         device: __SELF__

                    statechange:

                             ip: 0.0.0.0

                            ip6: ::

                      protocols: 00000003

                            mac: 842b2b5b5587

                          state: OK

                          stats: 0 0 54 0 0

                    statusvalid: 1

                           type: scanning

                     

                         device: proxyA

                    statechange:

                             ip: 10.x.b.102

                            ip6: ::

                      protocols: 00000000

                            mac: 842b2b5b5a3b

                          state: NETWORK

                          stats: 0 0 0 0 0

                    statusvalid: 1

                           type: director

                     

                     

                    Now...

                    When I log on to the management interface and change the priority on ProxyA -> 85 and Save changes then

                    pc1 can access the web through vrrp 8080. But... pc1 connections are now rejected! Of course FTP works.

                     

                    Any ideas?

                    • 7. Re: MWG7 - HTTP Listener address with Proxy HA setup
                      asabban

                      Hello,

                       

                      as the original topic was talking about problems without a 0.0.0.0 listener proxy port, is your last post related to this? I mean, if you add a listener port on 0.0.0.0, will the issue go away (and show up again once you remove the 0.0.0.0 listener)?

                       

                      From what I read so far I don´t think this is related. It would be helpful to check the mfend-lb -s output of both nodes once the failover was done and is in the "half-working" situation. Additionally it would be helpful to compare the Proxy HA settings in the UI to verify they look the same on both nodes.

                       

                      For VRRP a gratutious ARP request is used to tell the nearest router that the virtual IP address should now point to the physical MAC of the second node, once the first node failed. Can you verify the ARP tables were rewritten on the clients accordingly?

                       

                      If this does not show anything unsualy this is probably a better topic for support, since it would require some more research, but certainly feel free to post most data here.

                       

                      Best,

                      Andre

                      • 8. Re: MWG7 - HTTP Listener address with Proxy HA setup
                        tim.skopnik

                        Similar problem here:

                        We just added e new network to our proxy-ha-cluster and try to restrict client access to one of the (now) two network interfaces.

                        After removing the 0.0.0.0:xyz-listeners and adding listeners for the node-ip and the virtual cluster-ip (we use MWG 7.2.0.1.0 so the 127.0.0.1-listener seems unnecessary for us) the dashboard complains about "The listener on [virtual cluster-ip]:[port] could not be started." for the redundant node.

                         

                        On first try it was the ftp-listener on the second attempt it complained about the icap-listener.

                         

                        As the primary node is able to start the listeners the cluster is working w/o problems (we dont use loadbalancing - no port redirects configured).

                         

                        So the questions arising:

                        Why is dashboard not logging all listeners that failed? (i am quite sure the http- and xmpp-listener failed starting too - according to "netstat -nap")

                         

                        And (much more important):

                        Will the failover node start the listeners when the primary node is down?

                        Is defining listeners on virtual-ips "planned" by McAfee? Should this work? Or is using 0.0.0.0 as listener adress a "must" in the proxy-ha-case?

                         

                        cu. Tim