9 Replies Latest reply on Jun 13, 2013 8:52 AM by PhilM

    Agent conflict?

    PhilM

      I've been working with one of my customers over the past couple of days and have come up against something which has left us scratching our heads. Though, I'm sad to say, with v8 scratching my head is becoming an all-too-frequent occurrence.

       

      His HA cluster was running very slowly and as the culprit appeared to be DNS-related and he was only running 8.3.0, I recommended that he put 8.3.1 and 8.3.1P01 on the cluster as quickly as possible given there were specific references to fixes for BIND. He did so and that problem, thankfully, went away.

       

      Having encountered no previous issues with his configuration, we were both surprised that the upgrade process initially failed claiming that a couple of his rules were in conflict with others on the appliance. To allow him to get the patches installed as quickly as possible I suggested that he edit these two conflicting rules and replace the application with something completely innocuous and when the patches had successfully installed he could try to put them back again.

       

      In trying to do so, he is now seeing in the GUI the same error he encountered when first trying to install the patches.

       

      "IPv4 port conflict in zone internal between agent http_proxy and agent ssh_proxy on TCP port 22. The port conflict must be resolved by forcing all policy rules to use the same agent/application for this port. The http_proxy agent is referenced in the following rule and application sets:

                          <Rule name> : SFTP

       

      The ssh_proxy agent is references in the following rules and application sets:

                          <Rule name> : SSH

                          <Rule name> : SSH

                          <Rule name> : SSH"

       

      When I look at the application definition the customer has created for the application "SFTP", it is a plain and simple Infrastructure Service.

       

      Basically when the rule was first created it was to allow an outbound SFTP connection, but for reasons I assume to be related to Sam Leidl's document about how v8 applications and defense groups working, it did not work when using the SSH application. So the "SFTP" infrastructure service was created to stop the firewall from trying to "insert itself" into the conversation.

       

      As mentioned, prior to the 8.3.1 upgrade, creating this infrastructure service and rule didn't result in any errors - and the connection worked.

       

      What is most confusing about this error is the apparent insistence that the SFTP application and the rule which it is being used in are somehow causing the firewall to use the http_proxy agent. I have suggested to my customer that he use an application defense group which has HTTP set to "None" just to be safe. But this still results in the error when he tries to save the rule.

       

      Any inisght would be very helpful.

       

      -Phil.

        • 1. Re: Agent conflict?
          sliedl

          This is referenced, albeit only briefly, in the 8.3.1 Release Notes (PD24377):

           

          Policy validation

          Policy validation includes these updates:

          • Port conflict validation is performed on rules, which identifies rules containing different agents using the same port in the same zone.

           

          The firewall used to allow different agents on the same port but now it does not.

           

          The HTTP proxy and any 'generic' proxy you make are now, at v8, part of the same code.  We call it the 'mega-proxy'.  A generic proxy will audit as httpp sometimes because of that.

          1 of 1 people found this helpful
          • 2. Re: Agent conflict?
            PhilM

            OK - so I have two specific scenarios which were working in 8.3.0, aren't working in 8.3.1 and need to be working in 8.3.1.

             

            The first, as covered briefly in the OP is a need to allow outbound SSH and SFTP. By and large I would have expected it to work, because the SSH proxy is SFTP-aware. But when using an SSH application, the attempts to connect were failing because (I think) of the fact that you covered in your "how v8 does things" document that regardless of the assigned application defense, the SSH proxy will always be a proxy - and I believe the intervention of the Firewall is stopping the client and server from exchanging their keys. In the 8.3.0 scenario, creating and implementing a rule with a user-defined application running on TCP/22 did the trick.

             

            The second scenario concerns FTPS which I've always understood to be a bit of a hybrid protocol. It runs over port 21, but the native FTP application in MFE doesn't know, or reacts badly to, the additional TLS encryption. So, again, a user-defined application on TCP/21 did the trick.

             

            Both this rule and the rule for the SFTP connection were tied-down to specific destination endpoints, meaning that all other SSH and/or FTP traffic would be permitted under the mainstream rules.

             

            -Phil.

            • 3. Re: Agent conflict?
              sliedl

              Make sure the App Def Group you use in this rule has SSH set to 'something' (not <None>).  That may fix it.

               

              FTPS will only work if you open ALL ports above 1024, or simply set your server to only open the data ports on a certain range and then open that range.  That is all you can do; the firewall doesn't have an FTPS aware proxy.

              1 of 1 people found this helpful
              • 4. Re: Agent conflict?
                PhilM

                Hmmm....

                 

                The SSH solution I can try, but I am fearful that the issue I mentioned previously (taken from your "Basics of policy creation in version 8" document) will stop the client and server from being able to exchange their keys. I will ask the customer to give it a try and see what the results are.

                 

                I kind of support the fact that the Firewall doesn't have an FTPS-aware proxy as it does seem that FTPS is something of a cobbled-together protocol.


                But, sometimes a customer has a requirement (as is the case here) where an external party is making an service available to them, or requiring them to use this service and they have chosen to use FTPS. Can't exactly turn around and say "you can't do that" if it is necessary for their day-to-day operations. Equally interesting is the fact that in the 8.3.0 configuration this service worked without needing to open 1024-upwards. We simply used a custom application on TCP/21 in the rule and it worked. It could mean that the customer thought (or was told) that it was FTPS, but it wasn't

                 

                Had this been a v7 installation, I would have recommended using a packet filter service for this rule. Could we create a rule using the FTP application and simply change the app defense group to "connection settings"?

                 

                -Phil.

                • 5. Re: Agent conflict?
                  sliedl

                  The SSH proxy cannot pass PKI-based SSH traffic (SSH using certs).  It's impossible to do a MITM on PKI SSH traffic.  If that's what you are doing, then you need to use a generic proxy on 22, a packet filter, or (I think) leave SSH as <None> in the app defense.

                   

                  I've done a lot with FTPS.  You can decrypt/re-encrypt FTPS using the SSL rules on the firewall.  You can't do anything on the traffic (like scan it) but it does work to decrypt it.  There are different ways to do FTPS; using port 21 or port 990 for the control channel, explicit vs. implicit.  I had a customer 'claim' the same thing one time, that the FTP proxy passes FTPS, but it turned out he was incorrect and his customer was just doing plain FTP.

                   

                  Wikipedia warns you about FTPS and firewalls also:

                  Firewall incompatibilities

                  Because FTP utilizes a dynamic secondary port (for data channels), many firewalls were designed to snoop FTP protocol control messages in order to determine which secondary data connections they need to allow. However, if the FTP control connection is encrypted using TLS/SSL, the firewall cannot determine the TCP port number of a data connection negotiated between the client and FTP server. Therefore, in many firewalled networks, an FTPS deployment will fail when an unencrypted FTP deployment will work. This problem can be solved with the use of a limited range of ports for data and configuring the firewall to open these ports.

                   

                  Message was edited by: sliedl  -- control channel vs. data channel. on 6/12/13 11:12:55 AM CDT
                  1 of 1 people found this helpful
                  • 6. Re: Agent conflict?
                    PhilM
                    The SSH proxy cannot pass PKI-based SSH traffic (SSH using certs).  It's impossible to do a MITM on PKI SSH traffic.  If that's what you are doing, then you need to use a generic proxy on 22, a packet filter, or (I think) leave SSH as <None> in the app defense.

                     

                    What you're saying there is very interesting. Because, unless I've got my wires completely crossed, creating a generic proxy is exactly what we're trying to do, aren't we? - or are you saying that (as with older versions) when it is necessary to use a generic proxy, all rules needing access to that port number then need to use the generic proxy?

                    • 7. Re: Agent conflict?
                      sliedl

                      What I should have tried to impart in my blog post was that some of the applications will always post LISTENs when they are used (as they are proxies) vs. no LISTEN for a packet filter.  They tried to obfuscate the need for the admin to 'care' if he/she used a proxy or not.  That...was the plan.

                       

                      There are 65535 'doors' (ports) on the firewall.  Two people (processes) cannot stand in a door at the same time.  When you use SSH as the application it will always stand in the #22 door.   Therefore you cannot use the SSH proxy and a generic proxy on port 22 at the same time (they are different processes).  SSH is special, though, because the firewall has an SSH Server also.  This conflict is solved by making the server LISTEN only on the firewall IPs on port 22 and the proxy will do a wildcard * bind for all other traffic on 22.  At version 8 this is automatic (at v7 you had to edit sshd_config).

                       

                      When you use SSH in a rule it's the SSH proxy.  If you set the app def group to <None> for SSH it turns this 'smart' proxy into a 'dumb' proxy.  Do the same thing with the FTP proxy at v7 -- pull the app def slider to the bottom.  Now this is simply a 'dumb' proxy on port 21 - it will pass the control channel but it doesn't know how to pass the ephemeral data channel ports because it's just a generic proxy at this point.

                       

                      So, if you are doing PKI SSH you cannot have the SSH application be 'smart' (have anything but <None> selected in the app def group).  Or use a generic proxy.  Or use a filter.

                       

                      HOWEVER:  You CAN do PKI SSH using the SSH proxy if you set up all the certs.  This is detailed in all the Product Guides under the 'Policy in Action' section, "Decrypting and inspecting SSH content."  I don't believe any other enterpise firewalls can do that.

                      • 8. Re: Agent conflict?
                        sliedl

                        >> Could we create a rule using the FTP application and simply change the app defense group to "connection settings"?

                         

                        When you do this at v8 then the rule will use the FTP Packet Filter service (which is a separate service at v7).  That's how you configure it at v8 though.  It's a 'smart' filter service that can handle FTP.

                        • 9. Re: Agent conflict?
                          PhilM

                          Many thanks.

                           

                          I have taken the information from your various replies and have put together an e-mail for the customer so that he can understand why this has happened and what he can do to address these two requirements now he is running v8.3.1.

                           

                          -Phil.