Best Practices: Whitelisting/Blacklisting for SSL Traffic




    This document will cover the configuration of how to whitelist or blacklist SSL (HTTPs) based websites while using the SSL scanner.  There are many different configurations which a ruleset can be configured with however so we will provide some best practice ideas when implementing rules along with some information into how the SSL (HTTPs) process works.






    In order to block or allow SSL traffic on the Web Gateway, you are going to need to have some basic knowledge of how SSL traffic works with proxies and what needs to be done to inspect SSL traffic.  One major thing to take into consideration is that SSL traffic is different from HTTP traffic as more steps are needed to ensure the encryption of the traffic.


    In HTTP, the full URL ( is transmitted in clear text and whitelisting (or blocking) can apply to the full URL.


    For HTTPS requests, only the Host ( is transmitted in clear text. The full URL is encrypted inside the SSL Tunnel. To whitelist the full URL SSL inspection is needed and you have to ensure that the request is not blocked, before the full URL has been transmitted in the tunnel.




    Here is an overview of the difference between the two;







    This is what a HTTP connection looks like in Wireshark;




    This is what happens during a HTTPs (SSL) “CONNECT” command;





    Here is an example of a “CONNECT” attempt from the client which contains the HTTP header information and only the “Host”, not the full URL.  This is what the Web Gateway will be able to see if the request is “tunneled” or the “SSL Scanner” is not used.




    When working with SSL traffic, there are two very important scenarios to consider:




    When working with No SSL scanner you will need to keep the following things in mind;


    • After the “CONNECT” occurs, the data will be encrypted.
    • The “CONNECT” only contains destination host and not the full URL (and very few other headers that can be read).
    • The Steps below do NOT apply to setups without SSL Scanner. Without SSL Scanner, only hosts, not full URLs can be whitelisted




    When working with SSL scanner you will need to keep the following things in mind;


    • The “CONNECT” only contains destination host and not the full URL (and very few other headers that can be read).
    • If the CONNECT is blocked, the full URL will never be visible to the web gateway
    • If the CONNECT is NOT blocked, the traffic will be decrypted on the Web Gateway and at this time, the full url will visible.
    • When using the SSL Scanner, there are going to be two different “Request” Cycles that will need to occur;


      • One for equals CONNECT
      • One for equals CERTVERIFY


    (Both of these “Request Cycle” events need to be allowed for the full URL to be passed in the SSL Tunnel in order to allow the whitelisting to occur.)



    Example Scenario


    In our case, the website “” is blocked based on the “Information Security” category but still users should be allowed to go to one specific URL on the site;



    If this were to be an HTTP request you would block the category “Information Security” in your URL Filter block rules and then allowing this one specific URL in an exception before the blocking rule (for example, the global whitelist).  This will not work with SSL/HTTPS traffic;







    Blocking Rule





    For HTTPS requests this results in the following. The CONNECT request is being blocked:





    The Issue is, that the CONNECT request the client made for the https site, did NOT contain the full URL, but only the host As a result, your whitelist did not match, and the category block was applied.


    To resolve this issue, you would need to make a rule to allow the “CONNECT” command and “CERTVERIFY” to pass through without getting blocked.  We know what the URL.HOST is so you could make a rule like the following;






    This will then allow the “CONNECT” part of the SSL traffic to pass through the Web Gateway allowing the tunnel to be established so that the full URL can be requested in the tunnel.


    Note that no actual traffic (requests or responses) are exchanged during the CONNECT. Only the connections are established and certificates are exchanged. The “GET” request inside the tunnel, is still being filtered as normal with these new rules.




    The Web Gateway will now be able to decrypt the traffic and we will be left with the following as seen in a connection trace using the SSL Scanner;



    Decrypted Traffic.jpg




    In conclusion, the site will now load correctly for the part of the site which we did not want to have blocked (Full URL added to your whitelist earlier) while yet blocking the rest of the site.  Granted, it might be missing a style sheet and some pictures but this is because we only allowed the one very specific URL. The other URLs could be added to the whitelist as needed.


    Please keep in mind that when trying to troubleshoot SSL traffic, you will have to use Connection Tracing to see inside the tunnel.  This is enabled in the “Configuration > Troubleshooting” section and the output will be in the “Troubleshooting > Connection Tracing” section.


    Additionally, if you plan to use the connection tracing for troubleshooting, please enable it but restrict it to one test client or you can risk filling up the Web Gateway.


    Furthermore, the client side of the tunnel will be the traces labeled like “-C.txt” and the Web Server side of the tunnel will be labeled like “-S.txt”.





    With this information, you should now be able to whitelist and blacklist URLs on the Web Gateway.  Furthermore, you should have some additional knowledge into some of the troubleshooting steps which can be implemented for troubleshooting SSL (HTTPs) traffic as well.