8 Replies Latest reply on Aug 30, 2011 4:20 AM by tnt

    MWG7 Progress Page Problem

    dcaffrey

      Hi,

       

      I have a site where a user clicks a link which should display contents in the main page however it looks like the Progress Page is kicking in and breaking it

       

      The request line seems to be doing a dynamic TIFF to PNG conversion

       

      GET /Exe/tiff2png.exe/U000JSP8.PNG?-i+-r+105+-g+15+-h+21,2,5,36,14,4+%5C%5Cmedia-we bhost3%5CMaster_archives%5C000%5C023%5C229%5C063%5CU000JSP8.TIF HTTP/1.1

       

      This causes a redirect to this URL, if I paste this into the browser I get the progess page and the option to save a file U000JSP8.PNG

       

      http://images.kantarmedia.ie/Exe/tiff2png.exe/U000JSP8.PNG?-i+-r+105+-g+15+-h+21 ,2,5,36,14,4+%5C%5Cmedia-webhost3%5CMaster_archives%5C000%5C023%5C229%5C063%5CU0 00JSP8.TIF

       

      Any ideas what Rule I need to get this working ?

       

      Thanks,

       

      Dec

        • 1. Re: MWG7 Progress Page Problem
          dstraube

          Hello Dec,

           

          the Progress Page can break dynamic content and Ajax requests as you have noticed. If a request takes longer than 5 seconds (configurable, 5 seconds is just the default value) a progress page is displayed.

           

          You could just increase the timeout value for the progress page under Policy -> Settings and then in the TreeView under Engines -> Progress Page -> Default,  then change the Timeout for "Delay for redirects to progress page".

           

          Some sites might still be too slow, so you are probably better off if you change the rule set for the progress page.

           

          The following change would skip Progress Pages for everything with /tiff2png.exe/ in it:

           

          pp_bypass.jpg

          Hope that helps,

           

          Dirk

          • 2. Re: MWG7 Progress Page Problem
            dcaffrey

            Hi Dirk,

             

            Many thanks, that works great, I've also increased the progress page delay which should help with other slow sites

            I must admit the rule confuses me, I already had one which checks for certain media types and I added your check to it but the logic seems to be the opposite to what I would expect, can you shed some light on it ?

             

            What is the best way to check for CSS files ?

             

            I really only want Progress Pages for certain downloads such as Zip, ISO, Exe etc

             

            Thanks again,

             

            Dec

            • 3. Re: MWG7 Progress Page Problem
              dstraube

              Hello Dec,

               

              let me try to explain the Progress Indication default rule set. The rule in question is "Enable Progress Page" and it pretty much just triggers the event "Enable Progress Page". So the rule takes care of turning the progress pages on. The action is "Stop Rule Set", which is needed for skipping the Data Trickling part. If you have a Progress Page you don't want to do Data Trickling, but that's another story.

               

              So without any conditions our rule "Enable Progress Page" will always turn on the Progress Page with the Default setting.

               

              You probably don't want that, that's why by default it at least checks the User-Agent  request header to be sure that the request came from a browser. Only if this condition is true the progress page will be enabled.

               

              We now want to extend the check and add another condition. In my example it was URL.Path does not match */tiff2png/*. The "does not" part is important here, as we only want to enable progress pages if the url path does not contain the tiff2png part. If the url contains this string the condition will not match and it will not trigger the event. It's a bit confusing at first, as we want to make sure something is not there before we trigger the progress page.

               

              If you want to extend the rule to check for CSS files and skip the progress page there as well you need to add another rule like this:

               

              pp_bypass2.jpg

              Here I added another condition, where I check that the MediaType.FromHeader property is not in the list of Media Types that I don't want progress pages for.

               

              This isn't perfect, as it relies on the web server to send a (correct) content type header. You could also use Media Type.FromFileExtension (which is a list) for this check, but that would require the correct file extension. So it's a bit tricky.

               

              Hope this clears it up a bit.

               

              Regards,

               

              Dirk

              • 4. Re: MWG7 Progress Page Problem
                dcaffrey

                Hi Dirk,

                 

                That's excactly what I was looking for, many thanks for taking the time to go through this, it clarifies the process very nicely.

                 

                If you get a chance could you have a look at a site http://www.football365.com this was displaying incorrectly i.e. loading on the left because the CSS files were triggering the Progress page as they take a long time to load. The rule fixes the CSS files but all the .js files are not excluded and I can't figure out why, I've lowered the time limit to trigger the rule so I can see in Firebug which components are being moved, here is an example of the Headers in question

                 

                Response Headers

                HTTP/1.1 307 Moved TemporarilyLocation: http://www.football365.com/mwg-internal/de5fs23hu73ds/progress?id=eubNfsWjf0Content-Type: text/htmlProxy-Connection: CloseContent-Length: 276

                Request Headers

                GET http://www.football365.com/common/scripts/jquery.js HTTP/1.1Host: www.football365.comUser-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0Accept: */*Accept-Language: en-gb,en;q=0.5Accept-Encoding: gzip, deflateAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Proxy-Connection: keep-aliveReferer: http://www.football365.com/Cookie: rsi_segs=G07611_10219|G07611_10492|G07611_10499; c=undefinedDirect%20LoadDirect%20Load; s_ctq=0; s_cc=true; s_sq=%5B%5BB%5D%5D

                 

                Is it because of the "Accept: */*" in the Request Headers ?

                 

                Many Thanks,

                 

                Dec

                • 5. Re: MWG7 Progress Page Problem
                  dstraube

                  Hello Dec,

                   

                  the website http://www.football365.com sends a different content type header for javascripts than what MWG has defined for them. That's why our rules are not working here.

                   

                  In MWG javascript files are classified as text/javascript. The server sends them as content type application/javascript. Some of the advertising servers used on that websites even send their javascripts as application/x-javascript. As you can see pretty much every server could use a different name for it.

                   

                  We are now faced with a dilemma: The content types coming from the server do not have to match the predefined media types in MWG. Even worse: we can not change the media type list that MWG uses. So what to do now?

                   

                  We could use MWG's opener to check the media type of the file. In the default rules the opener comes after the progress page rule and that has a good reason. For the opener to decide if a file is really what it pretends to be it sometimes needs the whole file. The progress page wouldn't be of much use if we can only trigger it after MWG has already received it. That would be too late. So we can not use the opener to base our decision on. We have to use the content type that comes from the server, as that's the only thing we can work with at this point.

                   

                  There is a way to solve this problem, but it requires quite some work. We need to define our own media type list based on strings. But first we need to change our progress page rule. MWG knows that "MediaType.FromHeader" defines a Media Type and will not offer a string list we can use. We need to convert the MediaType.FromHeader values to a string first, so that we can compare it to a list of strings. We can do it this way:

                   

                  pp_to_string.jpg

                  The "is not in list" operator can stay, as our changed rule should work just like the one before, we just want to compare strings now and not media types. So we now add a new value (list of strings), that will contain the content types we like to bypass. I called it ProgressPageBypass, but you can choose any name you like.

                  We now need to add all the media types as string that we want to bypass. I included all that I had defined before in the media type list and added the application/javascript and application/x-javascript as well:

                   

                  pp_string_bypass_list.jpg

                   

                  You can imagine that it can be a lot of work if you need to maintain such a list and over time it can grow quite a bit. There is no perfect solution for all this, so I'm just showing ways to improve the situation and hope this helps a bit in understanding how MWG works.

                   

                  Regards,

                   

                  Dirk

                   

                  on 7/15/11 10:06:14 AM CEST
                  • 6. Re: MWG7 Progress Page Problem
                    dcaffrey

                    Hi Dirk,

                     

                    That's great I added this to my existing rule and it's catching the javascript now, what did you use to identify the content type as application/javascript ?

                     

                    Many Thanks,

                     

                    Dec

                    • 7. Re: MWG7 Progress Page Problem
                      dstraube

                      Hello Dec,

                       

                      That's great I added this to my existing rule and it's catching the javascript now, what did you use to identify the content type as application/javascript ?

                       

                      Glad to hear that it helped. I used a good old-fashioned network trace to check the server responses .

                       

                      Regards,

                       

                      Dirk

                      • 8. Re: MWG7 Progress Page Problem

                        Thanks a lot.

                        I solved my problem with this workaround.