cancel
Showing results for 
Search instead for 
Did you mean: 
dcaffrey
Level 10

MWG7 Progress Page Problem

Jump to solution

Hi,

I have a site where a user clicks a link which should display contents in the main page however it looks like the Progress Page is kicking in and breaking it

The request line seems to be doing a dynamic TIFF to PNG conversion

GET /Exe/tiff2png.exe/U000JSP8.PNG?-i+-r+105+-g+15+-h+21,2,5,36,14,4+%5C%5Cmedia-webhost3%5CMaster_archives%5C000%5C023%5C229%5C063%5CU000JSP8.TIF HTTP/1.1

This causes a redirect to this URL, if I paste this into the browser I get the progess page and the option to save a file U000JSP8.PNG

http://images.kantarmedia.ie/Exe/tiff2png.exe/U000JSP8.PNG?-i+-r+105+-g+15+-h+21,2,5,36,14,4+%5C%5Cm...

Any ideas what Rule I need to get this working ?

Thanks,

Dec

0 Kudos
1 Solution

Accepted Solutions
dstraube
Level 11

Re: MWG7 Progress Page Problem

Jump to solution

Hello Dec,

the website http://www.football365.com sends a different content type header for javascripts than what MWG has defined for them. That's why our rules are not working here.

In MWG javascript files are classified as text/javascript. The server sends them as content type application/javascript. Some of the advertising servers used on that websites even send their javascripts as application/x-javascript. As you can see pretty much every server could use a different name for it.

We are now faced with a dilemma: The content types coming from the server do not have to match the predefined media types in MWG. Even worse: we can not change the media type list that MWG uses. So what to do now?

We could use MWG's opener to check the media type of the file. In the default rules the opener comes after the progress page rule and that has a good reason. For the opener to decide if a file is really what it pretends to be it sometimes needs the whole file. The progress page wouldn't be of much use if we can only trigger it after MWG has already received it. That would be too late. So we can not use the opener to base our decision on. We have to use the content type that comes from the server, as that's the only thing we can work with at this point.

There is a way to solve this problem, but it requires quite some work. We need to define our own media type list based on strings. But first we need to change our progress page rule. MWG knows that "MediaType.FromHeader" defines a Media Type and will not offer a string list we can use. We need to convert the MediaType.FromHeader values to a string first, so that we can compare it to a list of strings. We can do it this way:

pp_to_string.jpg

The "is not in list" operator can stay, as our changed rule should work just like the one before, we just want to compare strings now and not media types. So we now add a new value (list of strings), that will contain the content types we like to bypass. I called it ProgressPageBypass, but you can choose any name you like.

We now need to add all the media types as string that we want to bypass. I included all that I had defined before in the media type list and added the application/javascript and application/x-javascript as well:

pp_string_bypass_list.jpg

You can imagine that it can be a lot of work if you need to maintain such a list and over time it can grow quite a bit. There is no perfect solution for all this, so I'm just showing ways to improve the situation and hope this helps a bit in understanding how MWG works.

Regards,

Dirk

on 7/15/11 10:06:14 AM CEST
0 Kudos
8 Replies
dstraube
Level 11

Re: MWG7 Progress Page Problem

Jump to solution

Hello Dec,

the Progress Page can break dynamic content and Ajax requests as you have noticed. If a request takes longer than 5 seconds (configurable, 5 seconds is just the default value) a progress page is displayed.

You could just increase the timeout value for the progress page under Policy -> Settings and then in the TreeView under Engines -> Progress Page -> Default,  then change the Timeout for "Delay for redirects to progress page".

Some sites might still be too slow, so you are probably better off if you change the rule set for the progress page.

The following change would skip Progress Pages for everything with /tiff2png.exe/ in it:

pp_bypass.jpg

Hope that helps,

Dirk

0 Kudos
dcaffrey
Level 10

Re: MWG7 Progress Page Problem

Jump to solution

Hi Dirk,

Many thanks, that works great, I've also increased the progress page delay which should help with other slow sites

I must admit the rule confuses me, I already had one which checks for certain media types and I added your check to it but the logic seems to be the opposite to what I would expect, can you shed some light on it ?

What is the best way to check for CSS files ?

I really only want Progress Pages for certain downloads such as Zip, ISO, Exe etc

Thanks again,

Dec

0 Kudos
dstraube
Level 11

Re: MWG7 Progress Page Problem

Jump to solution

Hello Dec,

let me try to explain the Progress Indication default rule set. The rule in question is "Enable Progress Page" and it pretty much just triggers the event "Enable Progress Page". So the rule takes care of turning the progress pages on. The action is "Stop Rule Set", which is needed for skipping the Data Trickling part. If you have a Progress Page you don't want to do Data Trickling, but that's another story.

So without any conditions our rule "Enable Progress Page" will always turn on the Progress Page with the Default setting.

You probably don't want that, that's why by default it at least checks the User-Agent  request header to be sure that the request came from a browser. Only if this condition is true the progress page will be enabled.

We now want to extend the check and add another condition. In my example it was URL.Path does not match */tiff2png/*. The "does not" part is important here, as we only want to enable progress pages if the url path does not contain the tiff2png part. If the url contains this string the condition will not match and it will not trigger the event. It's a bit confusing at first, as we want to make sure something is not there before we trigger the progress page.

If you want to extend the rule to check for CSS files and skip the progress page there as well you need to add another rule like this:

pp_bypass2.jpg

Here I added another condition, where I check that the MediaType.FromHeader property is not in the list of Media Types that I don't want progress pages for.

This isn't perfect, as it relies on the web server to send a (correct) content type header. You could also use Media Type.FromFileExtension (which is a list) for this check, but that would require the correct file extension. So it's a bit tricky.

Hope this clears it up a bit.

Regards,

Dirk

0 Kudos
dcaffrey
Level 10

Re: MWG7 Progress Page Problem

Jump to solution

Hi Dirk,

That's excactly what I was looking for, many thanks for taking the time to go through this, it clarifies the process very nicely.

If you get a chance could you have a look at a site http://www.football365.com this was displaying incorrectly i.e. loading on the left because the CSS files were triggering the Progress page as they take a long time to load. The rule fixes the CSS files but all the .js files are not excluded and I can't figure out why, I've lowered the time limit to trigger the rule so I can see in Firebug which components are being moved, here is an example of the Headers in question

Response Headers

HTTP/1.1 307 Moved TemporarilyLocation: http://www.football365.com/mwg-internal/de5fs23hu73ds/progress?id=eubNfsWjf0Content-Type: text/htmlProxy-Connection: CloseContent-Length: 276

Request Headers

GET http://www.football365.com/common/scripts/jquery.js HTTP/1.1Host: www.football365.comUser-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0Accept: */*Accept-Language: en-gb,en;q=0.5Accept-Encoding: gzip, deflateAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Proxy-Connection: keep-aliveReferer: http://www.football365.com/Cookie: rsi_segs=G07611_10219|G07611_10492|G07611_10499; c=undefinedDirect%20LoadDirect%20Load; s_ctq=0; s_cc=true; s_sq=%5B%5BB%5D%5D

Is it because of the "Accept: */*" in the Request Headers ?

Many Thanks,

Dec

0 Kudos
dstraube
Level 11

Re: MWG7 Progress Page Problem

Jump to solution

Hello Dec,

the website http://www.football365.com sends a different content type header for javascripts than what MWG has defined for them. That's why our rules are not working here.

In MWG javascript files are classified as text/javascript. The server sends them as content type application/javascript. Some of the advertising servers used on that websites even send their javascripts as application/x-javascript. As you can see pretty much every server could use a different name for it.

We are now faced with a dilemma: The content types coming from the server do not have to match the predefined media types in MWG. Even worse: we can not change the media type list that MWG uses. So what to do now?

We could use MWG's opener to check the media type of the file. In the default rules the opener comes after the progress page rule and that has a good reason. For the opener to decide if a file is really what it pretends to be it sometimes needs the whole file. The progress page wouldn't be of much use if we can only trigger it after MWG has already received it. That would be too late. So we can not use the opener to base our decision on. We have to use the content type that comes from the server, as that's the only thing we can work with at this point.

There is a way to solve this problem, but it requires quite some work. We need to define our own media type list based on strings. But first we need to change our progress page rule. MWG knows that "MediaType.FromHeader" defines a Media Type and will not offer a string list we can use. We need to convert the MediaType.FromHeader values to a string first, so that we can compare it to a list of strings. We can do it this way:

pp_to_string.jpg

The "is not in list" operator can stay, as our changed rule should work just like the one before, we just want to compare strings now and not media types. So we now add a new value (list of strings), that will contain the content types we like to bypass. I called it ProgressPageBypass, but you can choose any name you like.

We now need to add all the media types as string that we want to bypass. I included all that I had defined before in the media type list and added the application/javascript and application/x-javascript as well:

pp_string_bypass_list.jpg

You can imagine that it can be a lot of work if you need to maintain such a list and over time it can grow quite a bit. There is no perfect solution for all this, so I'm just showing ways to improve the situation and hope this helps a bit in understanding how MWG works.

Regards,

Dirk

on 7/15/11 10:06:14 AM CEST
0 Kudos
dcaffrey
Level 10

Re: MWG7 Progress Page Problem

Jump to solution

Hi Dirk,

That's great I added this to my existing rule and it's catching the javascript now, what did you use to identify the content type as application/javascript ?

Many Thanks,

Dec

0 Kudos
dstraube
Level 11

Re: MWG7 Progress Page Problem

Jump to solution

Hello Dec,

That's great I added this to my existing rule and it's catching the javascript now, what did you use to identify the content type as application/javascript ?

Glad to hear that it helped. I used a good old-fashioned network trace to check the server responses .

Regards,

Dirk

0 Kudos
tnt
Level 7

Re: MWG7 Progress Page Problem

Jump to solution

Thanks a lot.

I solved my problem with this workaround.

0 Kudos