cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted

Web Gateway max file size before redirecting to download page

Jump to solution

Hello Everyone,

We are developing a serious game using Unity3d  engine for an enterprise.
Our client uses McAfee Web Gateway.
Our game downloads successfully many files without any problem. However, when it comes to download a large file like 70MB, MWG redirects it to download page and our app cannot access the file.

From forum posts I've learned that they can re-configure their system to use "data trickling". However, changing their entire config just for us may not be a reasonable solution to offer to them.

Therefore, we are seeking a workaround. First approach is to divide the big files to smaller sized files. As files to download can reach 150-200MB each we need to find the maximum file size that MWG can handle without redirecting it to download page.

I will also appreciate any other possible solutions you may advice.

Many thanks in advance

2 Solutions

Accepted Solutions
Highlighted
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 6 of 8

Re: Web Gateway max file size before redirecting to download page

Jump to solution

it seems the OP seeks a way to avoid triggering the progress page from the server side. My understanding that the progress page is triggered when the the web server is slow to start download. I think time-to-first-byte and time-to-first-X-KBs are important. The other way to avoid the progress page is using a different user-agent string - by default the progress page is triggered only for user-agents which contains *mozilla* string in it. My suggestions:

1. use fast web server and fast pipes, try use AWS or some fast CDN for hosting. The download should start immediatelly without delay and the server/pipe should provide high bandwidth download speed.

2. ask the client to try wget/curl for download - if it successful, use a custom downloader.

You can provide a direct link to your file so I can check it.

 

View solution in original post

Highlighted
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 4 of 8

Re: Web Gateway max file size before redirecting to download page

Jump to solution

Hi @aatalay ,

There is no "max file size" before redirecting the the Progress Page.

The default Progress Page rules specify that if a transaction takes longer than 5 seconds to return data to the client, redirect to the progress page.

So, while a larger size can make it more likely to download/scan for longer than 5 seconds, it is possible for a small, but complex archive with many child elements to also take a longer time to scan. It is a function of both the size, and the complexity of the file being scanned that can cause a Progress Page to be triggered under "normal/default" circumstances.

View solution in original post

7 Replies
Highlighted
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 2 of 8

Re: Web Gateway max file size before redirecting to download page

Jump to solution

Hello,

 

there are only the 2 options, progress page and data trickling.

 

There is no need to fully use data trickling for all traffic. You can use different criteria to build this rule. Best would be if you know all important IP addresses or URL hosts used for these downloads. Then you could build a rule like:
URL.Host is in list <list with download URL hosts>, Action: Stop Rule Set, Event: Enable Data Trickling

 

Example rule set would look like:
Rule 1: URL.Host is in list <list with download URL hosts>, Action: Stop Rule Set, Event: Enable Data Trickling
or alternatively:
URL.Destination.IP is in list <list with download destination IPs>, Action: Stop Rule Set, Event: Enable Data Trickling
Rule 2: <default progress page rule for all browser related things>
Rule 3: <default data trickling rule for the rest>

Something like that is a common setup since applications cannot display a HTML (progress) pages normally. They have to use data trickling instead. Therefore, use a known simple criteria and configure a rule like above to separate this traffic in policy.

 

Let us know if you have further questions.

 

Regards,
Marcel

Regards,
Marcel Kutrieba
Technical Support Engineer
Highlighted

Re: Web Gateway max file size before redirecting to download page

Jump to solution

Thank you mkutrieba for your prompt reply.

In their LMS system, implementation of these packages may be done several times in time. Therefore, the training department will need to request a specific rule from security team for every implementation.

Although, we will share your comment with them, this is still doesn't seems an appropriate way of dealing  with large living implementations, hence it requires exception configuration each time.

Therefore, we still need to find an alternative.
As you mentioned that there is no other options, the best solution seems reducing the file sizes.

What is the maximum file size that MWG accepts before redirecting it to "progress page"?

Thank you

Highlighted
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 4 of 8

Re: Web Gateway max file size before redirecting to download page

Jump to solution

Hi @aatalay ,

There is no "max file size" before redirecting the the Progress Page.

The default Progress Page rules specify that if a transaction takes longer than 5 seconds to return data to the client, redirect to the progress page.

So, while a larger size can make it more likely to download/scan for longer than 5 seconds, it is possible for a small, but complex archive with many child elements to also take a longer time to scan. It is a function of both the size, and the complexity of the file being scanned that can cause a Progress Page to be triggered under "normal/default" circumstances.

View solution in original post

Highlighted

Re: Web Gateway max file size before redirecting to download page

Jump to solution

@swilkens1 thanks for clarifying the way MWG works. This gives us a good start point.

Highlighted
Reliable Contributor
Reliable Contributor
Report Inappropriate Content
Message 6 of 8

Re: Web Gateway max file size before redirecting to download page

Jump to solution

it seems the OP seeks a way to avoid triggering the progress page from the server side. My understanding that the progress page is triggered when the the web server is slow to start download. I think time-to-first-byte and time-to-first-X-KBs are important. The other way to avoid the progress page is using a different user-agent string - by default the progress page is triggered only for user-agents which contains *mozilla* string in it. My suggestions:

1. use fast web server and fast pipes, try use AWS or some fast CDN for hosting. The download should start immediatelly without delay and the server/pipe should provide high bandwidth download speed.

2. ask the client to try wget/curl for download - if it successful, use a custom downloader.

You can provide a direct link to your file so I can check it.

 

View solution in original post

Highlighted
McAfee Employee
McAfee Employee
Report Inappropriate Content
Message 7 of 8

Re: Web Gateway max file size before redirecting to download page

Jump to solution

What @fw_mon  mentioned regarding user-agent strings is true; the default Progress Page rules only apply if it is assumed to be a browser based on the UA string matching *mozilla*.

While slow server-side downloads can also be a factor, I have more often seen the bigger contributor to be scan times on the MWG itself.

Highlighted

Re: Web Gateway max file size before redirecting to download page

Jump to solution

Thanks fw_mon,

1- is obvious
2- It may be a longer way but if we can not solve it by decreasing the file sizes we will definitely try your solution

Thank you

You Deserve an Award
Don't forget, when your helpful posts earn a kudos or get accepted as a solution you can unlock perks and badges. Those aren't the only badges, either. How many can you collect? Click here to learn more.

Community Help Hub

    New to the forums or need help finding your way around the forums? There's a whole hub of community resources to help you.

  • Find Forum FAQs
  • Learn How to Earn Badges
  • Ask for Help
Go to Community Help

Join the Community

    Thousands of customers use the McAfee Community for peer-to-peer and expert product support. Enjoy these benefits with a free membership:

  • Get helpful solutions from McAfee experts.
  • Stay connected to product conversations that matter to you.
  • Participate in product groups led by McAfee employees.
Join the Community
Join the Community