Hello everyone. We just received our renewal license for Web Reporter and the appliance is now reporting that it is unable to download the TrustedSource Web Database because it does not have enough memory. In trying to download it manually, we receive the same error. Does anyone know what needs to be done to remedy the issue? Thanks in advance!
We are having the same issue except that this is a brand new install of Web Reporter and we have never been able to download the database successfully. I have watched memory utilization on the server while the download was occuring and all seems normal. However, we do have our database on a separate server and I have not checked it yet. Hoping someone will have some ideas.
I learned something from Erik Elsasser on a different discussion that might be helpful....
The "database" is loaded into RAM on the Web Reporter server. You should have at least 4GB of RAM on the Web Reporter server even though the product guide only says 2GB is needed. I guess 2GB would be sufficient if you were not downloading this database.
Minimum requirements: 2GB RAM
Recommended: 4GB of RAM + 64-bit Web Reporter 5.1.x or newer.
To be specific, the URL Database requires about 400 Mb block of contiguous RAM to load (it varies slighly from build to build). The reason Elsasser recommended 4 GB is to try and maximize the chance of having sufficient continuous free memory to load the list. I typically recommend 4GB and 64bit versions only.
Out of the box with a default configuration, Web Reporter will only use ~600Mb of RAM. The URL Database will be in addition to this. On 32-bit platforms, the operating system has a further restriction that each process can only have 2Gb of kernel memory. Since Web Reporter is single process, everything must fit into this 2Gb window. For that reason, I recommend 4Gb and 64-bit Web Reporter versions only (must be 5.1.x or newer).
On the other hand, loading the URL database in Web Reporter shouldn't be necessary since the category data should already be included in access logs. Here is a good article for MWG 6.8.x
The only uncategorized data should be from white-listed sites that were not categorized, but I still argue against having Web Reporter categories these because your reports may not reflect the policy enforcement on the proxy/gateway. For example, if a gateway policy mis-configuration skips categorization for www.some-adult-site.com, then the access log would show user jdoe went to www.some-adult-site.com (uncategorized) and was not blocked. However, the reports in Web Reporter would show user jdoe went to www.some-adult-site.com (Adult/Pornography) and was not blocked.
This could cause confusion because it looks like jdoe is being allowed to access a categorized adult site. The problem is that it wasn't categorized on the gateway. Seeing it as uncategorized in the reports would tip you off that there is a problem with your whitelist on the gateway.