In the instance, you have had Web Reporter up and running for a while now, it might be a good idea to check your settings for database deletions and data retention. One issue that is crucial to the health of your Web Reporter system is the amount of free space on the drive containing the data files. If the system runs out of space, there are many negative impacts, which can be introduced as a result:
* Keeping unnecessary data in Web Reporter can lead to degraded performance
"Page Views" are a default log parsing option that condenses log data by removing requests generated by embeded content. An example of this is taking all of the requests for one website including the components in the page like "style sheets and images" would get recorded as just on site. For more information on the page views, please referrer to the following communtiy article (DOC-4662).
If you disabled page views, this is something that you might want to enable to help cut down on space and to help reduce the amount of non-needed information in your database. To enable page views, you will need to go "Administration > Setup > Log Sources > **MyLogSourceName**" then with this selected hit the [Edit] button and go to the section labeled "Processing". In this location, you will see the following option;
Is this a shared system?
Shared system - If the Web Reporter is existing on a shared server with other applications, you might want to check the load of the system and the available resources. In some instances where this system could be used in VMware environments, there could be added impact from slower disk I/O operations due to the Web Reporter existing on the same system as other systems. Please ensure that you have enough systems resources for the Web Reporter as documented in our product guides. Additionally, the Web Reporter database can be moved to an external database in another location to reduce the load on the system. Please reference the following knowledge base article (68025) for additional information.
Properly configuring Database Maintenance in Web Reporter is the most effective way to control the size of the reporting database. This section will describe in more detail the options available. The location for the Database Maintenance settings are "Administration > Tools > Database Maintenance". In this location, you will see something like the following:
Some considerations you might want to take:
* For more information on "Detailed Records" and "Summary Records" please referrer to the following:
In the configuration of the database maintenance there is an option for "Roll ups" which in some ways might sound like a good idea but in the end, there is not a big benefit to having this enabled. It is recommended to disable roll ups.
* For more information about the database rollups, please reference the following Knowlegebase article: Web Reporter - Deployment Best Practices
If you have been using the product for a while and you have noticed that things could be slowing down or performance is lagging, this could be the result of fragmentation or a database which needs to have the indexes rebuilt. For more information about indexes please reference, ORACLE, Microsoft and MYSQL. In Web Reporter, you have the option to define a schedule for when you want the database maintenance to perform the database indexing. This option is located under "Administration > Tools > Database Maintenance" which looks like the following:
Database Setup Guides:
When deleting records for all databases types the information deleted is copied off to a temporary location which is known as a transaction log or temp table space. This location is essentially a buffer point for records while a specific task is performed on the database.
What does this mean?
As the transaction log can grow in size causing issues with an already full drive or system there are some things which can be done to alleviate this issue which are covered in the section for "Reclaiming Space From Deletions" for the Optimize, Shrink and Reorganization commands.
One thing that is crucial in doing database deletions on a system that is critically low on space is to reduce the deletion "Batch" size. This item is located under "Maintenance Options" the which looks like the following;
* This will cause Web Reporter to delete smaller chunks of data at a time and help keep the transaction log small. The default value "All" is recommended under normal operation.
After deletions are performed on a database, the database will not re-allocate space back to the file system automatically. This is a manual process which needs to be performed inside of the database using the information referenced below.
Generally there are three steps required to free up space if you ran out of disk space:
1. Free up some space on the file system (outside your database). See "Locations to get some space" below.
2. Now you have enough space for database maintenance see 'Database Maintenance' section above.
3. Shrink the database files to regain more space on the file system. See 'Reclaiming Space From Deletions' section above.
In the event that you are totally out of space and need to make some wiggle room so that you can start the deletion of records. Here are some locations, which can be cleared out to make more space on the system; some of these locations are covered below and in the knowledge base article (KB69642):
* Feedbacks need to be removed manually from the system. You can remove these to get more space in the location described in the dialog box.
Note: Must be performed after the disk cleanup!
In the event the Web Reporter went down in a unclean state or the processes crashed due to either an out of space issue or a environmental issue, this could result in database corruption. In the event of database corruption, upon logging into the Web Reporter and going into the "Administration > Setup > Database" location, the database could either be in a re-connecting state or a always offline state. If the Web Reporter internal database is corrupt, you will be able to see "Table 'scr_fct_web' is marked as crashed and should be repaired" errors in the server.log located at "/Web Reporter/reporter/log/" (Windows) or "/opt/Mcafee/WebReporter/reporter/log/" (Unix). If you are using MYSQL as a external database, you will have to check the locations /var/log/messages, /var/log/mysql.log or wherever the MYSQL log is stored.
Most of the time, the tables which can crash are the following:
In the instance of database corruption of the Internal or MYSQL database structures, you will need to use the "myisamchk" utility. If you need more information on the utility itself, please feel free to reference the MYSQL website.
To repair the database please follow the following steps:
myisamchk --repair ..\var\reporting\scr_fct_exact_access.myi
myisamchk --repair ..\var\reporting\scr_fct_web.myi
myisamchk -r ..\var\reporting\scr_fct_exact_access.myi
myisamchk -r ..\var\reporting\scr_fct_web.myi
myisamchk -r -f ..\var\reporting\scr_fct_exact_access.myi
myisamchk -r -f ..\var\reporting\scr_fct_web.myi
myisamchk -r -f --sort_buffer_size=64M ..\var\reporting\scr_fct_exact_access.myi
myisamchk -r -f --sort_buffer_size=64M ..\var\reporting\scr_fct_web.myi
In the event that you need to move to or provision a new database, please reference the following best practice setup guides;