6 Replies Latest reply on Feb 12, 2010 2:59 PM by wwarren

    VS8.7 on File server bogged down by Sharepoint crawling.

    wolfman099

      I recently started receiving complaints about performance on our File server.  This morning, it was noticeably bad.  I checked the On Access Scan log and noticed that our new MOSS 2007 server was crawling the File server and trying to index it.

       

      Here is a sample from the log:  There were thousand of entries throughout the night just like this.  The spscrawler account is the account he was using.

       

      2/11/2010 7:43:39 AM Not scanned (scan timed out) Domain\spscrawler System:Remote D:\SHARES\Somefolder1\file1.ppt
      2/11/2010 7:46:13 AM Not scanned (scan timed out) Domain\spscrawler System:Remote D:\SHARES\Somefolder1\file2.PPT
      2/11/2010 7:50:52 AM Not scanned (scan timed out) Domain\spscrawler System:Remote D:\SHARES\Somefolder2\file3.zip
      2/11/2010 7:58:52 AM Not scanned (scan timed out) Domain\spscrawler System:Remote D:\SHARES\Somefolder3\file4.xls

       

      I asked our MOSS admin to cancel the crawling and performance immediatley improved.

       

      I know that scanning zips can sometimes cause an issue. I know that I could disable that option. Is standard practice to disable scanning of zips on file servers for on-access scanning?  I perform daily and weekly scans of the file server which DOES include scanning of zips.

       

      The PPT and XLS files are most likely password protected because they looked like they were in financial type folders.  What i'm wondering is if there is a way to create an exclusion in Mcafee for the crawling service.  Keep in mind that the crawling is done remotely, so there is not a local process on the File server for it.  Is there any way to do this?  Our plan is to only do the crawling at night, but there will be an occasional instance where we need to do a full crawl of the file server.

       

      I appreciate any help that anyone can provide.  Thanks!

        • 1. Re: VS8.7 on File server bogged down by Sharepoint crawling.
          rmetzger

          wolfman099 wrote:

           

          I recently started receiving complaints about performance on our File server.  This morning, it was noticeably bad.  I checked the On Access Scan log and noticed that our new MOSS 2007 server was crawling the File server and trying to index it.

           

          Here is a sample from the log:  There were thousand of entries throughout the night just like this.  The spscrawler account is the account he was using.

           

          2/11/2010 7:43:39 AM Not scanned (scan timed out) Domain\spscrawler System:Remote D:\SHARES\Somefolder1\file1.ppt
          2/11/2010 7:46:13 AM Not scanned (scan timed out) Domain\spscrawler System:Remote D:\SHARES\Somefolder1\file2.PPT
          2/11/2010 7:50:52 AM Not scanned (scan timed out) Domain\spscrawler System:Remote D:\SHARES\Somefolder2\file3.zip
          2/11/2010 7:58:52 AM Not scanned (scan timed out) Domain\spscrawler System:Remote D:\SHARES\Somefolder3\file4.xls

           

          I asked our MOSS admin to cancel the crawling and performance immediatley improved.

           

          I know that scanning zips can sometimes cause an issue. I know that I could disable that option. Is standard practice to disable scanning of zips on file servers for on-access scanning?  I perform daily and weekly scans of the file server which DOES include scanning of zips.

           

          The PPT and XLS files are most likely password protected because they looked like they were in financial type folders.  What i'm wondering is if there is a way to create an exclusion in Mcafee for the crawling service.  Keep in mind that the crawling is done remotely, so there is not a local process on the File server for it.  Is there any way to do this?  Our plan is to only do the crawling at night, but there will be an occasional instance where we need to do a full crawl of the file server.

           

          I appreciate any help that anyone can provide.  Thanks!

          Since you are running Nightly and Weekly scans, you might consider changing the On-access Scanner to only scan "When writing to disk." This should stop the crawler from having every file read, scanned. If archives are written to disk temporarily during the crawl, they will get scanned, but that should not kill performance nearly as much as scanning every single file.

           

          Be sure to follow the AV Exclusions that Microsoft recommends for SharePoint:

          http://support.microsoft.com/kb/952167

          http://support.microsoft.com/kb/943620

           

          I hope this is helpful. Let us know how things are going.

          Ron Metzger

          • 2. Re: VS8.7 on File server bogged down by Sharepoint crawling.
            wolfman099

            Ron,


            Thank you for the reply.  I think I will run your solution by our IT Security team to see what they think.  I agree that we should not need to scan reads on our File Server since I am doing the nightly/weekly scan.

             

            I will be sure to add the Sharepoint exclusions.  Those servers just came online and I hadn't got to it yet.  I will make that a priority.  Thanks!

            • 3. Re: VS8.7 on File server bogged down by Sharepoint crawling.

              Generally, you don't need to scan archives on read/write. Scanning them causes huge amounts of overhead while the scanner tries to unpack and scan the files inside.

              • 4. Re: VS8.7 on File server bogged down by Sharepoint crawling.

                I strongly advice against turning off scanning on READ since it leaves the system memory vulnerable to a wide range of malware attaching memory, such as Conficker.

                Scanning for archives is disabled by default in VSE configuration and should stay off as it introduces performance issues. Potential threats will be scanned when those archives are extracted.

                 

                You can not exclude access from remote process from OAS, but you can exclude the SHARES with granular exclusion such as

                D:\SHARES\somefolder*\*.xls

                D:\SHARES\somefolder*\*.ppt

                ...etc

                 

                There is not point scanning these files if they are encrypted, as the scan engine is not able to access them.

                Preforming daily/weekly scans on the location excluded ensure there is no dormant infection waiting to be unleashed.

                 

                Good luck!

                Redouane

                • 5. Re: VS8.7 on File server bogged down by Sharepoint crawling.
                  rmetzger

                  rmajdki wrote:

                   

                  I strongly advice against turning off scanning on READ since it leaves the system memory vulnerable to a wide range of malware attaching memory, such as Conficker.


                  Redouane

                  Redouane,

                   

                  While I would agree that leaving security absolutely tight is recommended in some cases (military grade, etc.), I would disagree that turning off scanning when Reading from DISK has any large value against malware attacking MEMORY. Or so my experience with Conficker has been to date. I suppose scanning RAM and cache should be addressed separately from Reading from disk.

                   

                  The benefits of scanning when reading from disk seems to occur only when a large time frame happens between regular scans that include 'all' files on the system. (The benefit to scanning when reading from disk is to catch files written to disk under an older signature file that has yet to include detection, that may get caught more recently and before the scheduled read scan. By this time the infected FILES have been Written to disk. At this time, neither read or write would have caught the infection since the signature (and possibly Artemis) would not yet include the infection in it's database.) If this gap time is too large, infected files written to disk prior to the scheduled scan can get by. Since McAfee signatures typically occur daily, a scan, post signature update (daily), would handle this exposure. Now, the gap time has been limited to hours which should catch most zero-day exposures that are detected within a new signature update.

                   

                  Normal (average) server activty usually includes from 4 to as much as 8 reads to every 1 write to disk (typically). This is constant activity. Add to that a scan for every single read and for a busy server, the impact is pretty large. Now add this level of impact when Indexing software like MOSS 2007 is running and the impact can be debilitating. All for a short delay in detection that can happen during non-peak (production) hours. (This assumes regular, scheduled scans of the system.)

                   

                  In my experience, Tight Security and Good Performance are a balancing act, requiring compromises based on individual (company and environmental) needs. A balance point must be made and all involved must be aware of the risks and trade-offs of each choice. Good security is a process, not a result.

                   

                  wolfman099 seems to be talking to his/her IT Security team and the level of exposure is theirs to decide. Good security processes would review this exposure from time to time.

                   

                  One thing we do not know, and probably should not know on this forum, is whether this server in question is Internet facing or protected behind other layers of security. This too impacts these decisions.

                   

                  Now, if you would like to take this discussion off-line and respectfully educate me on why I am wrong I am eager and would enjoy hearing from you. Please send me a private message with instructions on how to continue this discussion. I really would like to know more about what you have said.

                   

                  Respectfully,

                  Ron Metzger

                  • 6. Re: VS8.7 on File server bogged down by Sharepoint crawling.
                    wwarren

                    This is of benefit to all forum readers -

                     

                    A configuration where you only Scan on Write is vulnerable, not only because of the obvious security trade-off by not scanning on read, and notwithstanding the various counter-measures one can take to minimize the risk of a scan-on-write only configuration -

                    It is vulnerable because there is a situation where the file can be written to disk AND executed BEFORE it has been scanned.

                     

                    When a file is written to disk and closed, this triggers VirusScan to scan the file when scan-on-write is enabled.

                    If for some reason we cannot get access to the file, since we are trying to scan the file object from a separate process and context to the original file-write+close, thus if we cannot access it at that time we implement what we call a Delayed Scan. The scanner will retry scanning after a short period of time.

                    If scan on read is not enabled, and the file were launched during this delayed scan window, you just potentially ran some malware.

                     

                    There are other scenarios too that customers run into very often, not because of any neglect necessarily, it's just how life goes in the busy corporate world - Servers fall behind in their DAT-signature updates and nobody catches it until too late. All it takes is one user to access an infected file remotely to begin an outbreak.

                    Even with updated DATs the server does not scan it because scan-on-read is disabled. Relying on ODS assumes the ODS can scan the entire filesystem in its allotted time.

                    A user who accesss a remote infected file does not trigger a scan from their own local AV software because it would require having "On Network Drives" enabled AND "Scan on Read", the former is not enabled by default (nor recommended). Actually, this is often a temporary configuration implemented by folks who are dealing with an outbreak, until their servers have been cleaned and configurations set appropriately.

                     

                    If we stick to ideal-world scenarios, we cannot ignore the risk inherent in Scan-on-Write only configurations due to imposed delayed scans. The only counter here is to enable Scan-on-Read. There are numerous configuration tweaks capable in the On-access scanner to maintain a scan-on-read configuration without performance impact. I admit there are scenarios too where the overhead is too much... but those are few and can be handled as special cases for risk mitigation.