- Do you know how VSE determines if a file should be scanned?
- Do you know how a *.doc file goes from a shortcut on your Desktop to being opened in Microsoft Word?
- Do you know how that understanding can be applied to VSE's scanners, to improve performance with minimal impact to security?
If you paused at any of the above, this blog is for you. It is what I'd call 201 level course content.
How VSE determines if a file should be scanned
In brief, file activity is processed by our file-system filter driver (a kernel driver), which creates scan requests for our scanner service (McShield.exe, a process). At these two locations, the filter and the scanner, decisions are made for whether the file being accessed is in need of scanning. You control that decision making process via the On Access Scanner configuration, which I'll now explain.
- avoiding scans when safe to do so, and when told to do so,
- making that decision as early as possible (at the filter).
Decisions the Filter can make
"What to scan"is a configuration screen in the On Access Scanner properties. You can choose "All Files", "Default Files" (with sub-options), or "Specified Files Only" (also with sub-options). Each of these has nuance to the behavior of the scanner.
All Files, means all files are scanned for all malware. This is the most secure configuration option.
Default Files, means files matching our Default Extension list are scanned for relevant malware. Notice the key differences in definition?
Default Files + Additional file types, means the same and the additional file types you add are scanned for all malware, making this option more secure than the prior one.
Specified Files only, means the Specified file types you add are scanned for all malware. This is typically the least secure option, but if the file types you add match the Default Files extension list then this option is more secure than that one.
"All Files" carries the least risk, providing the most security, and is Intel Security's recommended and default setting.
The "what to scan" configuration is remembered by our file-system filter driver, which allows it to make decisions on whether a file operation we see should be blocked until a scan request is completed.
"When reading from disk" and "When writing to disk" are also configuration settings our file-system filter driver receives, because when it sees a file operation it knows if the file operation is a Read or Write.
NOTE: Intel Security recommends to always have the Read scan option enabled; you would only disable that option when accepting the security risks1 that implies, which is, any known or unknown malware has the potential to enter the environment and execute before having been scanned. Or in other words, you risk having an outbreak.
1 if used in conjunction with VSE's scanning profiles (described later) then the risk is minimal and tightly controlled.
"On Network drives" is required when you want the on-access scanner to scan file operations on remote systems, such as file shares or mapped drives. It is not a default-enabled setting because in a secure environment the remote system will have its own AV protection, and because this option incurs a heavy performance penalty. This setting is remembered by the file-system filter driver because it knows when a file transaction is occurring locally or via a network redirector.
"Opened for backup" is another filter driver setting, because the filter driver knows if the file operation being requested has the appropriate flag used by Backup software. Thus, if this option is not enabled, our filter can quickly decide to ignore further processing for the file.
"Scan cache" is not a setting you control but it is crucial to performance gains with the on-access scanner and needful to explain a little here. For each READ file operation that may result in a scan request, the filter checks the scan cache to determine if we have seen the file already and if the file was determined to be clean or excluded - in other words, we're trying to avoid a scan. The cache cannot be used for WRITE scans because a write means the file is being modified, so we know it will always differ from what may be in cache, therefore we purge that file from the cache if it existed. IMPORTANT: If it's not clear what that means I'll restate - A scan configuration where "When reading from disk" is not enabled gets no benefit whatsoever from the scan cache.
NOTE: There is unavoidable overhead from our file-system filter's presence in Windows when file operations are occurring, because to improve performance overall we calculate a hash for file objects we see - that allows cached data to be more reliable. But calculating that hash has a cost. This additional granularity was introduced with VSE 8.8.
NOTE: The scan cache loses almost2 all data when certain actions occur -
2 "almost" because some data we obtain about certain files does not change, and can therefore remain in cache.
Decisions McShield makes
McShield is where exclusions are processed. The filter does not know a file object is excluded until McShield says so, and thereafter the filter will remember that specific file object is excluded (for as long as that file's scan result remains in cache). Even if you configure to exclude a Folder, for every file inside that folder that gets accessed there will be an initial scan request sent to McShield because the Filter doesn't process exclusions. Relying on exclusions is a less effective way to avoid scanning (and reclaim performance) because the decision is being made by McShield, which is the latter part of the design for determining the file should be scanned (or not).
As you can see, we have moved much of the decision-making process into the filter driver, to provide us the earliest chance for avoiding a scan if it's unnecessary, based on configuration and file history.
How a *.doc file goes from a shortcut on your Desktop to being opened in Microsoft Word
First, a basic truth about Windows. "Programs", "Applications", and "Processes" are interchangeable terms in many contexts, but the latter one has more meaning and helps describe how Windows works. Let me explain by showing you -
- Open Task Manager. (Rt-click your Task Bar, and choose "Task Manager". Another way to open it is hold [ctrl]+[shift] then tap [esc])
- Notice the "Applications" tab. Go there. This shows your currently open applications, or active programs (those two terms are interchangeable). This is the high-level view of that software, the user-friendly name. NOTE: Under-the-hood, an application could be comprised of multiple processes and some of those processes may also be services (a term we might explain later).
- Rt-click one of those programs, (try Microsoft Outlook, if it's open) and choose "Go To Process"
Notice that Task Manager jumped you over to the "Processes" tab and it has highlighted the specific process name that is controlling the application you chose in Step 3. In other words, this is how Windows sees that program, via a Process.
Back on topic -
A *.doc file is usually associated with Microsoft Word (WinWord.exe is the process name). This association is stored in the Windows Registry - an internal database of sorts that Windows uses. When you're looking at the *.doc file on your desktop, Windows has already gone through the under-the-hood steps necessary to show you the WinWord icon for the *.doc file... and when I say "Windows" has already gone through those steps, I really mean Explorer.exe (a process) has gone through those steps. Explorer.exe is the process that shows you the Windows desktop, all "Explorer" windows, the Task Bar, and Start menu, and all those icons on your desktop. Point being, in Microsoft Windows it's always3 a process that is doing work and processes can tell other processes to do work too!
3 Please forgive the simplification; I feel it's not necessary to go into specifics of the "System" process, or process threads for the purpose of this blog
For example, when you double-click that *.doc file on the desktop, Explorer.exe opens the file to read some basic information from it, recognizes the file has a .DOC extension, discovers from the registry what it should do when someone double-clicks .DOC files, sees that it should Open them using WinWord.exe, and then launches WinWord.exe for you with the appropriate parameters to tell WinWord to open the *.doc file that you double-clicked! [breathe] Then WinWord.exe launches, and reads the *.doc file, opening it for you and displaying the contents. In this seemingly basic example of double-clicking a Word .doc, you can see at least 2 different processes were involved in opening a single file; some work was done in response to one process (Explorer.exe) telling another process (WinWord.exe) what to do. Also in this example, Explorer.exe can be considered the parent process to WinWord.exe because Explorer.exe is the entity that invoked WinWord.exe.
If you understand the concept of processes and the work they perform, then you're ready to comprehend VirusScan's advanced on-access scanner configuration capabilities: Having different scanning policies for High/Default/Low risk scanning profiles.
Improve Performance with Minimal Impact to Security
Let me begin with a mock scenario, and make it feel like a grade 10 math exam (apologies in advance for any trauma this awakens for anyone ) -
"File Finder X-stream", an application I just made up. It uses FileFinder.exe (a process) to scour your storage media (hard disks, USB sticks, DVD drive etc.) to find files. For each file it finds, it Reads the first 4kb of the file. I open a CMD prompt (cmd.exe) from the Start Menu, and navigate to where FileFinder.exe is located on disk. Then I launch FileFinder.exe simply by typing FileFinder.exe and pressing Enter.
Question 1: What kind of file operations will be performed by the process "FileFinder.exe"?
A. Read operations
B. Write operations
C. Finding operations
D. All of the above
Answers are hidden below. Mouse-highlight to view. (I tried to get fancy but the site doesn't allow it )
This question is intended to test reading comprehension; the answer can be found in the problem statement.
"A" is correct. 10 pts.
Question 2: If there are 1000 files on the local hard disk, 4 on a DVD, and no attached USB storage devices, how much data will FileFinder.exe write to removable media?
B. 4,001,600 bytes
D. 0 bytes.
This question is intended to see if you're paying attention to detail. It also stresses the importance of knowing whether file I/O is a READ or WRITE operation.
"D" is correct. 20 pts.
(We established earlier that the process FileFinder.exe only reads files, it does not write any data anywhere.)
Question 3: (Bonus) Which statement or statements are true about CMD.exe and FileFinder.exe? Check all that apply.
A. FileFinder.exe is a child process
B. CMD.exe is a parent process
C. CMD.exe is a child process
D. FileFinder.exe is a parent process
This question is intended to test your awareness of Windows processes and their parent-child relationships.
"A" = +10 pts. It is a child to CMD.exe.
"B" = +10 pts. It is the parent to FileFinder.exe.
"C" = +10 pts. It is a child to Explorer.exe because it was launched from the Start Menu (that Explorer.exe controls)
"D" = Incorrect. Lose ALL bonus pts.
[continuing from previous mock scenario] FileFinder.exe finishes its task after 1 minute. When VirusScan is installed, it takes FileFinder.exe 4 minutes to complete.
Why is it slower when VirusScan is installed?
You might assume it takes longer because the on-access scanner is scanning files accessed by the FileFinder.exe process. But, you don't have to assume, you can discover and confirm if that's what is happening. Here's how.
- Should you find performance is still poor, then it's not because of scanning. And if that's the case, you might need assistance with investigating/solving the issue because it would seem to involve our file-system filter driver code - not the scanner.
- Should you find performance is much better then your initial assumption was correct.
What can we do with the scanner configuration to improve performance? Examine what you know so far -
- What is the process name? FileFinder.exe
- What does the process do? It reads files, and it reads every file it finds on all storage media - and that's all it does.
The approaches to solving this are explained herewith. Disappointingly, the most commonly used approach is a terrible one and is a disaster just waiting to happen!
#1 Disable Scan On Read - in other words, don't scan anything this app does and don't scan any files READ by any other processes either. Truly, it's difficult to fathom how vast a gaping hole this creates in an environment. Never do this.
#2 Disable Scan On Read for the Low Risk profile and add FileFinder.exe to that profile - But you just said "never do this" for #1, so, which is it? A goal of this blog is to help folks realize it is very important to know the name of the process that is initiating the file activity. Because by knowing that process name, you can apply a different set of scanning criteria for the work done by that process instead of configuring something potentially bad for All processes. This technique is worlds different to #1 and a relatively safe option.
In other words, for FileFinder.exe, when we add it to the Low Risk profile and disable Scan On Read, we tell VirusScan Enterprise not to scan any files READ by FileFinder.exe... but to keep scanning files read by anybody else. We will also still scan FileFinder.exe itself when it gets Read/Opened/Executed, which means, if ever a piece of malware enters the system and it's named FileFinder.exe, we will still scan it. Please read #2 again, until this concept has sunk in.
#3 Exclude the target File or Folder - but in this scenario we know that FileFinder.exe reads all files on disk, so, creating an exclusion of "all files on disk" would be a really bad idea. That eliminates this as a potential solution.
#4 Exclude the target File or Folder within the Low Risk profile and add FileFinder.exe to that profile - It's a plausible option, more secure than #3 by a lot, but it isn't a good choice in this scenario because FileFinder.exe reads all files on disk - meaning, you'd have to exclude "all files on disk". The security risk of adding the exclusion is greatly reduced because it only applies to work done by FileFinder.exe (and any other processes you have in Low Risk profile), but if you review "How VSE determines if a file should be scanned" above, you'll see that performance gain will be minimal because McShield (the User mode scanner) still has to do all the work of processing the exclusion of each and every file that FileFinder.exe touches.
#5 Exclude the target File or Folder within the Low Risk profile for Reads Only and add FileFinder.exe to that profile - The same as #4 in terms of performance, but #5 is more secure.
There should be enough information here for one to cogitate on the functionality and conclude how to apply it to real-life scenarios. However, real world scenarios can be very complicated and/or difficult to track down; there could be:
- multiple processes involved
- unknown process behaviors to first identify and understand
- For example a sporadically occurring problem, hard to catch but common enough to annoy
- pressure from a demanding team/manager
- perhaps one whose patience has bottomed out wherein colorful language and metaphors are being thrown your way
- a 3rd party vendor is involved and that vendor has no clue what to do
- inconsistent results (which is never a good sign)
... to name a some of the complications. Whatever the case, if you get stuck or need guidance, our Support team will be able to help.
Watch the video! Wait, there's a video? Apparently https://kc.mcafee.com/content/tutorials/vse/Public_Hi-LoRiskProfiles.html
I made this video some time ago and nigh forgot all about! It's still useful even today.