What exactly are you trying to prevent users from doing? If you're trying to prevent them from executing apps, VSE isn't going to do that. By excluding files in On-Access scanning, you are telling the scanner to not look at them when they are being read / written to disk. VSE does not let you make hash based exclusions. I don't think though that excluding files is actually what you are looking for.
If you're trying to prevent users from running applications then you'd be best to look at something like McAfee Application Control which will ensure that only whitelisted applications are allowed to execute and make changes. If you want to search your environment for files based on hash, in real time, you can look at McAfee Active Response. Active Response can also be used to based on the results of a search execute defined actions that could do things like remove the file, or anything that an administrator writes the appropriate script / action for.