6 Replies Latest reply on Apr 11, 2014 2:57 PM by Peter M

    Real-time scan blocks sqlite database operations

      I work on a FOSS web service that can use sqlite3 as its back-end database.  As I understand it, sqlite3 opens its database file for exclusive access and then closes it on each operation (this avoids the need for a persistent DBMS process).  Also as I understand it, real-time scan reads the file after every operation.  I'm doing development work on WinXP, where I recently upgraded from AntiVirus Plus to Total Protection, and now am getting errors on sqlite3 operations:


      "OperationalError: unable to open database file"


      This doesn't happen on every access, and not on the same access on each run, but generally hits within the first dozen accesses when starting the application for the first time, when it creates all its tables. So I'm assuming this is timing-dependent -- if accesses are happening frequently enough, real-time scan will still be reading the file when sqlite3 tries to open it.  Since sqlite3 opens for exclusive access, it fails.  I'm getting these errors while sqlite3 is creating empty tables on the very first run of the application, when the database file is small.  As the database has actual data added to it and grows in size, the errors would presumably become more likely.  (This is moot at the moment, since it's failing before it even manages to create all its tables...)


      If I turn off real-time scan, the errors don't happen.  I have not (yet) tried asking McAfee support if I can deinstall Total Protection and install AntiVirus Plus again, to see if there really is a difference, or if this is a red herring (e.g. maybe some mystery setting is different, or the version of real-time scan just happened to change across the upgrade).  (Note I did not change the real-time scan settings in either case -- both were default.  The database files are inside the My Documents directory.)


      This is not a new problem.  (A web search for "mcafee real-time scan sqlite unable to open database file" turns up lots of reports on other sites, going back at least *5 years*.)  Nor is it restricted to sqlite3 -- it's just the opening and closing on each operation that makes the interference more likely.  (Many installers, which may create lots of files rapidly, recommend turning off AV protection for this reason.)  But...some of those other reports say this only happens with McAfee real-time scan, not with other anti-virus products.


      I'm trying to find out several things:


      First, is there a material difference between real-time scan in AntiVirus Plus and Total Protection that could account for this?


      Second, I'd like to gather information to support reporting this to McAfee and lobbying for doing something about it.  So:  Has anyone else had a similar problem?  Was there also a change in going from AntiVirus Plus to Total Protection?


      Third, I'm wondering about workarounds in the application.  Changing the behavior of sqlite3 may not be an option -- it really does need to let go of the database file after each operation, and it does need exclusive access...at least, it needs exclusive write access, and exclusive access between sqlite3 threads.  It's arguably not important if some other process, like real-time scan, sees inconsistent contents because sqlite3 is writing the file at the time.  I don't know if Windows filesystems have an open mode for exclusive write, but that still allows reads.  Lacking that, I could have the application accessing the database retry if it gets this error, or, better, if it can tell it got the error because the file was open.  (If it got an error due to file permissions or incorrect access info or no such file, there's no reason to retry.)  If you're a software developer who's dealt with this, do you have a recommendation?


      (Apologies if asking multiple questions in one post interferes with rewards -- this is my first post here, and I didn't see a good way to split this up without heavy redundancy.)




      -- Pat


      Message was edited by: ptressel on 2/18/11 12:42:07 AM CST