cancel
Showing results for 
Search instead for 
Did you mean: 
psolinski
Level 10
Report Inappropriate Content
Message 1 of 14

Serve performance degradation after installation of P4

We recently experienced many performance issues with our IT infrastructure:

-NAS NetApp load raised 3x above average

-file copy operation became much slower - both between servers and workstation<>server

-Citrix logons were couple of times slower than before, user drives fail to map and printers to connect in Citrix

As all of it started around the same time we updated out VS88P2 to Patch 4, it was the main suspect, anyway will virtually all systems already on P4 and no easy return to P2 (we cant easily install VS88P2 due to another McAfee issue - expired certificate in VS88p2 installer...) we tested influence of VS just stopping its services. There was no improvment while VS is not running so we were investigating other possible issues - like network (there was a network change the same time as P4 deployment).

Two days ago our serve team discovered that servers with VS completly removed performance is back to normal.

Knowing this I just tested write speed between 2 windows2k8r2 servers without and with VS88p4 installed:

in MB/min            out MB/min      VS state

12,249.90             22,883.30            none

12,091.90             24,038.46            none

24,650.78             15,764.58            none

24,350.65             15,143.87            none

16,025.64             21,849.96            none

It's between 2 virtual machines on production vmware so speed fluctuates, but is between 24-15MB/min

With VS88p4 installed on one server - the one I was writing to:

19,230.77             5,128.21              VSE88p4 installed

21,367.52             5,494.51              VSE88p4 installed

24,038.46             5,898.55              VS services stopped

So write speed went down to 25% of normal speed!!!!! Even with VS services stopped.

So what can do this mess when VS services are not running? Filter drivers......

Disabled mfe*.sys and performance went back to normal:

20,215.63             25,273.80            services stopped filter drivers renamed

So decided to enable them one by one:

20,661.16             25,974.03            mfewfpk.sys enabled

20,449.90             16,713.09            mfewfpk.sys enabled

20,449.90             26,338.89            mfewfpk.sys enabled

21,367.52             6,183.02              mfehidk.sys enabled

20,242.91             5,933.54              mfehidk.sys enabled

22,624.43             4,610.42              mfehidk.sys enabled

18,315.02             25,641.03            mfehidk.sys disabled all other enabled

18,668.33             25,641.03            mfehidk.sys disabled all other enabled

20,242.91             23,166.02            mfehidk.sys disabled in reg all other enabled

It clearly shows that mfehidk.sys kills your system performance.

The same happens on vm and on phisical machines fo it's not specific to VM.

SR 4-5606236293 opened with McAfee.

It's still not clear if/how P4 affects our Citrix and NetApp environment - we are curentlly reverting what we easily can to P2.

Some other test by our Server Team:

Please see below speeds achieved with different patch level of VS

OS

no VS

P2

          -  

P4

P4
On-Access Scanner disabled

P4
All McAfee services stopped

Window 2008 R2

Writing

MB/s

    264.55

    240.62

NA

     87.67

            77.27

            90.12

Window 2008 R2

Reading

MB/s

    561.80

    500.00

NA

    533.62

          438.60

          533.05

Window 2008 R2

Writing

%

100%

91%

33%

29%

34%

Window 2008 R2

Reading

%

100%

89%

95%

78%

95%

OS

no VS

P3

P3 On-Access Scanner disabled

P4

P4
On-Access Scanner disabled

P4
All McAfee services stopped

Windows 2012 R2

Writing

MB/s

    390.32

    110.72

    110.72

    126.97

          132.24

          119.85

Windows 2012 R2

Reading

MB/s

    726.74

    258.13

    248.02

    603.86

          542.30

          463.82

Windows 2012 R2

Writing

%

100%

28%

28%

33%

34%

31%

Windows 2012 R2

Reading

%

100%

36%

34%

83%

75%

64%

So if you experience any strange issues after P4 - it's probably P4. Dont upgrade if you still on P2!

-----------------------------------------------------

Googling for mfehidk.sys I came across:

"Obviously MFEHIDK.SYS IS Related to mfehidk.sys VirusScan Enterprise from McAfee Agent,

IMHO it is one of the three most problematic malware apps for win 7 & win 8 in terms of stabiity and crashing,

I would replace it with almost anything else."

13 Replies

Re: Serve performance degradation after installation of P4

Nearly every type of Anti Virus = Performance degration.

You mention Citrix, have you looked at their Best Practices for AV exclusions?

Are you excluding anything from being scanned on read/write or both?

Some simple exclusions on file extensions or even folders can improve the performance you expecrience.

Message was edited by: jesperdb on 4/9/14 7:48:43 AM CDT
psolinski
Level 10
Report Inappropriate Content
Message 3 of 14

Re: Serve performance degradation after installation of P4

Nearly every type of Anti Virus = Performance degration.\

I wouldnt say that 75% of performance degradation is anything close to acceptable.....

You mention Citrix, have you looked at their Best Practices for AV exclusions?

Are you excluding anything from being scanned on read/write or both?

Some simple exclusions on file extensions or even folders can improve the performance you expecrience.

We have long list of exclusions - for both folders, file types and for processes. Much more than recomended by McAfee "if you have more exclusions than your fingers it is wrong".

And with P2 and the same list of exclusions "McAfee tax" was just 10%.

djjava9
Level 11
Report Inappropriate Content
Message 4 of 14

Re: Serve performance degradation after installation of P4

I have dozens of large customers that upgraded to p4 with no issues.....as mentioned by others take a close look at your policy and check exclusions.....also confirm that you are not scanning inside zip files/archives.

psolinski
Level 10
Report Inappropriate Content
Message 5 of 14

Re: Serve performance degradation after installation of P4

Obviouslly we dont scan archives. And the file we use to test is not compressed - just generated by genfile:

"The default data pattern for filling the generated file consists of first 256 letters of ASCII code, repeated enough times to fill the entire file."

500MB

As I mentioned to kill our server we dont even have to start VS services - it's enought to load  mfehidk.sys 

This clearly means that it doesnt depend on exclusions - nothing should be scanned by OAS shen VS is not running, so nothing to exclude.

Ask your customers to test SMB2.1 file write operations to server with and without vs88p4, they might change their mind.

Or better dont.

Just finished another test: replaced mfehidk.sys version 15.1.0.656 (Patch 4) with version 15.0.0.515 (Patch 2) and performance is back to normal level.

Message was edited by: psolinski on 09/04/14 17:49:29 CEST

Re: Serve performance degradation after installation of P4

Would you please call support, open a case and get them a MER? That's the best way to make sure that we get all the data we need.

psolinski
Level 10
Report Inappropriate Content
Message 7 of 14

Re: Serve performance degradation after installation of P4

SR 4-5606236293 with Tier II now.

MER and procmon logs already with you.

Pmaquoi
Level 11
Report Inappropriate Content
Message 8 of 14

Re: Serve performance degradation after installation of P4

Could you post us the conclusion of your opened SR ? i would like to know the McAfee response to this issue before eventually tryning a few upgrade myself on some servers. Thanks

psolinski
Level 10
Report Inappropriate Content
Message 9 of 14

Re: Serve performance degradation after installation of P4

Pmaquoi wrote:

Could you post us the conclusion of your opened SR ? i would like to know the McAfee response to this issue before eventually tryning a few upgrade myself on some servers. Thanks

Still with Tier II, we provided ETL trace logs, still  no conclusion.....

We are preparing to rollback to P2, our Citrix servers are back on P2 already as with P4 they became completly unusable...

Re: Serve performance degradation after installation of P4

"mfehidk.sys            Host Intrusion Detection Link Driver. This component is used for Access Protection and by the Filter Driver and Entercept (Buffer Overflow) Driver. Altitude:321300.00"

Have you looked in to Access Protection logs to see if anything could trigger the performance decrease there?

maybe even take a look at this from another post:

"

Hello Everybody!

I usually use McAfee Profiler in this cases to know what files n process VSE scans.

https://kb.mcafee.com/corporate/index?page=content&id=KB69683

With this toll I Know how many times i have I/O of the process.

If the Process who takes long scans and this process is a valid process maybe can be valid  Exclude this process of VSE

I hope I have helped!"

Message was edited by: jesperdb on 4/9/14 11:53:39 AM CDT