In my environment, I have a setup as below, IPS sensor is placed in the middle of switch and firewall to inspect the traffic and forward suspicious file from user segment and email server towards ATD.
However, due to my IPS is in front of spam filter, there is one week that there are lots of SPAM Bot from internet that is attempt to send huge amount of spam with attachments to our email server.
Since those SPAM contains attachment (such as zip file, flash file, VB scripts etc), IPS forwards all of it towards ATD for analysis and it caused a huge amount of analysis delayed (around 5k pendings jobs in 1 week)
Hence, I would like to seek advise to see if how you guys usually handle such situation on either IPS or ATD, by dropped/block the SMTP file before sending to ATD to save it resources from unnecessary analysis on SPAM file. I only can figure of changing the network diagram by inspecting the traffic from proof point towards email servers, but that will be my last attempt.
Thank you and hope to get some advises from you all.
You could potentially create a connection limiting policy at the IPS based on GTI reputation, but that would affect all traffic, not only the SMTP traffic going to the mail server.
Otherwise, if you have the option, you could ignore the SMTP traffic at the current inspection point, and then add another IPS inspection point after the spam filter - this should prevent most of the malicious email to be scanned by the IPS.