Frustration with Defender for Office 365. High Confidence Phishing.

Posted by nbritton5791@reddit | sysadmin | View on Reddit | 26 comments

Running into an issue where Microsoft's algorithms are consistently marking items from a couple of different vendor email addresses (two different domains) as High Confidence Phishing and sticking the items into Quarantine.

The email items contain no links, phishing attempts, or suspicious information. Attached are simple PDF's and HTML files with no dangerous content, and zero links of any sort.

Issue has been occurring for a little over a week at this point.

We have tried mail flow (transport) rules, whitelists in every panel we can think of, but it appears that Microsoft really does just prevent these mail items from being delivered. Link below basically tells you all of their controls no longer apply when an item is flagged as such.

Secure by default in Office 365 - Microsoft Defender for Office 365 | Microsoft Learn

We have been submitting these items (several hundred of them now) to Microsoft for false positive (and checking the box to allow items like these in the future) yet they continue to get flagged.

Does anyone have experience with this and have a clever solution to get these to deliver to a user inbox automatically?

[-]

Borgquite@reddit

Been there. Your only options are to report to support, and keep doing admin submissions, until they listen.

[-]

Rex_Bossman@reddit

Maybe you've tried this but this really should work:

Create a transport rule that says if senders domain is: then modify the message properties to set the spam confidence level to -1. Then set the priority to 0 and stop processing more rules. Make sure it's enabled.

[-]

nbritton5791@reddit (OP)

Thanks, we have tried that but it does not work. It's noted in the Microsoft documentation in my post as well that high confidence phishing over rides mail transport rules (insane!).

[-]

Frothyleet@reddit

There's a good reason for it - the old "well we can't figure out our spam filter's reasoning, so let's just whitelist shit" model is a big vulnerability as soon as anyone spoofs those domains, or their email is compromised.

[-]

dragery@reddit

We use Proofpoint, and don't want Microsoft touching email email sent inbound through it. We whitelist inbound mail from that connector.

The problem here is Microsoft continually finds new features to automatically turn on to further "protect" us and it ends up being a hunt to see where and how they screwed it up again.

[-]

nbritton5791@reddit (OP)

That's not really a good enough reason to pull away control from IT in such a manner.

[-]

AppIdentityGuy@reddit

If you release the email from quarantine does it get delivered?

[-]

Frothyleet@reddit

It's what you're paying for. If you don't like the functionality, you can use a third party email security solution. But, honestly, they're not wrong (as long as their product is working properly, which unfortunately might be the case for you).

[-]

Rex_Bossman@reddit

You're not wrong but I will say sometimes trying to figure out MS spam filter reasoning is like trying to figure out why exactly a crazy person is crazy. The domains I've whitelisted are also owned by my company and managed by myself so while I do understand the security risk it's an acceptable risk for me.

[-]

Frothyleet@reddit

Oh, for sure. All of the advanced spam filters these days are very opaque because they are all heuristic, based on machine learning (oh sorry I mean it's AI, we have to call it AI now).

It's very much like modern EDR vs the old signature models of A/V.

[-]

Rex_Bossman@reddit

That is crazy, I've had those rules set in my tenant for years and they've always worked. But it's Microsoft so who the hell knows. Sorry, hope you can get it worked out!

[-]

patmorgan235@reddit

What does it say for the "detection technology" and "primary override source"

[-]

nbritton5791@reddit (OP)

Detection Technology is blank "-"

Primary Override: Source is listed as "none"

[-]

Frothyleet@reddit

What is MS support saying? They suck, but, they may be the only ones who can answer your question.

[-]

nbritton5791@reddit (OP)

We haven't been able to raise a ticket. Four days into the madness. We go through Pax8 and they have been unhelpful in escalating in a timely manner.

[-]

releak@reddit

Had this happen once and also did not know how to fix it. Customer had some links in the signature in this case that was getting flagged, and I asked the customer to remove them, or I could open a case with Microsoft.

Customer was cheap and breakfix, so knowing this, I communicated upfront that any time with Microsoft support would be billable.

Customer said no no no.. You recommended me this Microsoft stuff, so you fix it for free.

I told my manager I didnt wanna work with cheap customers like this, and he said sure thing just close the case so I did and never heard back from the customer

[-]

Kuipyr@reddit

You have to allowlist high confidence marked addresses via submission. You can’t directly add it to the TABL.

[-]

littleko@reddit

Check the vendor's auth first. HCP flags are almost always correlated with failing DMARC or broken DKIM alignment on the sender side, not the content.

If they're sending through a relay that isn't in their SPF or the DKIM signature is misaligned, Defender weighs that heavily regardless of content. Run one of their messages through a header analyzer and see what actually passes.

If auth is clean and it's still happening, the Tenant Allow/Block List at the URL/sender level is the only thing that overrides secure-by-default, transport rules won't do it.

[-]

nbritton5791@reddit (OP)

Thanks,

It's technically two different vendors and one is sending with broken DKIM and failing DMARC, but the second has full SPF/DKIM/DMARC pass/alignment.

Auth being clean on the second vendor and still being unable to impact the verdict ourselves as an MSP is maddening and negligent on Microsoft's end.

Very frustrating.

[-]

littleko@reddit

Yeah HCP is the one verdict Microsoft won't let you override with ETRs or allow lists, you have to submit through the Submissions portal and let them retrain the model.

Painful but it's the only path that actually sticks for the clean vendor.

[-]

Cbeckstrand@reddit

Tenant Block/Accept will not whitelist anything flagged as high confidence phishing. We have run into the same issue as OP multiple times on clean messages that were already whitelisted.

[-]

littleko@reddit

You're right, my bad, HCP bypasses TABL. Only thing that actually works is getting the vendor to fix their auth or opening a Microsoft case for a false positive submission.

[-]

excitedsolutions@reddit

This is due to the MS stance that exchange online and its IPs are their property and they can take any action they deem to keep their reputation status safe. If you came from operating exchange on -prem this is a hard reality to come to grips with - as situations like this make it clear you are not in control of everything anymore.

The only solutions available are to use an edge service as a mail filter that you do have control over, or continually submitting the emails to MS as incorrectly classified and hope someone eventually puts a rule in place to adjust that classification. They keep it very “black box” with respect to what and how they do any of this on purpose.

[-]