KnowBe4 - False Positives
Posted by mrhoopers@reddit | sysadmin | View on Reddit | 63 comments
When we run a campaign we get:
-
If you forward a phish you get tagged for training
-
if you report phish (abnormal) you get flagged for training
-
if you peek at the URL on your phone you get flagged for training
Does anyone have some pro tips to help reduce/elimitate these? We've talked with our KB4 people which really don't have any better answers.
I thought, maybe, there would be some way in the logs to see what happened then to send out training as a batched solution rather than automated.
fahque@reddit
I'm pretty sure this is a setting, probably on the mail server. We fixed this years ago so I don't remember the specifics but I do remember knowbe4 support gave me the answer.
Jipijapa00@reddit
I experienced this from 8-28-24 to 9-4-24. Barracuda Sentinel was scanning Phishing Training emails and knowbe4
Alarmed_Stable5887@reddit
https://support.knowbe4.com/hc/en-us/articles/115004326408-Bypass-Safe-Link-and-Safe-Attachments-in-Microsoft-Defender-for-Office-365?auth_token=eyJhbGciOiJIUzI1NiJ9.eyJhY2NvdW50X2lkIjo0NTMzNTgsInVzZXJfaWQiOjQwNDgxNjgyOTA1NCwidGlja2V0X2lkIjoxMDE3NzcxLCJkZWZsZWN0aW9uX2lkIjozMjg3ODYzMDcyMDE0NywiYXJ0aWNsZXMiOlsxMTUwMDQzMjY0MDgsMjMxNTA5ODY4LDIzNTcwOTI0N10sInRva2VuIjpudWxsLCJleHAiOjE3Mjc1NTc0Njl9.dregrFbbc4t3RXRjKg8ADdaJcOKVRrNL2kNEAmzLMcs
Happy_Kale888@reddit
PDF cause issues for us lots of false positives from therm.
nightlyear@reddit
I had this problem with InfoSec. In 365 I had to set specific transport rules to stop the false flagging.
beepxyl@reddit
Garbage-tier software sold to people that don't know any better. You have been phished.
iama_bad_person@reddit
This shouldn't happen. I should be marked as an Open, but not a Click (which I assume is what you mean when you say they get marked for training). Make sure there are no automatic processes which visit the URL, we have an exception in our Exchange backend which makes sure the URLs are not scanned or anonymized by Microsoft to prevent this.
Reporting Phishing using Outlooks built in Report Phish button means Microsoft's bots will click the link. KnowB4 usually does not tag this as a click as it has a list of Microsoft IPs to ignore, but some might slip through the cracks. We added KnowB4's phishing button to Outlook and removed the option to report Phishing to Microsoft entirely, and train users to click the new Phishing button. Been 6 months and click through rates are down to 3%.
Peeking the URL is the same as clicking it, as the mobile device is still going to the web page and getting information, no matter if it is a peek preview or not. KnowB4 does not (and technically, can not) see the different between a click and a peek as they both look like user activity. Just don't peek links.
mrhoopers@reddit (OP)
Yeah, open vs clicks. If you click I own you. But I don't care if you open. You can tell some things based on the subject but that's a pretty sophisticated flex for some of the users. They tend to be a lot smarter on the open but not clicking. We try to run a drive-by campaign, a click through and a credential each once a year then double up on one of them.
You helped, thank you...
I have a new person doing our phishing and I'll bet dollars to donuts she's not getting the urls whitelisted. Wow, great call. Thank you! I knew this but had directed her to do that whitelisting.I'm sure that's part of the issue.
We have several layers of email security. They use abnormal as it, historically, been really good. I will need to check if they can whitelist as well.
I suspected as much. Makes me wonder if I can run a clean.up and at least identify them.
BananaSacks@reddit
Random Q - Do you have a self-service email quarantine/release service? If so, who do you use, and would you recommend? If so, what's your pros/cons?
mrhoopers@reddit (OP)
Fabulous question actually. We are implementing one now but I don't remember the name. It was quite the selection process. Honestly, it came down to click count for the users.
BananaSacks@reddit
I'll buy you a digital pizza if you can find out the name and a literal pizza if it turns out to be worth a damn. This is one of the lowest priority, but & yet banes of my existence :/
joshadm@reddit
my previous org used Proofpoint for this and it worked good. Current org doesn't use one and instead just calls the Cyber on-call instead
TechFinAdviser@reddit
We use Proofpoint too, and it seems to work well.
Ilikeyoubignose@reddit
Assuming you mean EXO here, how do you add the exceptions?
iama_bad_person@reddit
We have four, which I have checked into an Imgur album so I don't clutter the thread
https://imgur.com/a/hJgZxQC
They do what they say on the box. Skip Clitter, skip Junk, skip ATP scanning of both attachments and links.
Ilikeyoubignose@reddit
Thank you sir. Your_a_good_person.
ApricotPenguin@reddit
Question for both you u/iama_bad_person and u/mrhoopers - What do you mean by peek at links?
Is that doing a long press on a hyperlink to see the URL in it? or did you mean some kind of Preview Link / Preview Page functionality, similar to what's described here.
Jolly-Explanation188@reddit
The issue is that currently on iOS a long press to view a hyperlink URL will also start loading the page as the ‘peek’ above the contextual menu.
draeath@reddit
Like some sort of speculative preloading?
While they can alert a real phisher that they have a live addresses, I wouldn't consider this a "user did a bad" at the same time as far as training goes. They're trying to do the right thing (inspecting the URL) and their devices is trying to do consumer bullishit that they probably didn't ask for.
iama_bad_person@reddit
I agree, but from a technical standpoint there is exactly 0 a website can do to tell the difference between a load in this fashion and a normal one, so there is no way to NOT assign training.
ApricotPenguin@reddit
Ooooh. That's definitely not expected behaviour [for me], and that's effectively loading the page anyways.
Thanks for the context!
BrackusObramus@reddit
lol peeking links is a way to check if the link looks legit before clicking. What a great anti phishing training................. not!
Chareon@reddit
Doesn't peeking/previewing links literally just load the page in a window? I don't see how that is any more secure than just opening the link.
vagueAF_@reddit
I thank god the security team manages knowbe4, so o can ignore it.. and when my manager bug me that he's getting emails that I haven't done my knowbe4 'training' I ignore it some more.
I'm a sys admin, I don't need to do this training.(Which is watching cringe videos & questions from knowbe4)
Entegy@reddit
Forwarding should count as a failure. You shouldn't forward suspicious emails.
If you're using Microsoft's report message button, no amount of whitelisting will save you. Its scanning will always trigger links. Replace this button with KnowBe4's Phish Alert button to fix this issue.
As for peeking the link on mobile well... It's loading the link. That's a click. There's no fix here except turning off the long-press loads a preview feature if the app in question has such a feature. The Outlook app does. Don't know about Apple Mail or Gmail mobile apps.
Ilikeyoubignose@reddit
Do you know if you can switch off the report option in the Outlook mobile apps?
Entegy@reddit
It's controlled by the same setting for desktop.
Ilikeyoubignose@reddit
Thx.
mrhoopers@reddit (OP)
I 1 quintillion percent agree that forwarding a phish is bad. I tried to enforce that but, for a minute, our report phish button wasn't working so we had to drop back to old school reporting. If I try to go back to not forwarding I'll get a riot from my users. So, for another minute, I have to go with the forwarding. I hate it.
I want to use KB4's but...also, not my choice.
And I didn't think there was anything we could do about peeking but wanted to ask in case there was a signature I could look for.
Honestly, opening the email isn't the problem, it's the clicking and acting on the emails that's the problem. IMHO. I don't "like" that someone opens them but just opening and going, PHISH, and deleting, is far better than clicking every damn link they're sent.
Entegy@reddit
If it's not your choice, then you should tell the above to someone whose choice it is.
Point 1 is retraining but point 2 and 3 are technical limitations there are solutions for, but if they're going to refuse the solution well. 🤷🏼♂️
mrhoopers@reddit (OP)
You are where I landed. Hey, it hurts if we do this. Don't do that? No, we like doing this.
Me: O.o -- Okay?
Entegy@reddit
And then you get blamed when things still don't work! Good luck and get everything in writing!
richie65@reddit
Other than the monthly assignments - A user would have to click on a link, or open the attachment in one of the test emails - Before they get assigned additional / remedial assignments.
Additionally, I have a few powershell scripts running that end up disabling the users AD account if that failed to complete any training assigned to them...
One script uses API to KnowBe4 to find anyone who did not do their training, and sends that to a txt file.
Another script, later in the morning, reads that text file, and disables the users AD account...
This locks them out of everything, including KnowBe4 (because SSO)
The end result is that we can inform our insurance company with full certainty that 100% of our users who have access to company resources are complying with the IT security training that the insurance requires...
Because if they are out of compliance - the don't have access.
When their supervisor request their account be unlocked - They are given until 6am the next day to complete all assignments - Or they get locked out again...
We intentionally make it a pain in the ass for the department, and the user.
Of the about 200 employees in North America - We now see three or four users getting locked out on the first of each month - Before the above was put in place... 20-30 users who ignored the KnowBe4 emails etc...
Our insurance costs have decreased as a result of the 100% compliance.
One_Ljfe@reddit
For what it’s worth… Phish Alert Button is FREE, and it’s the only integration to report phishing emails in Shared Mailboxes that we’ve found, so that’s two huge pluses.
You can argue that you want people to report versus forward since forwarding passes a threat onto another person versus being sorted by IT to vet out, also, it works for all reported emails, fake or real.
We setup an Inbox Rule to sort the Phish Alert emails into a folder in a shared mailbox to manage. Kind of a pain at times since we run thin, but it’s been very helpful in funneling those to place… minus Microsoft’s Quarantine that’s picked up some emails, so we’re working on cleaning that up better.
Hope that helps a little. Good luck!!
hbk2369@reddit
Tip #1: don't make your phishing exercise feel like it's punitive.
mrhoopers@reddit (OP)
The only thing that goes that way is if they don’t at least click the link for the 3min video. Seriously.
hbk2369@reddit
Where I work they’d complain
mrhoopers@reddit (OP)
They complain, but our CISO can escalate which solves it from the top level down. I'm blessed to have senior leadership support from the top all the way down.
captkrahs@reddit
There aren’t settings for these?
DualPrsn@reddit
We had similar problems. Knowbe4 recently published an updated whitelist document. If you have not recently updated your whitelist you should checkit out. If you are using the native outlook report message plug-in, this is a known issue with Knowbe4. Knowbe4 has its own plugin.
mrhoopers@reddit (OP)
I did not know they updated their whitelist! That's important! Thank you!
We actually CC our mail to multiple solutions so, maybe, I can add phisher as well? I don't know. Thanks, that whitelist update is a great tip.
DualPrsn@reddit
Your welcome. When I did my latest phishing campaign it was tagging everyone as having failed. All they did was open the email.
dunnage1@reddit
Yeah I chewed out the ISSO on click vs open. So no, I won't be playing this game because I'm still heated from that conversation.
egbur@reddit
If the header of the message contains the words 'X-PHISHTEST', delete the message and stop processing more rules on this message.
fresh-dork@reddit
If the header of the message contains the words 'X-PHISHTEST', apply the phish theme to the message and stop processing more rules on this message.
mrhoopers@reddit (OP)
I think this is going to need some additional rules to only allow phish from KB4 for this to work. Otherwise a bad guy could flag themselves with actual payloads and have the report go unnoticed. But…I love the options offered.
egbur@reddit
Sure. I guess I wasn't specifically answering your question. I just find their campaign emails annoying so I setup a mailbox rule like that one everywhere I go :)
mrhoopers@reddit (OP)
Hahahahah. Yeah, shhh, don’t tell anyone else!!
mrhoopers@reddit (OP)
I’d not gotten around to inspecting the headers yet. This is fantastic! Thank you!!
EnableNTLMv2@reddit
I use an email header filter to grab KB4 messages into a separate folder.
ranhalt@reddit
Boils down to you using the product wrong. You can either have Abnormal ignore all submissions of simulations (if possible ) or use PhishER for reporting suspicious emails so KB4 ignores simulations (unless they report it twice).
mrhoopers@reddit (OP)
This is why I asked. I knew I was overlooking something. Thanks!
falter@reddit
1 is standard practice, at least this is what they told us
mrhoopers@reddit (OP)
Back in the day forwarding was how you were supposed to do it. Then the button. But then we had to go a half step back for a minute because the button didn't work. It works now but telling people to not forward is going to be met with resistance.
I take what I can get sometimes.
falter@reddit
Button works well for us, but old habits die hard!
Brufar_308@reddit
I got a test phish from knowbe4. I identified it as a phish and usually I just delete them, but since it had an attachment I uploaded the attachment to virustotal. Virus total opened the attachment to scan it and I got flagged for phishing training. The other IT people and auditor got a chuckle out of that when they received the notice I got phished.
I deleted myself from the phishing training campaign. Some time goes by and. A notice goes out that my training is overdue. I go back I see I am back in the phish training so I marked training as completed and moved on, since removing myself didn’t work. Several weeks later a notice goes out that I haven’t done my training. Everyone is rolling now as I keep getting popped for being phished.
I open a case with knowbe4 to see why this is happening and why marking complete or deleting myself from campaign isn’t working. Support tells me that when that happens I am also added to the phished group and if I am a member of that group and someone modifies something (I’m not clear what) anyone that is a member of that group get re-enrolled into phishing training.
Well at least I could provide some amusement to my coworkers I guess…
And yea that lack of link preview on phones is a pita. Nabs more users than anything else.
redyellowblue5031@reddit
Did you follow the whitelisting guides they have?
It's hard to know exactly what you need based on your setup but you'll need to bypass it in your external mail filter (if you have one), Exchange/M365, and it also helps to enable DKIM message signing to prevent DMARC enforcement.
mrhoopers@reddit (OP)
Yeah, when I set it up it was mostly working. I hired a new person and told her what to do. I just assumed she was doing it but I think she's forgotten or skipping steps. Your DKIM signing is brilliant. That's something I know we didn't look at. Thank you!!!!
redyellowblue5031@reddit
No problem! It doesn’t hurt to double check everything. Exchange and mail filters will often sandbox links if it’s not fully bypassed and they easily become false positives.
You can download the failures in CSV and see if the failures match your expected public IP. That can give some insight into how the “failures” are happening.
After we got it setup right our failure rate dropped significantly and only our public IP (except mobile failures) shows up in the reports.
mcjon3z@reddit
Not sure if you can do with KnowBe4, but we had to whitelist mimecast and Microsoft IP ranges in phishing box to keep them from falsely reporting clicks due to sandbox analysis of the URLs in the emails.
mrhoopers@reddit (OP)
Yeah, my new person isn't the most technical so this doesn't always make sense. I need to circle back and make 100% sure we're whitelisting right.
It's surprising how technical KB4 is when you look at it through the eyes of a non technical person.
Reverend_Russo@reddit
What email service are you using?
Number 3 will always get flagged, that’s a literal click.
Have you looked at their KBs on white listing?
mrhoopers@reddit (OP)
Microsoft. I think it's a whitelisting issue based on the feedback from another post. You've got it.