I almost screwed up and let a hacker get away with credentials
Posted by j1mmyava1on@reddit | sysadmin | View on Reddit | 98 comments
I work in L1 Help Desk and last night this guy called in asking for a password reset because he was locked out of his laptop. He introduced himself with his name, employee ID, and home address. SOP for password resets done over phone is to send a 2FA code to their email or phone number but I completely fucked up and forgot to authenticate the user.
I reset the AD password without authenticating the user and then notified the guy over phone that I sent his temporary password to his email. He said he didn’t have access to his email so I said “okay I can send it over Teams”. He said he didn’t have access to Teams on his phone and then tried to coerce me in providing the password over phone. I told him that I couldn’t do that because it wasn’t SOP (I managed to remember that part) but he kept trying to push and guilt trip me.
I wanted to see what job position this guy had so I looked him up on Teams and saw that he was a VP. But what stood out to me was that it showed his status on Teams “In a meeting”, yet the guy over the phone said he didn’t have access to Teams. I pinged the guy on Teams and asked “Hey are you calling me from xxx-xxx-xxxx?” I get a reply back saying no and that he was presenting something to his coworkers. I immediately hung up with whoever called me over the phone and notified the network engineer who handled all cybersecurity incidents. I got into a call with several other people including my manager, head of IT, and the real end user himself, and explained everything. I found out from the real end user that his LinkedIn had been hacked a few years ago and that was probably how the attacker was able to provide his employee ID and address. During the meeting, my manager reiterated SOP but he and the head of IT complimented me for standing my ground and not causing a breach so I know the team has my back.
Long story short, I forgot to follow SOP and almost let an external attacker get away with credentials.
ilyas-inthe-cloud@reddit
Honestly, the good sign here is you stopped at the last step instead of reading the temp password out loud. That's exactly how these calls work. I'd treat it as a real incident though. Note the account, flag it internally, and ask your team to tighten the reset flow so the verification step can't be skipped when you're tired or getting pressured. A near miss is still useful if the process changes after.
admiralporkchop@reddit
Ok you almost learned a terrible lesson. ALWAYS OPEN THE SOP ON YOUR SCREEN AND FOLLOW IT STEP BY STEP.
never assume you remember, ever. I've seen people in your role get shit canned for this.
-King-K-Rool-@reddit
As a security officer, dont beat yourself up about missing the 2fa, ya you slipped up on that but you caught it in time and prevented the actual damage. I'd give my L1 help desk a pat on the back over this. Everyone slips up now and then, a huge part of cyber security is social engineering, the important thing is to catch the slip ups before theyre catastrophic, which you did.
Good_Ingenuity_5804@reddit
If anyone contacted the help desk with all of those details, it is obviously a fake call. Most users especially VP's have no clue about their employee ID and will have their exec assistant open the ticket
Zatetics@reddit
Take the win, youre L1 help desk and you werent pressured by someone pretending to be a VP. And you had the gut feeling to confirm with the individual over teams.
Everyone is susceptible to social engineering if the right lever is pulled.
talexbatreddit@reddit
Yep. In a security setting, you have to follow the SOP. If you're unsure, loop in a co-worker, or even better, a supervisor, into the conversation to make sure you're doing the right thing.
Wizdad-1000@reddit
Our Security team has advised all users be verifed no matter whom(exec or not), they also do random vishing tests to both staff and the service desk. The SD has got it down pat and has prevented at least three actual attempts. Thankfully the staff defer the users to the service desk or the 2FA password portal as only admins can do password overrides.
TheRufmeisterGeneral@reddit
Especially the execs. The higher up the chain you go (or the more admin access you likely have), the more important it is to be really sure who the user is!
Ssakaa@reddit
And the eye to catch "I don't have access to my Teams" -> user is in a meeting ... OP: waitaminute...
BuffaloRedshark@reddit
To be fair my teams shows me in a meeting if there's a teams meeting on my outlook calendar even if I haven't actually joined. Teams status is meaningless.
waitingforcracks@reddit
But not if your teams client is off or set to away or similar. It shows you in meeting even if you have not joined but you are on your laptop and teams is atleast open in the background.
Ssakaa@reddit
Yeah, but "misleading" and "they're showing as online at all" can still be valuable, especially when someone just claimed they're not able to get into it. And especially when OP's about to make some amazingly rough mistakes...
T0mKatt@reddit
There was no lever pulled, they couldn't even handle basic skills or requirements for a level 1.
Zatetics@reddit
You must be great to work with.
Did issue occur? No
Did they verify users claims? Yes
Will they do it again? Definitely not.
They've learned and they are better for it and the company didnt have to suffer for the lesson.
Everyone makes mistakes, everyone brings down production, everyone forgets sop on occasion, everyone implements a bodge quick fix that ends up staying in production for a decade. These are part of working in this field. It's vast and complicated. As long as you pick up on your errors and learn from them, youre doing fine.
T0mKatt@reddit
I can see from the hate in this thread for my opinion, as this isn't even a remotely tricky or sophisticated work based on the scenario, why so many companies or employee hacks just continue.
'It's OK little one, you're just a level 1, not a big deal ignoring the process. I to was once a lazy idiot. But look now I'm you're supervisor'
stingray75ma@reddit
Good for you! But as SOP, now find yourself as a family member not familiar with SOPs, etc.
If this almost made you complete the reset, what risks must your family member know but still go through?
Teach them. ALWAYS have questions to be answered to verify the authentication.... On bank accounts, on WhatsApp, on new phone numbers, etc. !!
rainbowlolipop@reddit
Gj! If we worked together I'd take you out for lunch. Yay team!
Trickshot1322@reddit
Good for you mate.
Also good for you on owning up to your (initial) mistake.
Perhaps your department could take this as a learning opportunity and implement some automation to ensure that password resets physically cannot be processed without a 2FA code (with some sort of manager override ability for out of the ordinary cases)
bionic80@reddit
Yup, all core staff with skin in the game in our org have specific 'guided' password 2fa resets configured. Because we've got 70k+ staff though it's not feasible, but everyone with monetary/IT control on any level get special action.
ancientpsychicpug@reddit
Yes OP did good. Theres a reason theres so many layers like this. They will never forget to do that again.
jfoust2@reddit
What does a once-hacked LinkedIn have to do with knowing an employee ID?
aVarangian@reddit
Zoom is only secure if you trust the CCP
PatrickWTTV@reddit
Good job owning the initial mistake. This is why we have so many layers to the process. Humans make mistakes but put enough hurdles in and we will catch one of them.
cayosonia@reddit
Good job spotting the hack. My only question is why you would ever put your employee ID on LinkedIn
Morkai@reddit
Honestly, the biggest takeaway from this story for me, is that you recognised what happened, checked when something felt off, then owned up and acknowledged what went wrong.
Geminii27@reddit
I'm not even going to ask how.
Own_Newspaper9850@reddit
SOP saved you all
UnionThrowaway1234@reddit
No.
Long story short, you did follow SOP and prevented a security breach despite not following ALL SOP.
Also good on you for admitting it to your superiors and trusting them.
MairusuPawa@reddit
None of these are "encrypted" fyi.
awful_at_internet@reddit
The meat at the keyboard is always the biggest vulnerability. You may have forgotten one part of the procedure, but you listened to your spidey-sense and stopped the breach.
Bet you ain't gonna forget next time, are ya?
You'll do okay. Well done.
smeego78@reddit
Who puts their employee ID in Linkedin?
hbdgas@reddit
Some people scrape ID badges from images on Twitter, Linkedin, etc. Maybe someone could have had a visible number on one?
bofh@reddit
If you can’t think of two or three ”LinkedIn influencer” stories that begin with something like “here’s what I learned as employee #8192 at wally’s widgets” then you cannot have ever used LinkedIn influencer your life.
itfosho@reddit
This! Like I don’t buy that at all.
RBeck@reddit
I know Steve Wozniak's number but not my own.
haroldthehampster@reddit
execs do the weirdest things
dodexahedron@reddit
Pretty sure half the posts on the sub could be given that title, verbatim, and then simply closed, without changing their value.
BuffaloRedshark@reddit
Good that you caught it.
But what good does sending the new password over email or teams do? It was reset,they won't be able to log into either.
GremlinNZ@reddit
Unless you force all devices to log out, authentication can hang around for a short while before it needs to re-auth
BuffaloRedshark@reddit
When I had my work email and teams on my phone they'd stop working within minutes of my password changing
tallanvor@reddit
Username and password alone shouldn't be enough to access anything from an unmanaged device if your policies are setup properly, so there's no reason to expire existing tokens.
At my company the only reason you might need your password is if you lose all 2FA methods, so user passwords are routinely reset to random values.
Flashy-Dragonfly6785@reddit
Good catch. Those social engineering attacks are difficult to pick up on in real time!
Techwolf_Lupindo@reddit
That was not a hacker. That was someone with bad intents getting info to do bad stuff. Hackers don't do bad stuff. They hack John Deer tractors so the owner can repair them.
Mrhiddenlotus@reddit
lol have fun trying to change the lexicon.
bob_apathy@reddit
I’m curious how his LinkedIn account being hacked would have provided anyone with his employee ID. I don’t have an LinkedIn account but I’d find it odd if they asked you to provide it.
ChampOfTheUniverse@reddit
I've seen it a few times where employees post a picture of their ID badges over the years reflecting promotions. I've worked at two places where the employee ID was printed on the badge in small print. That's my hunch.
Mrhiddenlotus@reddit
or he did something dumb by signing up for linkedin with his work email that might be
emp_id@company.combob_apathy@reddit
That actually feels like something a company might do because they don’t consider the consequences of that type of information being used against them. The bad actors do their homework and use very trick in the book to their advantage.
chillyhellion@reddit
This would be my first red flag, honestly.
In all seriousness though, you handled it well.
badaccount99@reddit
Teams, Zoom, and Outlook are super not a safe place to send passwords unless you do some extra encryption.
I yell at my guys all of the time for sending passwords via Slack. The non IT folks put passwords in Sharepoint. sigh
Seriously. Find a thing like LastPass or Bitwarden to share passwords.
DolanUser@reddit
Yeah, I almost loughed out loud as I read it.
Crimtide@reddit
Social engineering..... beware
lyenax@reddit
Awesome job rebounding. Must have been a little bit of a panic attack.
There's a colleague of mine who told me, "The only person who does nothing wrong is the one who does nothing." so mistakes are part of our growth.
This is interesting because, a lot of the times people hear system admin and think technical work on servers. Reality is though that the processes (SOP), automation, workflow are all tied up to our systems.
It's a good reminder that L1/Service Desk are also system admins in a way.
Necessary_Emotion565@reddit
Self reset passwords ftw. No need to call
St0nywall@reddit
These guys that do this are wizards at social engineering.
He likely had dozens of methods to try that could have compromised even the best SOP's. The fact you did your due diligence and caught onto something like the Teams presence makes me think you are very observant and inquisitive. All great qualities in any support position.
Good on you OP, proud of you for this!
machacker89@reddit
I definitely give Kevin mitchnick's books a good read
tj818@reddit
At the end of the day the guy didn’t get in so I’d say good job.
bobs143@reddit
Good for you for spotting that this was a hacker. The team should make this a trading issue on SOP.
But a huge pat on the back for not giving credentials over the phone.
bobsmith1010@reddit
you're security is only good as the weakest link. But also you have to be right 100 percent they only have to be right 1 time. So it hard.
This is why I tell my boss we need an automated system but he kept saying how the help desk has process to authenticate someone. Yet how do you know that when that help desk person is resetting a password or factor that they actually did what they were suppose to do. Even audits only help after the fact but doesn't stop the attack if they got in before you had a chance to audit the interaction.
SuperDrewb@reddit
You may have made a small mistake in the flow, but overall your awareness likely saved your organization from ransomware. The tactics you are describing are the TTPs of a very successful threat actor/ransomware group.
JerikkaDawn@reddit
You realized it in time and hopefully management gets that you're not likely to let it get that far again. Sounds like they do.
wrincewind@reddit
Yep, the swiss cheese model. Assume every layer is imperfect and has holes; each layer has a chance of catching something the layer above 'should' have caught.
bishopExportMine@reddit
Defense in depth is never the wrong call
dodexahedron@reddit
And assume that every user and every input from every user is hostile.
Excellent-Program333@reddit
What are you all using to send MFA’s codes to known devices? 3rd party tools? Need something in our org. Employee ID, social last and DOB are no longer reliable.
KStieers@reddit
Duo can do it... they also partner with Persona, so they have to show you ID and match a picture...
That said, Persona has its own stack of privacy concerns..
Excellent-Program333@reddit
Thanks. We are slowly rolling out Duo for local machine logins. I think we need to escalate.
ChampOfTheUniverse@reddit
I'm surprised that the home addy is a method of verification since that is fairly easy to obtain and typically isn't something any coworker should be able to find internally without good reason. How his employee ID got out into the wild is crazy, like what would possess him to put that out there? But man, social engineering can be scary effective. This is a great learning opportunity, especially as to why being honest about mistakes is appreciated. You could have remained silent and caused havoc which would have eventually lead right back to you, but you did the right thing and owned up to it and took action quickly. Good shit.
aguynamedbrand@reddit
r/helpdesk ≠ r/sysadmin
SAL10000@reddit
Nice dude
pinkycatcher@reddit
Director here: if one of my L1 guys did this I would want to know and I would take them to lunch. Security happens in layers and anyone can make a mistake, but questioning yourself and double checking is the best thing to do. You did great
silentstorm2008@reddit
SOP to have users home address? thats a breach waiting to happen
j1mmyava1on@reddit (OP)
I didn't say that it was SOP, just how he introduced himself.
SirLoremIpsum@reddit
I think that's a common tactic - they data dump all the info they have on the person without asking so you go "oh obviously they're legit they have his inseam length, car colour and favourite football team. Must be legit"
Nonaveragemonkey@reddit
Right? That's one of the easiest things to get lol
SirLoremIpsum@reddit
This is why you have MULTIPLE parts of the swiss cheese model.
The holes have to align for something to "mess up"
And whomever designed your SOP did it right.
You had to verify Identity, send 2FA code AND send over proper channels.
Don't see this as a total loss - people who do this are very good at browbeating people into giving up stuff they shouldn't. And you didn't.
So treat it like a win. The process worked, the attacker did not compromise the network.
You demonstrated you are trustworthy to your boss by immediately raising it up the flag pole and owning up to any mistakes.
dispatch00@reddit
You're hired.
BoltActionRifleman@reddit
I’m becoming more convinced every day LinkedIn just takes user information and sells it directly to criminals.
its_FORTY@reddit
Why in the hell would anyone put their employee ID on LinkedIn?
fr33bird317@reddit
Oh boy, i would have so much fun messing with this guy. Calling him stupid because he can’t type in a password correctly.
Ssakaa@reddit
Humans are fallible. It's exactly why we have those procedures and requirements, to defend ourselves against those mistakes. As "support", you get browbeaten with "be helpful" so much... your very position is a huge source of risk for exactly the scenario you landed in there. Your first instinct is help... but you did the right thing and paid attention to the clues, and even though you made a mistake along the way, you still validated things and avoided the breach. You could've done better... and I suspect this close call just filled in that tiny gap in training, you will do better every time in the future. Good work. The fact that you're looking at this with the level of clarity that you are is a pretty good sign for you, too. You almost messed up big, but you didn't.
JerikkaDawn@reddit
And in this case OP didn't try to hide it. Anyone else could have gotten that call and been similarly brow beaten by a "VP" and who knows how that would have shaken out or of it's happening. Management should definitely take this as a lesson learned for them well and shore up the controls.
Ssakaa@reddit
Yeah, sounds like OP has a good team there overall, acknowledging the missed step, but also acknowledging the outcome. Hopefully they take it back and look over "how do we keep someone from making this mistake the rest of the way through?"
aimless_ly@reddit
This is a perfect example of why you should never rely on a single security control and instead deploy defense in depth. The initial control failed but further ones prevented a compromise.
4xi0m4@reddit
Exactly. The social engineering aspect is what makes this so insidious. The desire to be helpful and solve problems quickly is exactly what attackers count on. That gut feeling when something feels off is there for a reason. OP caught it because they stayed curious and didnt just follow the path of least resistance. Defense in depth only works if the humans at each layer actually pause and think.
nayhem_jr@reddit
No one up there wants to speak with us so casually.
aguynamedbrand@reddit
r/helpdesk
H3xu5@reddit
You did really well. This is exactly what any security training would tell you to do. That's not a knock at you at all. It may sound silly to us having to retake the same shit every year. But a lot of people have to be reminded of this.
Level8Zubat@reddit
Job well done, process working as intended.
Ozone23@reddit
I mean this ended probably as well as it could. You didn’t follow SOP, but it still saved you. I’d personally call that a win and an opportunity for more training.
Ssakaa@reddit
I mean, training for others, mild review for OP. They clearly know what they did, and this was the best friggin live fire training you can get for that procedure... OP has seen exactly how important that step is, and why.
Ozone23@reddit
Yup, a little pucker will burn that into your brain and you’ll never forget it! We all have those moments with different things.
joerice1979@reddit
Yes!
If in doubt, fail closed and listen to your (educated) gut.
T0mKatt@reddit
Almost like you are proud of yourself, and same with the baby feeding replies.
Wasn't even any form of tricky social engineering, you just simply ignored a very simple process already in place.
Wow you deserve a pat on the back for that one.
Adimentus@reddit
Good job holding your ground man. Things like this can happen to anyone at any level. SOPs are there for a reason and we found out the reason for this particular one. Don't beat yourself up over it and I doubt it'll happen again.
RunningAtTheMouth@reddit
Mistakes are how we learn. The critical parts are:
Now, had you failed to notify your chain, or tried to hide it, I'd choose to chuck you out the door. There's no place for that kind of behavior. We need the right people to do the right things, always, no matter the conditions.
You did the RIGHT THING. And you've learned something about social engineering in the process - the kind of less you cannot get from the classroom or online courses.
I wouldn't worry about a thing. They'd be idiots to punish you for that. I assume you wouldn't work for that kind of idiot.
Specific_Expert_2020@reddit
You didnt follow the whole SOP..stood your ground and didn't say the password as it was not SOP.
Mistakes happen, but (work in Cyber for a MDR for years) you did stop it.
Also reported it.
This is why we do lessons learned as we have to be right every time and threat actors only need to get right once
Phenergan_boy@reddit
Don’t sweat it, you did good. That’s why we have multiple failsafes in place for this kind of thing