How do you guys cope with the ever-looming threat of cyber attacks?
Posted by Tylerjackx@reddit | sysadmin | View on Reddit | 108 comments
Do you guys loose sleep over it too? Have you done anything to help cope with the stress/anxiety of it?
Good_Principle_4957@reddit
It really doesn't worry me. Here is what I do to not stress about this.
1) Have backups
2) Test backups
3) Create a disaster recovery plan and print it out or store it someplace that can be accessed if your network is down. Go through the steps couple times a year.
4) Get cyber security insurance if possible. Before we got our cyber insurance we also paid for a 3rd party pen test. They bragged about how they usually get Domain Admin access in a couple hours. They had their pen test laptop connected to our network for a week and they never got in. This made me feel pretty good about all the work we had done up to that point, and even though they never got DA access, they still provided a lot of good info on weak points we missed.
5) Use MFA with conditional access rules with some of the most basic stuff like don't allow login from outside your country, etc.
6) Train users about phishing and how to watch out for it.
GloveLove21@reddit
I don't stress about what I can't control.
en-rob-deraj@reddit
If it happens it happens. Hopefully we are prepared enough where it doesn't.
maziarczykk@reddit
I’m sleeping like a baby.
qordita@reddit
Wake up every three hours and cry?
ScroogeMcDuckFace2@reddit
and crapping my pants
mcdithers@reddit
I don't. I'm solo IT for a small manufacturing company (~100 users and they pay more than my previous global gaming/resort employers did), and technically fall under the Engineering department. My boss, and the COO are all on-board with my recommendations, but the owners don't want to pay for them.
If shit hits the fan and the owners want me gone, my boss will make them fire me. I have standing offers at 2 casinos and 3 insurance companies, so I'm not too worried about it.
KirkpatrickPriceCPA@reddit
We've worked with a lot of people who have felt the same. Building a clear roadmap, documenting responsibilities and using tools to track risk and controls can ease that looming feeling.
However, stress is real and mental health is something everyone needs to take care of. Make sure you take breaks, set boundaries, and don't be afraid to lean on other people for help.
Blue-Purity@reddit
I look forward to it. I get to show my manager the consequences of cutting the IT budget.
Keanne1021@reddit
Not the answer expected, but Yeeaah, ok, we sort of understand where you are coming from 😁.
ClearlyTheWorstTech@reddit
A company I manage has picked the worst possible option any time I have suggested solutions in the past to "meet requirements". They got hit by Safepay on Monday. I was able to restore from backup after kicking them out of the systems and closing the door. Restoring was slow because it was coming from a residential ISP with asymmetrical speed. They contacted their cyber insurance and now, after skating by for years and not expending anything substantial on IT, they're under the scrutiny of a cyber security company that will report everything back to the insurance company. They're fucked.
TeflonJon__@reddit
I hope you have your recommendations documented to CYA though, we all know how the business likes to blame “IT” for all their issues
ClearlyTheWorstTech@reddit
Worry not, my tag is to instill minor doubt while trying my utmost to make good decisions. If not for them, but for me.
KB3080351@reddit
entirestickofbutter@reddit
nicotine
qordita@reddit
I have way more anxiety over that mysterious hyper-v cluster I inherited.
Ruachta@reddit
Follow policies and have insurance. Sleep like a baby.
Humble-Plankton2217@reddit
I don't anymore. We went through it twice already - 1.) so I know the drill and have excellent backups and 2.) leadership invested heavily in prevention tools after the 2nd incident.
When it happens again (not "if") I'm ready. It will suck, but I'll get through it and we'll come out the other side better than we were before.
XCOMGrumble27@reddit
I take roles where it won't be my problem or I won't be dealing with it alone. Also helps to work in organizations that are large enough that there are teams dedicated to handling it.
Twinsen343@reddit
Common sense and attack surface reduction
Foreign_Impress6535@reddit
It's nice only dealing with an air-gapped network.
JerryBoBerry38@reddit
Lose sleep??? 9 to 5. Go home and forget it. It's a job to pay the bills. Nothing more.
rootkode@reddit
You won’t be going home at 5 when it happens to ya
BeagleBackRibs@reddit
I will. I'm not fixing anything when I warned about it years ago. They can hire an MSP
CoolNefariousness668@reddit
My time is not free, so whatever.
_Meke_@reddit
Yep, still getting paid.
sir_mrej@reddit
And/or you'll be going home at noon when it happens
SevaraB@reddit
No. My job is to build in anticipation of it happening- in our industry, attacks are constant and getting breached is a question of "when", not "if."
My day-to-day is preaching common-sense controls to both developers and fellow engineers, for example:
BastardOPFromHell@reddit
I keep my resume current
CEHParrot@reddit
SOC Manger here, job security.
OpacusVenatori@reddit
Have done all the CYA steps with written memos and recommendations all the way up the chain. If they don't care enough to act on it, then not going to lose sleep over it.
bamaknight@reddit
Thats the way to do it. Then when you get hit and they trying to find someone to scapegoat you git your papper trail. They try and fire you than you go to whomever is investing it and give them your notes. They will get ehats coming to them.
I_T_Gamer@reddit
This is the correct answer IMO. "I can explain it to you, I cannot understand it for you". I really do do my best to not sound all "doom and gloom" but honestly some of the expectations of cost, and implementation are scary enough on their own.
CYA, its the only way to get good sleep for me.
ZAFJB@reddit
Start with the mindset that you will be compromised, it is just a case of when.
Based on that assumption make sure that you have implemented proper immutable backup, business recovery plans, and disaster recovery plans.
Overlay that with protective and preventative measures:
Encourage a no-blame culture, and encourage staff to report suspicious activity, and to tell 'oh dear I clicked at thing'. If you shit on you staff for making mistakes they will simply not tell you when they fuck up.
User training - start with phishing training. Train everybody from the CEO down. No exceptions.
MFA, everywhere. Don't over do the frequency, MFA fatigue is a thing
Use a proper email filter
Implement 24x7 monitored XDR. Pay a third party organisation to do this.
Minimise or eliminate inbound connections to your stuff. Use reverse tunnels or similar
Proper next gen firewall with geo blocking and IP block lists
Move your email to the cloud
Manage your mobile devices
Get certified. In the UK Cyber essential Plus is a good place to start. Doing certification forces you to get your hose in order.
bukkithedd@reddit
By isolating and constantly testing my backup infrastructure and backup-jobs, in order to be able to recover quickly when shit hits the fan. Because it IS a question of when, not if.
Plus of course giving my superiors a written statement about what's what in terms of deficiencies and risks, just to cover my own ass for WHEN shit hits the fan.
If they choose to not do anything to rectify those deficiencies, whatever happens isn't on my head.
ClassicPap@reddit
The same way everyone copes with the ever looming threat of anything. Anyone walking across the street could be mowed down by a bus, you could be hit by a stray bullet etc. You do what you can and you live your life
GloomySwitch6297@reddit
Simple. like the other person said. I do my job and go home.
You want me to work longer? Happy to be paid 2-3x more for overtime.
Your anxiety is exactly the same like you would be worried about head on crash from a driver that does not pay attention. It may happen, but you still driving and hope that it won't happen
sean____m@reddit
Test your backups. Document the dependencies you know about. Practice rebuilding the stuff you don’t think about from scratch: DHCP, DNS, directory services. When it happens, make sure the incident coordinator knows the docs and plan exist. Never gonna be perfect, just act like a professional (whatever that means to you).
patjuh112@reddit
Cybersecurity: 95% prevention, 5% dealing with results of attacks should be the aim (imo).
Benjamin Franklin said it I believe: An ounce of prevention is worth a pound of cure.
BlueHatBrit@reddit
I work hard, and then I go home.
When I'm at work I do my best to highlight and fix issues. On the rare occasion that leadership tells me to do otherwise, I make sure it's in writing.
Then I go home and forget all about it.
The job is there to pay my bills. If they get hit by something and it's my fault, I'll do everything I can to fix it. If it's their fault, I'll make a professional effort during my work hours and will make sure I get overtime or additional holiday for anything additional.
I make a habit of not taking crap jobs, so usually there isn't much to worry about anyway.
If you're taking your work home with you, I think you need to review your relationship with your job. Maybe that's therapy, maybe it's a new job, maybe it's finding some hobbies to occupy yourself. Whatever it is, work to live don't live to work.
Talt45@reddit
A little. But we do cyber security audits twice a year, and training is circulated to everyone in the company. There comes a point where it's on your colleagues to be on board - you can't control their clicking.
pm-me-your-junk@reddit
I tick the boxes I'm supposed to tick to meet our compliance obligations, and send emails BCC'ed to myself to cover my ass for the boxes I can't tick. Literally nothing else I can do about it so beyond that I don't care - not my problem.
Ill-Detective-7454@reddit
For years i was not worrying about it because got a lot of security in place and every attack got blocked at the first line of defense.
Then a hacker chained a few 0days to get root on one of our public server (hardened and fully updated monthly ubuntu apache2 php stack) but they made mistakes and we got alerts of the intrusion immediately and nuked the server before hacker could pivot to other servers.
I could barely sleep for 2 weeks after that security incident.
Lesson learned, if something had a RCE vulnerability in the last 10 years, it should never be exposed to the internet because its gonna have them again. Now we hide everything behind static ip whitelist or behind wireguard if it is important.
LeakyAssFire@reddit
It's security's job to worry about. Not mine.
My worry is how to recover from it when if\when it does happen... and I have that locked down.
bobsmith1010@reddit
I would disagree, not knowing the exact role you're doing but if you're supporting any application then you should be making sure your secure. Security doesn't know your application(s) in and out and any time a security guy tries to act like they do then that means they really should own it. They can give guidance but can they tell you if you have the right user access setup, the right fail overs, etc?
LeakyAssFire@reddit
Oh, for sure, and that is all considered here, but the stuff I do is all intertwined with O365\Azure AD and low level on-prem AD stuff. The production\user side is available to anyone with an E3. Access to resources inside of it are all user controlled (doc sharing, Teams team access, auto attendants, call queues, etc..) based on ownership of said resource and well defined policies. Admin side is controlled by security and wrapped in PAM.
It's a large org with a large security umbrella that is not my responsibility... even if I do have a well documented and valid concern. My only responsibility is to conform to the policies and procedures in place that are dictated by said umbrella. If it's breached, then everything is breached. and at that point there's a much bigger problem.
I get that security is everyone's responsibility, but in the case of a threat actor at my place of work, we're talking about things outside of my control and my responsibility regardless of how I feel about it.
Mr_Dobalina71@reddit
Its sort of mine as I look after backups, but I’ll blame security as they should have never got in :)
wezelboy@reddit
I don't lose much sleep over it. There are so many vectors of attack that have nothing to do with what I do. I figure if an APT really wants to fuck with us, they will somehow. I just try to make it so that it isn't super easy for them.
bobsmith1010@reddit
I've had an attack and the sad thing was on my own I had started to put changes in place long before the attack occurred. But because I had to keep fighting for all the changes it didn't happen fast enough.
After the attack they had to rebuild and they were lucky that some of my changes actually saved the day and allowed them to stay up and running. But, they're taking their time to fix all the security holes that they realized. Our security group is dragging their feet on stuff I'm ready to address, to the point I've told them if we get compromised again I'm turning off my ringer and not getting out of bad since you did it to yourselves and none of my guys are getting involved either.
redyellowblue5031@reddit
I leave each day knowing I did what I could to make us a little more resilient. That’s the most I can reasonably do.
neuronlog@reddit
Personally I sleep better after good backups, strong PWs, and 2FA. Still paranoid sometimes, but at least prepared.
redmage07734@reddit
Backups lots of back ups
AerialSnack@reddit
Boy am I glad I manage an air gapped network.
slippery@reddit
My style is impetuous. My defense is impregnable. And I'm just ferocious.
RobieWan@reddit
I'm always so tense I wish I could loosen up when I slept.
But I don't think about it. I do my job the best I can, letting things as secure as I can, and try to keep up on updates, patches, etc.
If someone is gunsi get in, their gonna get in. I can only do so much.
Have you done anything to help cope with the stress/anxiety of it?
You need serious therapy my dude. That's not a dig, but an honest truth. There is zero reason to be stressed out anxious over it. Build better security, they're build a better hacker. That's the way of the world.
And here is nothing wrong with therapy!
Laservvolf@reddit
What do you mean "threat"?
orten_rotte@reddit
By being fuckin awesome
Smith6612@reddit
Insurance. Being proactive. Establishing your basis and documenting thoroughly why security controls and IT budget are important, with plenty of CYA.
Insurance is neat because good policies often require that previous sentence.
Splask@reddit
Reading these comments makes me feel glad that my place of employment emphasizes cybersecurity. Like a lot.
Keanne1021@reddit
ISO 27001?
Erutan409@reddit
My employer took the Zero Trust approach. Anything provisioned has absolutely nothing accessible on it. I have to submit a ticket to get access to RDP.
Sure, it's about a 2-3 week turnaround for these ports to get opened and blocks me from getting my work done. But, hey - security.
You could try that 🤷♂️
CostaSecretJuice@reddit
True zero trust is dynamic, not static. Meaning stuff like RDP is enabled, it’s just checking everyone, every way possible.
Erutan409@reddit
What?
CostaSecretJuice@reddit
Basically, what your employer has going on is NOT Zero-Trust. Sorry bud, I tried to say it in a nice way.
Erutan409@reddit
I feel very untrusted. And I'm reminded of it on a daily basis. Especially when my tickets get push back.
According to my cursory search of 'zero trust security', it certainly covers my scenario. They're verifying the source before the target can be reached.
Turbulent-Pea-8826@reddit
Yea… there’s a whole lot more to it all than that. Manually requesting RDP access, might be not trusting the source but it is not utilizing all of the other security principals. I don’t feel like doing a deep dive on Reddit but basically Ya’ll need a decent VPN and PAM.
ClericDo@reddit
Zero trust means that you don’t trust devices just because they are in a special private network or use a certain IP. It involves every network service performing authentication/authorization for everything. So having firewalls to block access isn’t really part of Zero Trust by definition, since it’d be using your network location for allow/deny
Erutan409@reddit
I know what zero trust means. The phrasing of that response made zero sense to me.
ClericDo@reddit
Oh yeah their reply was confusing. I kind of took it as them meaning that static rules (firewalls) aren’t part of zero trust, but dynamic rules (authorization) would be
sir_mrej@reddit
Full RBAC with approvals can be seen as Zero Trust tho
SilenceEstAureum@reddit
The only one that stresses about it is the aging boomer on my team that thinks cyberattacks manifest out of thin air. We enforce best practice as well as we can. EDR with 24/7 monitoring by an actual security company, 2FA across the board, good firewall, no local admin rights for users, multi-layer backups, etc…
I worry no more about cyberattacks than I do about the weather.
Turbulent-Pea-8826@reddit
We have a budget and buy products to protect ourselves. We have good backups and we spent a lot of money on a decent system. We have good firewalls. We have agents on endpoints that do monitoring. We have a privileged access management system. We monitor our network. We have a disaster recovery plan.
We aren’t full proof but there are easier fish to fry and bad actors are much more likely to hit them.
RegisHighwind@reddit
Backups. Immutable. Off-site. Quorum enabled. I'm gonna restore and make the CIO buy me lunch.
musashiro@reddit
Keeps me employed so im good 🤣
InevitableOk5017@reddit
Small shop here, it’s a constant thought in my head but I keep everything up to date in my control and let people know their stuff if vulnerable or out of date that is out of my control. I’m sure it’s annoying for them but it affects me so.
MadMan-BlueBox@reddit
NGL, it does occasionally keep me up, and we do security pretty well.
But the way I try to think of it is like this / Mindset I try to instil in my team and directors:
Everything is hackable and exploitable, it's not a question of if but when! However every small improvement, vulnerability patch, firmware upgrade, EOL replacement, process improvement, user access review, privilege review, access request (Scrutiny and approvals) Backup check, DR/BC Test helps us keep moving that 'when' further out.
I encourage the mindset that security and our preparation for D(isaster)-Day is never done, its a constant moving target that we need to prepare for and be ready for!
degoba@reddit
I lose zero sleep. Maybe something will finally fucking change if we get hit bad enough
sardu1@reddit
Having cyber security insurance, updated firewall, trained users, and backups helps
Kahless_2K@reddit
Accept that they are just another day at the office, and harden systems appropriately.
Salty1710@reddit
Three Words: "Immutable Offsite Backups"
n3tiz3n_X@reddit
100%. This is the best peace of mind you can get. Just make sure to test them regularly.
n3tiz3n_X@reddit
Hire a good pen tester and/or external auditors once a year. Embrace the friction between ops and security; it's a necessary evil that I've come to appreciate after many years in IT.
ipreferanothername@reddit
I'm not sure how the department I work in technically keeps anything working to start with, so it's just a nice thing to get paid despite constant self invoked problems.
man__i__love__frogs@reddit
We are passwordless with yubikeys and CA requires compliant devices, would like to see someone try.
We also pay for a pen test every 2 years.
donewithitfirst@reddit
I’m ready to retire. Bring it on.
RookFett@reddit
Dark forest theory. Become a black hole on your forward facing ports, double firewall redundancy, vlan separation, limit access to what is the bare minimum needed.
Isolate best you can, contain iot devices to separate network/vlan.
Robust training for users.
Log - monitor - alert scripts running. Run PEN testing on a monthly basis.
Kill/disable services not required.
Follow best practices.
This should get you started.
OB71@reddit
Safety is an illusion. Do the best you can with what you have and realize you still have to live your life. You'll go crazy constantly worrying on the edge of your seat waiting for "The Big One". That said definitely will be cathartic to say I told you so to the big wigs who dont listen about security awareness and say every IT purchase besides a brand new PC for them isn't critical.
meathead67@reddit
Whiskey...
Regular-Nebula6386@reddit
Nervocalm, drops
TournamentCarrot0@reddit
I think it’s best to plan for it and treat it as an inevitability rather than an event. Have the plans, practice what to do regularly for different scenarios, document things, ensure your patch cycles are aggressive and you have a good vuln program and lastly be upfront with leadership across the org that this will happen and here is what we’re doing to prepare; include business folks in tabletops even when relevant.
ApricotPenguin@reddit
The first part of what you said concerns me for your health.
Consider this - Do you develop extreme anxiety (about road accidents) when you commute home?
The key thing is realizing that you can do everything right, but still lose.
So all that's left is for you to do your best, so you don't have regrets.
rusty_programmer@reddit
It only takes one misconfigured system or some zero day you never thought would exist wrecking the things you’ve built.
It’s all dust in the wind anyway. Keep truckin’.
concretecrown85@reddit
don't own any servers. only use SaaS apps. use one of the top 3 XDR's (we use MS Defender). use one of the top 3 email security services (we use Abnormal).
rusty_programmer@reddit
I’m not worried about it. Eventually, you’ll reach a point of serenity realizing it never was and never will be your equipment or data. You’re the custodian.
As long as you perform due diligence and due care, you’re fine.
Weird_Presentation_5@reddit
Let one of the 45 security teams worry about that. We make shit work.
I_LICK_PINK_TO_STINK@reddit
SentinelOne, mostly.
TheWino@reddit
Before the attack we went through we had a plan in place. We got hit and moved forward with the disaster recovery plan. Afterwards the anxiety hit me hard and took a while to finally move on from it. Have a plan that will help tremendously.
bratch@reddit
Defense in depth.
L3TH3RGY@reddit
Give all the info and evidence it can happen. Save all correspondence. Jobs done afaic. Pay me now or pay me later.
InvisibleTextArea@reddit
I have offered my expertise and experience to make recommendations. C-Suite then make the decisions either to follow my advice or not.
I have suitable CYA.
Ultimately it's not my train set and not my trains.
TheBlackArrows@reddit
I have a corner.
Reverent_Revenants@reddit
Whats your core concern? Job security? If you stay aware of vulnerabilities and inform leadership, then its not your problem.
Is it that you genuinely care about compant data? Well, you should care to some degree because youre being trusted securing ppls PII, but see #1.
badogski29@reddit
No, I learned to stop mixing work and personal life.
Redemptions@reddit
Scotch
miscdebris1123@reddit
Updated resume.
Sensitive_Scar_1800@reddit
I routinely put pennies up my butt and have them out at DEFCON….been doing it for years….and I easily hand out like $300 in coins….which is like 30,000 coins!
I figure there’s a good chance a hacker has touched one of my butt pennies and that makes them hard to fear
PontiacMotorCompany@reddit
I understood you can never fundamentally erase risk in any arena, You could have quantum encryption and a baby pressing the wrong button will still cause an outage or incident. Hyperbolic but you get the jist.
Do your job well, Let management know about any glaring vulns and mitigation is up to them. Wipe your hands and head home.
TinyBreak@reddit
You put up the best defenses you can muster. If you don't have the budget you call that out, and then you hope for the best and prepare for the worst.
Its a bit like how every time you jump in a car you could have an accident.