Post Outage Actions
Posted by trevormcneal42@reddit | sysadmin | View on Reddit | 4 comments
Recently had a pretty big outage at work. Our storage, which held 95% of our VMs, had a hardware malfunction and unalived itself. Luckily, we had backups but not of every server. We had no budget/resources to setup replication or even an ounce of DR. That’s #1 action is to get replication and DR setup.
What’s something you experienced during an outage and fixed afterwards?
whatdoido8383@reddit
Well, sounds like you were underfunded and I bet the business magically finds budget for proper backup/DR now...
Don't forget to protect yourselves from malware as part of the rework and budget in testing too. Backups/DR is useless unless you test.
trevormcneal42@reddit (OP)
Our internal fan quit on dell Emc. And yes, we found some money to fix things. Funny how things happen
KindlyGetMeGiftCards@reddit
When something like this happens take advance of it to get budget and changes you need, keep a wish list always and when disaster strikes take it to task to get things done that should have been done already.
Most companies have a limited budget and the focus moves to put out spitfires as needed, so in this case you use this to highlight the issues and how to fix it.
mixduptransistor@reddit
ugh, fuck off