Are there any other fulll time admins who just look after backups and recoverys here?

Posted by Mr_Dobalina71@reddit | sysadmin | View on Reddit | 23 comments

It's a full time role for me, personally I really enjoy trying to keep the backups as close to 100% as I can, guess its like a game for me.

Also being able to recover servers when the shit hits the fan is quite invigorating.

Not saying it can't be frustrasting at times though.

[-]

Sree_SecureSlate@reddit

You aren’t just an admin; you are the Last Line of Defense in a world where ransomware has turned backup recovery into a Tier-1 survival function.

While others build the ship, you’re the digital paramedic who ensures it actually stays buoyant when the hull hits an iceberg.

[-]

AbandonedHope83@reddit

I work for the major backup tech company so yes. Thats all I do lol

[-]

2 jobs back, most of my job was babysitting all the systems. Like way more than 1 person should do but I was good at it. But for low $20's per hour, I asked for a raise, got denied, said I was looking for a new job then, they fired me weeks later "because clearly I didn't want to work there" and almost all the IT systems collapsed, they're out of compliance, getting fined by their insurance companies, getting sued, and after I left and I heard they're attempting to replace me with 3 people.

The moral of the story is: fuck companies like that and it's a fine job as long as they pay you enough.

[-]

Lando_uk@reddit

So how many servers/instances do you look after to make this a full time job?

I'd think for most admins nowdays, looking after backups is just another line item on their long list of roles/responsibilities.

[-]

iDontRememberCorn@reddit

and those are admins without legal retention requirements.

[-]

MathmoKiwi@reddit

At the last MSP I was at, we had a T2 tech who did this full time.

[-]

TrickleFicky@reddit

I have a sysadmin who spent a year drploying 80 laptops to existing users... No special setup, lpgin, profile environment...done. did nothing else... He's on pip now...

[-]

malikto44@reddit

I worked at a small (200-400 employee) place, and backups became almost a full time role for me. Mainly because a backup needs such a huge stack of infrastructure. Network issues? Backups miss windows. I/O too slow on the deduplication database? Lots of errors, and more missed windows. Machines too taxed to handle backups? Yet more issues.

Bored CPU? Just a media server has all 512 cores pegged.

However, it is awesome when the stuff runs, and your automation grabs VMs at random, and tests them to verify restorability automatically. Same with restoring random filesystems and seeing how data is there.

[-]

beneschk@reddit

Yeah this was pretty much my first IT role.

There are some places that have backups in so many different places, confirming and verifying them is a full time job. People are happy to pay for the piece of mind, so do the job with diligence.

For simple monitoring of backups in multiple locations, backupradar can be helpful, but requires a fair bit of configuration depending on your different backup types.

The only natural next step is to start thinking like large enterprise and how they manage the issue. This could be planning to move backups into a single vendor to ensure you have a single plane of glass.

If you use tools such as Veeam or Datto BCDR units, you can configure automatic test restore procedures that boot the backups in a virtual environment and runs a script to ensure the critical applications/services start correctly and provides reporting off the back of the script.

There is always a solution to your problem if you think how do the people in 10x larger environments work this out.

[-]

TrippTrappTrinn@reddit

When we had very distributed backups with questionable reliability (lots of BackupExec on remote sites) we used Bocada for reporting. One report to check what backups failed.

[-]

tepitokura@reddit

I had nightmares with BackupExec. Move away from It 6 years ago aprox.

[-]

StuckinSuFu@reddit

About a decade ago working for the Dept of Interior, that was me. We were the offsite (3 of us) who just had 250 or so offices plus the main office backup to us each night and we were ready to swamp over to the "production" site if the main office on the other side of the country was nuked or whatnot. Each morning wed do our rounds- row after row of Netapp racks for broke drives. Theyd send us new drives via Taxi as it was quicker than fedex/UPS. Think we had about 40Petabytes at the time which seemed so big back then lol.

Wed run tests twice ish a year where the Feds would call in a "emergency" and make sure we could swap up a small subset of production to our datacenter instead of the main one.

[-]

Tx_Drewdad@reddit

Back in the day when tapes had to be swapped out, sent to Iron Mountain, retrieved from Iron Mountain, etc. it was a significant part of one of our team member's duties.

Modern backup solutions have significantly reduced what was once a huge workload.

[-]

felix1429@reddit

Hi that's me

[-]

JustinVerstijnen@reddit

Of course every admin should. A non-tested backup is not a backup.

[-]

barthem@reddit

So by that logic you believe every admin should be a fulltime backup engineer?

[-]

shemp33@reddit

I worked in a larger company, and we had a team of people who are dedicated to enterprise backups.

There are:

People assigned to adding new servers into the system;

People assigned to make sure the system is running everyday;

People assigned to handling restore requests;

People handling cloud estate backups and making sure cloud stuff is safe…

For larger (Think: Fortune 100 and above) companies, this function can be a smallish team (say 6-10 people).

So what you’re doing isn’t really that big of a surprise. Except that a lot of folks in this sub seem to be one-person shops where they do everything from backup to vpn to email to sql server… depends on org size mostly.

[-]

picardo85@reddit

One of the MSPs i worked with had a bunch of people just responsible for backup and recovery

[-]

Pocket-Flapjack@reddit

Backups are one of many plates I spin 😂

Veeam was game changer, I love it and I have all my replicas spin up each month using surebackup to test them!

Wonderful bit of kit

[-]