Time to learn from my big mistake

Posted by natariimei@reddit | sysadmin | View on Reddit | 13 comments

I usually backup data, it's on my anxiety todo list. However, I recently, temporarily, used a portable hard-drive for a customer to hold half of their data until I could get their 12 yo computer upgraded.

Merely two days before I was able to successfully get their computer upgraded; the 12 yo had a fit of shutdowns, and completely wrecked my temporary Hard-Drive solution. Now, they're missing important files, and I have officially given myself a bad name.

The poor connection at their office, barred me from uploading their documents to a secure location, so I got crafty, or so I thought. Not everything uploaded to the secure location like I thought it did.

Now my boss is scrambling to help me fix my temporary SSD, and it's not looking good.

Sending prayers to the universe that he's able to mount it and get information from it.

[-]

kero_sys@reddit

How long have you been supporting said customer? Is this a one off support request for a consumer?

Or is this a business with a SLA with yourselves? If it's a business you have been supporting, surely they would have been aware the computer could die at any point and lose the data?

We took on a customer back in 2017 and as part of the onboarding we reimaged all their window 7 machines to windows 10. Their company management, our project manager sent out multiple emails saying upload data to the server if stored local.

We did a presentation to all staff about upcoming changes to the service. Reminder about devices being wiped, copy data to the server.

Anyway, one of the heads of department didn't move 14 years of data from their D drive, I wiped the device.

The customer ended up sending the drive off to a specialist data recovery service, costing them £1200 for the pleasure of getting 80% back.

Things happen, don't worry about it. I laugh about it now.

[-]

natariimei@reddit (OP)

We onboarded this client fairly recently. We're an MSP for small businesses, so this comes up fairly regularly where we bring on clients with older computers. Our knowledge was passed on, and backup solutions were discussed, just never implemented. Many messages were sent about the severity of the computer. It was shutting itself down on a weekly basis, I honestly thought the computer would break before my little portable.

[-]

CombJelliesAreCool@reddit

Don't sweat it. These are directly the consequences of their own actions. You can't fix every fuck up.

[-]

natariimei@reddit (OP)

I've been a tech for 4 years now, this is my first loss. I've never had to tell someone that their data is lost because I wasn't fast enough. Is there a way to do it without leaving a bad taste in their mouth for my practice?

[-]

mrjamjams66@reddit

Hey don't worry, it won't be your last loss.

I don't really have any great advice for the customer interaction other than verifying with whoever owns the customer account at your MSP how they want to handle the situation before addressing the customer directly.

Apologize profusely, be able to communicate improvements to help prevent this sort of thing ESPECIALLY including anything you guys have already proposed that got shot down.

You'll be alright

[-]

natariimei@reddit (OP)

Your added edit is really solid advice. I hope you were able to find another job, and one that can work with you, at that.

[-]

mrjamjams66@reddit

Most certainly.

In fact, it pays better and has better benefits. Which is the whole reason I left my job of 8 years for the job I was at for 2 weeks.

It all worked out in the end. Now I'm internal IT and my work load is less than half of what I'm used to. I often find this to be extremely boring but it encourages me to study and get my next cert, which I've been procrastinating about.

I'm already thinking of my exit strategy from this job, but not because I dislike it or because it's a bad job. I just like being challenged. It feels like a great place to be for my needs.

[-]