It's really nice when money is no object, only deadlines.
Posted by GallowWho@reddit | sysadmin | View on Reddit | 43 comments
I support a product that's basically the Pied Piper Box, it needs a hard drive replacment. The other company that server maintenance has been subcontracted to out of OEM warranty told me today they'd need to order a new drive.
Figured it would take a few days to arrive but it is what it is. Nawh, I just got a email with a tracking number before EOD. The harddrive is being Fed Ex'd overnight to the data center so no MW is going to be missed this week.
Overnight shipping probably cost more than the harddrive.
Responsible-Slide-95@reddit
One of our VMHosts shit itself because the cache battery on the RAID card died. Spent the day migrating the more critical VMs to other hosts with spare capacity while a call was put out to our MSP to get a replacement battery shipped. After spending a not inconsiderable amount on shipping to get it same day the package arrived and we opened it to find
A CR2032 coin cell battery.
Needless to say, we didn't renew the contract with that MSP
Templar1980@reddit
We have full vendor support on one of servers. One day I’m sitting on my desk I get a tap on the shoulder only to be handed a package with a new HDD that was overnight shipped. The Server called home to report a predicted drive failure and they dispatched a new one before I was even aware.
No_Investigator3369@reddit
Ahh so you also work at a place that has no problem paying $200k invoices.
......but hire an additional $120k body? ARE YOU OUTTA YOURE MIND!!!!!
hornethacker97@reddit
Capital versus operating expenditure. Makes all the difference as one is tax deducted and the other isnt
matt11126@reddit
I like dell, they overnight you a shipping box for your broken laptop. Then they overnight the laptop to their service center, and then they overnight the laptop back to you.
Even better, they send a tech to your workplace, oftentimes the next day, to replace whatever part you are having issues with.
FireLucid@reddit
Had this happen with I think Acer? Either way, the service centre was a 2.5 hour drive away. Guy brought the wrong part, had to drive back and get it and came back later that day. Don't think he closed any other jobs that day.
rosseloh@reddit
Except I can do the replacement faster than their tech can, and I don't have to listen to him waffle on about right-wing politics while he's sitting in my office either. Or when it was a hot swappable server disk the one time - I had to escort him through the factory floor anyway, I was standing there the entire time he was working on it. Might as well just send the part to me and I'll do it...
(I live in a fairly rural area, we have like...one guy who answers those calls from Barrister [who Dell contracts with at least for my region]. It's always the same guy.)
malikto44@reddit
One thing that is nice about Dell. If you have a big PC shop where you have techs on site, you can get them Dell certified. That way, they can dispatch parts, do the work there. Dell even sends a small amount of money because they didn't have to send a tech out.
This comes in handy in areas where you are handling confidential stuff, and the amount of paperwork to get anyone inside is enormous.
This, plus having spares, plus the fact that if a tech can't fix it immediately, dump it in the mail and have Dell's depot fix it in 6-8 weeks is a good backstop.
MrYiff@reddit
iirc if you are big enough Dell will even setup a spares store on your site so you would have instant access to parts for your most common kit.
GallowWho@reddit (OP)
I had a Dell Tech come to my home and replace my laptop keyboard and trackpad at my dinning room table. Dude even remembered his ESD strap.
whodywei@reddit
Storage is cheap, availability is expensive, you are paying for the availability.
KickedAbyss@reddit
I've seen same day plenty of times on under warranty stuff, but I've also seen Parkplace rush ship international... I don't want to know what it costs to rush ship from California to the EU but I'm sure it's not cheap.
mtgguy999@reddit
“ Overnight shipping probably cost more than the harddrive.”
Yeah but the support contract probably cost more then your salary
cats_are_the_devil@reddit
Exactly, the SLA on that drive replacement doesn't give two shits about the cost...
xendr0me@reddit
Our Dell 4 hour response warranties on PowerEdge servers runs around $1200 on a 5 year warranty purchase, and it's worth every penny to have a replacement drive couriered to your location when a RAID is sitting on a hot spare and you are just hoping nothing else goes wrong.
DheeradjS@reddit
1200 a year or 1200 for 5 years?
Either way, that's a relative steal.
DaemosDaen@reddit
Dell warranties are pretty cheap in general. Even on the first renew. It's that 3 and so forth that you really need to pay attention to.
Fortunetly we don't have to use them too often either. Most of our gear has issues outside the window. For servers it's HDD several years down the road. For client systems, it's laptop screens.
Hotshot55@reddit
They're basically free when you compare it to a fully speced out server.
bayareaadmin@reddit
You would be amazed at the next day rates these vendors get. Don’t worry… the shipping wasn’t even close to disk drive price.
GallowWho@reddit (OP)
Where the lie 😭
trueppp@reddit
Imagine when you have 4-Hour onsite warranty for your server....Has a Lenovo tech come out at 2AM to replace a raid battery.
TheJesusGuy@reddit
What actually happens if they don't meet the SLA though. Its not like you can do anything about it. I think my Lenovo gear is next business day but these are extended warranties.
jebuizy@reddit
If they breach SLA they owe your company money based on the terms of the contract. They also probably have a bunch of meetings with angry SVPs. You can't do anything about it, but your bosses can definitely make their account team sweat over it
MSXzigerzh0@reddit
That's impressive 👏👏
trueppp@reddit
No not impressive, just expensive.
rosseloh@reddit
You can do just about anything with sufficient manpower, money, or both.
I still occasionally go watch that youtube video of when a rail line in Tokyo switched over from one station track set to another in the ~6 hours of overnight closure (because that's the only time they can, they have millions of people relying on their service being there during the day). It looks like swarming ants.
saysjuan@reddit
Guess i’m spoiled all our systems have 4 hr onsite parts and labor warranty. It’s great when they don’t need an escort and their already badges for the site.
adamphetamine@reddit
I used to do support for a large medical systems company. I asked their support guys once what was the cheapest thing they sell-
'probably a maintenance contract, but if you are under contract we'll charter a helicopter to get an engineer and a part to a remote hospital'
Absolute_Bob@reddit
Overnight is considered a slow response for anything critical.
Assumeweknow@reddit
My refurbished servers get the same service. But i usually keep drives on shelf.
whetu@reddit
Way back in my helldesk days, I received a late Saturday night call from an oil rig off the coast of Taranaki here in New Zealand, which at the time was run in part by Shell. They had a rack of servers on the rig and one of them had shat itself. Probably a Compaq DL380 G1, as this was definitely prior to HP purchasing Compaq.
Bit of basic diagnostics later, I escalated to the on-call engineer. He agreed with my conclusion that the RAID card had died and went about organising a replacement.
Compaq woke someone up, sent them to a warehouse to get the part and deliver it to the engineer at the airport. He had a chartered flight - just him, a bag of gear and the RAID card to an airport closer to the oil rig, where he caught a connecting chartered helicopter flight.
All this in the middle of a Saturday night. Funny how Shell Oil money makes things happen.
The next time that engineer was on his on-call rotation, I happened to call him up and asked the obvious "hey by the way, how'd that oil rig job work out?"
He fixed the problem and Shell decided to just leave him out there until the next scheduled helicopter rotation. So he was paid his on-call rate to stay on the rig for a week playing pool and arcade games. Pretty sweet deal!
tacobowl8@reddit
From what my dad, who used to work in the oil business told me, even an hour of oil production/extraction being down is a HUGE amount of lost revenue, like on the order of millions back in the 80s.
So Shell Oil paying the big bucks to have all of that arranged actually makes a ton of sense for that use case.
GallowWho@reddit (OP)
That's a great story, my initial thought was "stranded on an oil rig for a week would suck" but for everyone else its just another day.
Awkward-Candle-4977@reddit
Next time buy storage array that includes empty cradle in all slots. I see super micro, tyan etc. sell this kind of storage array.
Then add samsung, wd etc. server ssd.
Buy them from Authorized distributors or resellers to get manufacturer rma service
wild-hectare@reddit
during the 2010 GoM oil spill....i approved orders for 100+ side-by-sides (ATVs)
yes...i was one of the many "IT People" working the incident
GallowWho@reddit (OP)
That's a production outage I'm glad to have not been on the bridge of
theducks@reddit
I once needed a replacement PRAM battery for a Sun server.. I couldn’t take it down to look, and it was listed as a CR3032.. which I couldn’t source locally, so I bought it from Sun for like $200.. Turns out it was a typo and was a CR2032 which you can get at any corner store. But they packed it well at least
GallowWho@reddit (OP)
Scammed 😞
LincolnhamLincoln@reddit
Had a stick of RAM go bad in a server. They FedEx’d the new stick. Then I got an email that it wasn’t going to be there within the SLA so they sent another stick by courier. Thing is it was going to a colo and the tech wasn’t scheduled to be there for another couple days.
GallowWho@reddit (OP)
At least they tried, sucks when it happens though
Either-Cheesecake-81@reddit
My SAN has NVME solid state drives that cost $20k each. It would have to be overnighted from Japan for the overnight shipping to cost more than the drive.
aguynamedbrand@reddit
The best is when the part is guaranteed to be onsite within 4 hours.
Zazzog@reddit
The other company's contract almost certainly more than makes up for that.