So I’ve been working at my company for a few years now. It’s a 70-year-old family-run business where people tend to stick around forever — some employees were even hired by the original founder. As you can guess, the general IT knowledge is... let’s say “historic.” Combine that with a “less text = less important” mentality, and you occasionally get monumental screw-ups. Like the one I’m about to share.

We manufacture sensors, and those sensors need to be calibrated. Originally, this was a fully manual process. About 15 years ago, it was “semi-automated” — because why let a computer do something faster and more accurately when a human can take longer and introduce errors?

Fast forward to today, and new ISO standards require full traceability. So we developed a new piece of software: it connects to the CRM, interfaces with lab equipment, controls testing, and logs all changes. Basically, it replaces a Frankenstein system made up of Delphi 5, Excel sheets, and handwritten notes. A big improvement — on paper.

Two weeks ago, our CRM provider announced an upcoming update: database changes, module tweaks, new rules, etc. Alarm bells immediately went off in my head. I warned my manager, the lab manager, and the quality manager.

A week passes. Nothing happens.

Three days before the update, I follow up again. Their response? "The changelog isn’t that big, so we think it’ll be okay. Bigger changelog = bigger changes = more risk. This one’s small, so it’s probably fine." I tried to explain why that logic was flawed, but they didn’t want to hear it.

Then the update hits. Cue total panic.

The lab software breaks. Nothing works. Chaos ensues. "Why didn’t anyone warn us?" "How do we fix this?"

My coworker and I remind them — politely but firmly — that we did warn them. Multiple times. We suggested a dev server. We suggested delaying the update. We even pushed for an SLA with the development agency. But that €15k SLA was “too expensive.”

Now? The company we hired to help with the project is unavailable for a month. So the current “solution” is to have two of our highest-billed employees (both normally booked at over €500/hour) manually patch the database as needed — work that should’ve been automated.

I want to believe they’ve learned their lesson. But deep down, I know they haven’t.