Anyone else finding “retired” systems that still power one critical report?
Posted by enterprisedatalead@reddit | sysadmin | View on Reddit | 27 comments
We were cleaning up some older servers recently and found a few apps that were basically still running because one scheduled report depended on them. The surprising part wasn’t the infrastructure cost — it was how many undocumented workflows were quietly tied to systems nobody had touched in years.
A couple of them had turned into “do not touch” systems even though almost nobody fully understood the dependency chain anymore.
Curious how common this is in other environments.
SkyrakerBeyond@reddit
There used to be an ancient apple II at one of our clients that handles the access control system because it was made in the 80's and the company went out of business. We now have its software running on a VM on a modern machine, but for decades it couldn't be touched lest everything break.
TKInstinct@reddit
It's an access control system, I'm no expert here but how much does that cost? I feel like at some point someone would bite the bullet and just do some janky shit for a week or two until they got a new system installed.
jihiggs123@reddit
depends on the size of the building. might need new wiring, new card scaners, door lock actuators, motion detectors, it can be a pretty hairy rabbit hole. whether or not its "too expensive" depends on which budget its coming out of and how cheap whoever controls that budget is. then you get into a pissing match between facilities and IT, each telling the boss it needs to come out of the others budget. its usually a huge shit show.
d00n3r@reddit
FundEz.. and fucking Lotus Notes. LOTUS NOTES ffs. Some goon decided that was a good place to store legal documents once upon a time.
stickytack@reddit
About 6 years ago we got called into a large construction company to replace a server of theirs (Active Directory, file services, Quickbooks). They had two existing physical servers, one was "old and retired but they said to leave it on" and the second was a Server 2012 machine running AD, file shares, and quickbooks. When we started digging around we discovered the OLD server (windows 2003) still had all of AD's primary roles and one file share that was used by one person. We created three VM's on the new server, one for AD, one for file shares, one to run quickbooks desktop. Migrated all of the files, migrated AD roles, decommissioned the 2003 machine, decommissioned the 2012 machine, and finished up the job. A few weeks went by.... they call us in a panic because they had a bad waste oil tank overflow and the service company for the tank was freaking out asking why the old server had been decommissioned because the tank's level sensor was somehow connected to that old 2003 machine and since it was off there was no warning that the waste oil tank had filled up. We went and plugged the old win2003 server back in and the sensor system came back online. Today, that server is still plugged in and turned on, and we're still waiting for that company to change out that sensor system to something that doesn't rely on a 20+ year old server.
Enough_Fall_3127@reddit
are you new or something? this happens on the reg
heydatsmyrice@reddit
We still use AS400 for some sort of data used by certain departments. I got into technology department in Nov 2007 and they were talking about getting rid of that system, almost 10 years later, it is still around.
geekworking@reddit
Part of the reason why scream testing is part of many orgs decom process.
AdultContemporaneous@reddit
What is scream testing? Unplugging everything and waiting for tickets to roll in for some random thing?
tankerkiller125real@reddit
Correct. Depending on the org the scream test might last a full year (because EOY reporting stuff), or might only last a few weeks.
Hauber_RBLX@reddit
Basically that. Plug something out or turn it off and see who "screams" (aka complains) first
i7xxxxx@reddit
things like this honestly give me anxiety at work. the longer a system sits untouched the higher the chance it’s gonna break when you do need to touch it.
hate it lol and yes we have some too
noodlyman@reddit
I find the trick is to make sure it's in writing that the system is out of support, and you'll do your best, but are unable to guarantee recovery in the event of a failure. Pass the buck up the chain, if you can.
tomster2300@reddit
The answer is the world’s critical systems all run on 60 year old spaghetti code.
NightMgr@reddit
It’s been a long time but we had a sacred report that legislation demanded be sent on thee Sacred Floppy.
oceanicpanopticon@reddit
AI post
matroosoft@reddit
That's why every server or system has an owner. The owner knows how or why it's used and if not, it can be shut down.
Bendy_ch@reddit
Been there.
A Client had an ancient Desktop computer that held some obscure batch script that ran every hour. It was imperative that the script remained undisturbed because… reasons.
Nobody at the clients knew why, only that the sky would fall if it didn‘t. We were never allowed near it to examine it, but my best guess is that it talked to some similarly old industrial control systems.
TKInstinct@reddit
That seems like a common theme in threads on this board. The machine costs millions to replace, the vendor went out of business 10+ years ago and has some weird dongle or card on it that cannot be virtualized.
IAmTheM4ilm4n@reddit
I built the hardware and OS for a financial system almost 20 years ago; it was last updated around 2012.
I recently heard that the last of its' data was finally converted to the newer system and the old was one shut off.
It's probably a lot more common than expected.
knightofargh@reddit
I’ve consulted with fed TLAs and work at a large bank.
The answer is common.
Candid-Molasses-6204@reddit
The amount of Windows 2003/2008 servers and Mainframes running everything from earnings to payroll is just crazy. The ROI is awesome but the risk is off the charts. Like I think there's like 2-3 companies for stuff that old but good luck actually protecting it.
iPlayKeys@reddit
Instead of “do not touch” they should just label it what it is: RISK.
ddrive01@reddit
if it works dont touch it
IlPassera@reddit
Until it catastrophic fails and now nobody knows how any of it works and vendors refuse to touch it because the software it uses went EOL a decade ago.
bitslammer@reddit
Dealt with a lot of old decrepit systems in my past but they were never considered "retired." They were usually connected to some very unique or expensive device like a mass spectrometer which ran OS/2 in 2001 or a Win95 machine in 2010 that was attached to a $5M offset press. Also had to deal with a lot of medical devices and OT stuff when I worked in healthcare and manufacturing. Far more common than many may think.
Honky_Town@reddit
"This information is classified" at our work place since ages