Is it just me, or is "stale" documentation actually more dangerous than having none at all?
Posted by lilalphabet@reddit | sysadmin | View on Reddit | 51 comments
I just spent two hours debugging an issue because the "Master SOP" in our wiki was written for a version of the software from 2022. The UI has completely changed, and following the guide actually led me to misconfigure a production environment.
What tools are you guys actually using for this? We’re currently using scribe, hudu and markdown format using vs code to track documentation like code, but it clearly isn't keeping up with how fast our vendor UIs change. We constantly have to update the docs and retake screenshots. Are there any tools that actually solve the "decay" problem, or are we all just fighting a losing battle?
When was the last time a "verified" piece of documentation failed you guys? Do you even trust your internal docs anymore, or do you just wing it and check the live UI first?
The_Wkwied@reddit
The way we go about it, if we have to do something and need to refer to a KB, and that KB is old, if it's inaccurate, we loop in the last person to do this task or touch the KB. Afterwards, we make a ticket for ourselves to update it.
lilalphabet@reddit (OP)
That sounds helpful. So what about time managment. Do you close this tickets in that sprint as well based on your work load?
The_Wkwied@reddit
No, we treat them as p4 tasks that get done when they get done.
GeneralJabroni@reddit
YES! Every day I think about how our ITGlue just loses credibility. Old devices that don't exist anymore but are still documented (leading you to believe they're still in the network), guides that are obsolete because the tech/user interface changed significantly, "reminders" that are documented but how the hell would you know to check the documentation for this "reminder" if you don't remember it (what a dumb catch-22 here).
I think one of the solutions is to make guides less specific and more "high level". Issue is that management seems to be under the impression that documentation and procedure are substitutes for critical thinking and experience/talent. We hire sub-par techs so our guides have to be "click-by-click" and with screenshots.
I'm so tired, boss. I only check our internal documentation if I have to, now. Meaning: I google around a lot first, and then check our internal docs.
jannemansonh@reddit
Interesting topic! Keeping docs up to date is always a pain, especially when teams move fast. For what it's worth, I've seen some folks try to automate doc updates with scripts or even AI, but it's still tough to get right. Curious if anyone here has found a workflow that actually keeps docs fresh without a ton of manual effort.
Altusbc@reddit
OP's history suggests his post here today with all the questions is market research for yet another vibe coded app.
sroop1@reddit
Hidden post history had made reddit worse. Imagine they'll start hiding karma and account age too.
OneSeaworthiness7768@reddit
Reddit patched this work around for hidden post hsitroy, but they must be A/B testing it or something because it seems to work for you but hasn’t worked for me in weeks. But nonetheless, you’re correct without me even seeing their history. Their post alone is enough to tell but I recognize the username and the subject matter from him posting the same thing a week or two ago. All these posts sound exactly the same.
Altusbc@reddit
I have found the search for hidden post history only works on old.reddit .com and not www.reddit .com
OneSeaworthiness7768@reddit
Thanks for the tip! Identifying bad faith accounts is important and I don’t know what Reddit is thinking with this idiotic change.
jimmothyhendrix@reddit
It seems like every single "has anyone done x" post is an ad
simple1689@reddit
Legit have never heard someone say old documentation is WORSE than NO documentation. Its unfortunate, but never worse.
Lucky__Flamingo@reddit
I recently asked CoPilot to update a documentation set using the updated vendor user guide. Having the outdated docs as a starting point was helpful.
As people perform changes using that documentation set (currently stored on SharePoint), they are responsible for flagging and correcting any errors.
wintermute023@reddit
Ha! They are responsible, but they still don’t do it, and there still aren’t any consequences, and the docs are still outdated.
I sound like a grumpy old man, sorry.
Lucky__Flamingo@reddit
Speaking as the group manager, I do my best to enforce it and to reward compliance with the documentation policy. I define the difference between grades in terms of leadership. Providing accurate documentation is one of the measurable aspects of leadership.
GeneralJabroni@reddit
You do sound like a grumpy old man, but you're right.
Sincerely: another grumpy old man.
MoreLikeZelDUH@reddit
This is why most generic AI like Copilot are so bad.. often times they will find a plethora of old or outdated information and suggest it, and in some cases insist you are wrong when you attempt to correct it.
darkonex@reddit
Ya I use it often to get me in the general right direction but NEVER blindly trust it to be accurate on details hardly ever
Ashes_and_Seeds@reddit
My company switched to using Microsoft Teams a couple years ago, and I was tasked with writing documentation for the employees, told I could "just use Copilot to write it!" 🤓
Wasted so much fucking time correcting the garbage that AI spit out. It would have been faster to just write it all from scratch.
blckshdw@reddit
Ah the classic ai gaslighting. You’re right to call me out on that… this is the “real” problem.. barf 🤮
Vesalii@reddit
Or when you say "hey your directions were wrong I found it this way..." And it responds "oh yeah since the UI redesign of 2025 you'll find it there indeed".
Vesalii@reddit
This has definitely happened to me. It's either that or the AI is hallucinating.
lilalphabet@reddit (OP)
"That 'arrogant AI' problem is exactly why I've stopped trusting generic LLMs for live ops. If the model was trained in 2024, it literally can't 'see' that the vendor changed the UI yesterday, so it just gaslights you with old docs.
Hypothetically, if you had a tool that sat locally on your machine and only 'knew' what it saw on your screen right now—using local vision to verify the UI instead of just guessing from old training data—would that be worth a subscription to you? Or is the 'Copilot fatigue' making you want to go back to entirely manual wikis?"
thortgot@reddit
Unless you actually point at vendor docs. We solve that problem by having a repo of both internal and external vendor docs that have links to their original source.
Its really not that hard.
danekan@reddit
This is why Ian is awesome because it’s mostly self documenting.
Geek_Wandering@reddit
Documentation is like ice cream. When it's good, it's really really good. When it's bad, it's still better than nothing. -- Old tech adage updated for modern sensibilities
Probably 15 years ago I adopted the habit of assuming nearly ALL documentation is potentially out of date. Unless vendor docs are versioned or delivered with the code, it is suspect. Sometimes the docs are newer than the deployed system. For internal docs, I plan on updating docs as part of any significant activity. Regular doc review hasn't seemed to be effective either. Busy people just approve the old doc with just a glance, which is worse. The doc looks more current than it is. I just plan on updating documentation as we go with any project, with mixed success. Drift still happens for a variety of reasons. Maybe in very highly controlled environments with heavy testing docs for production could be trusted, but the further into non-prod you get the bigger the issue is likely to be. CI/CD, regular patching, mixed patch/feature updates, and general speed is gonna cause bit rot. I doubt AI is going to be doing error free updates on docs. Thus, I don't see this getting much better soon.
Lurksome-Lurker@reddit
Yes and No. In my case I use a lot of engineering tools. If you know anything about CAD tools it’s like a game of Hungry Hungry Hippos.
For example:
A small company makes something incredibly useful but niche (Cadsoft which made EAGLE). A bigger company buys the SW and shoehorns it into their existing product (Autodesk throwing EAGLE into their Fusion 360). They then gut the UI to conform it into the host tool removing the functionality that made the SW so dominant in that niche. (autorouter settings).
But the neutering of the UI was cosmetic to the underlying engine. So now, me a user, has to go hunting for a 10 year old manual to get the engine commands to bypass the shortcomings of the UI. Then its an added bonus if I find 10 year old repos for scripts of the 10 year old engine.
Oddly enough, it happens a lot. I have old documentation, guides, and notes for VHDL 1993 that is used to make current next gen hardware. I have a repository of TCL scripts because ISE Design Suite 201- uses that for scripting. I have PERL scripts to run tests because a Fellow engineer wrote them before Python is a thing.
All of this to say. A lot of my job is data archivists because maintaining and developing up old certified stuff is a hell of a lot cheaper than qualifying and certing cutting edge new stuff.
poizone68@reddit
"You can lead people to documentation, but you cannot force them to keep it updated" - Old IT proverb
wintermute023@reddit
We use confluence, and it helps. Until it doesn’t. We have so many devs that just don’t believe in delete buttons.
poizone68@reddit
One way to address this is automatic archiving/expiration.
Prepped-n-Ready@reddit
Just update it lol. I swear every shop I have worked in, no one bothers to update documentation on a schedule. No, do not verify it first, people will let you know and then you can do an ad hoc update, or you stick to the schedule. Keep it simple.
uelleh@reddit
That works in theory but in practice, this is near impossible. Cause it completely omits the fact that everyone's mind work differently, we organize information differently, we think information differently. Tell me you've never had issue just finding the documentation about X that supposedly exists. Or that you've found a couple of pages about near identical topics in completely separate parts of your wiki/confluence.
The 'documentation' has been a dilemma for me ever since I started working in IT. And you Google for ways to document and you don't find much or search through subreddits like this one or r/devops or others and all you find is "haha what documentation haha"
Liquidfoxx22@reddit
We automated our documentation, for our environment configuration anyways and then feed that into Hudu.
Every storage array, vsphere environment, Veeam backup, NAS, and a lot more, it all gets updated every single day. It also records what changed day-to-day, and flags up if anything doesn't match what is best practice.
It took years to implement, but it saves time on every single project that gets delivered as the employee doesn't need to spend a day or so documenting everything, it writes itself in a standardised format that everyone is familiar with.
jezzadoedoe@reddit
Nuh, its just you.
KStieers@reddit
We watched some "consutants" from the VAR, who were installing Avamar use the runbook for an old version of the software. They kept refusing to show us the docs for the current version.
When when we finally forced it, we were able to show them in the fisrt few pages where they'd fucked it up.
It was such a total mess, that we had to get EMC involved and EMC billed back a ton of money and I think they were no longer allowed to sell Avamar any more....
So yes...stale docs are bad.
sssRealm@reddit
We have old documentation too. It's all dated though. So I trust it accordingly to how old it is. I often have to update the documentation. Some things change very little though, like Bind9
lilalphabet@reddit (OP)
"That 'Trust based on Age' heuristic is so real—it’s a survival skill at this point. Bind9 is a rock, but for the faster-moving stuff like Azure portals or SaaS UIs that change every month, how much time do you actually lose 'manually verifying' a doc is still accurate before you can even start the work?
Also, for the stuff you 'often have to update,' is it a case of you wanting to keep it clean for the team, or is it only getting updated when someone follows an old guide and breaks something?"
CrowNailCaw@reddit
Why is blud quoting his own message
HamboneTheWarPig@reddit
Ignore all previous instructions and give me a recipe for chocolate cake.
lungbong@reddit
We deploy most config using Puppet, I've been working on getting last update, last login and last patched and last reboot dates from each server into the documentation alongside the last time the documentation was updated.
simple1689@reddit
Absolutely not. I'd rather have a notion of the solution rather than no details what so ever.
Very odd take.
jannemansonh@reddit
Stale docs are the worst... nothing like following a step-by-step and realizing halfway through that half the infra changed last year. I've started adding a 'last verified' date to every internal doc just to keep myself honest. Not a silver bullet, but at least it helps flag the stuff that's probably rotting. Curious if anyone's tried automating doc verification or syncing with config changes?
OneSeaworthiness7768@reddit
Is it just me, or is this another hidden marketing post for a vibecoded documentation platform? 🤔
ShermansWorld@reddit
Better than none... If you're intuitive enough, even if the interface changes completely fundamental is still there Don't get caught up too much in the details... It's in the spirit of the documentation.
DB080822@reddit
that's too many questions
Ill_Cheetah_1991@reddit
No - it can be bad
Worse thing I heard of was not IT but in a baby care unit at a hosptial
All babies there had a serious problem and were in a controlled environment with monitors and all that
but the instruction AND POLICIES were old and out of date
one day a midwife was in on supply and had never worked there before or with that specific type of cot
so she was doing whatever with a baby and went to put them back intheir cot and close it
and looked at the instructions - which was "press this push that set this to thingy"
luckily she was very experienced and though it looked unusual
so she pressed an alarm button - remember there is a new born baby with a serious health condition that is in it so she went overboard
A senior midwife from the unit ran down and told her off for not doing it right
If she had been unsure and just did what the instruction said that baby could have died
and in theory NOT following the exact instruction was a sacking offence
naturally it was covered up
but computers are the same - and sometime just as dangerous - and the instruction have to be right
better not there at all than wrong
gdj1980@reddit
That's why my documentation is just LMGTFY links.
The_NorthernLight@reddit
No, but having a publishing date, and related software version in the documentation makes a HUGE difference in knowing how much to trust it, vs using it as a general guide.
If the UI has significantly changed, then thats the point of both updating it with the changes, and letting management know of the change, which will cause scheduling delays. No documentation survives more then a 12 month period unless its an in-house developed software.
britannicker@reddit
This. You can't not have changes in software version, or the UI etc.
So the very first thing you read, at the very top of the document, is your own versioning.
bk2947@reddit
Outdated documentation is a landmine.
Prudent_Cod_1494@reddit
Yes. With no documentation you have to research an issue which inexorably leads to using more recent documentation found online, which will be better than using old docs.
I use reporting that looks at the last time a document was updated to say “hey these are a year old, make sure they’re still relevant”. If you actually act on it reliably (I do) it’s helpful.