When is it safe to update? Sort of about the 2025 stealth upgrade.
Posted by Dazionium@reddit | sysadmin | View on Reddit | 20 comments
So, this is really a procedural, or workflow question, but when do you know it is safe to update a server?
This question is directly spawned by windows servers automatically updating to 2025 issue, although I've personally dodged that bullet with my 2019 server.
I'm a solo sysadmin for a small (\~50 people) company with a single Windows 2019 server for the accounting program and basically everything else in the microsoft cloud.
As such, I go in a couple times a month to the server and manually update it through Windows Update, nothing automatic or any 3rd party programs.
I am now going to delay that update until next week in order to make sure the 2025 stealth upgrade won't happen as I assume Microsoft will have pulled the update by then. (Reading other posts it sounds like they have already pulled it.)
But that made me realize just punching that update button isn't the best procedure, so:
-How do I know this patch has been pulled and it is safe to update?
-Going forward, what do I check (news sites, Microsoft news feeds, something else?) to make sure it is safe for me to push that "Update" button on my server?
This is one of those no-right-answer questions I realize, and my specific use case is hampered by the fact that I am not going to get any budget for purchasing an additional management program of some sort.
Having said that, it is still an intriguing question and I'm interested to hear other opinions on the subject.
hoeskioeh@reddit
If you only have one single server, do it manually. Document the process in case you are on vacation at patch time.
Wait for joshtaco to finish. Add a few days to be safe.
Always monitor the Patch Tuesday Megathread in this subreddit...
joshtaco@reddit
we've been fine since we did our monthly patching literally the night of. Seems like the 2025 upgrade issue happened afterwards from what it seems?
jdptechnc@reddit
When /u/joshtaco posts next week that he accidentally upgraded 5000 servers to Server 2025, you will know that it is time to slam on the brakes.
joshtaco@reddit
we've been fine
ZARSYNTEX@reddit
As I know some guys clicked on install and ignored the license and upgrade warning.
This is not an normal update it is an upgrade! So it is not for free. If you have no SA, you must buy new Windows server licenses and CALs....
Vel-Crow@reddit
It is categorized as a security update, and you don't get the prompt when approving it in a patch management system. Can't remember the last time I connected to a server to do updates.
RCTID1975@reddit
No, but it was clearly named 2025 upgrade. Why are people either blindly accepting all patches, or have their systems to auto accept patches?
Don't get me wrong, this mislabeling never should've happened, but lots of people's bad practices are coming to light and rather than change those, they're just pointing their finger at MS.
fatbergsghost@reddit
An upgrade, that they want people to agree to new terms and licensing on, should be nowhere near an automatic patching system. It should not be possible to grind through the patching system because there should be no way of not knowing that the licensing terms were agreed.
If it can be, I wonder whether there are legal shenanigans where we can all safely ignore the new terms, because we were forced into an upgrade we didn't want, and therefore never agreed any new terms. Microsoft may or may not be able to prove it was agreed afterwards, but this should probably render their legal terms ineffective. Also, those legal terms involve financial requirements to purchase the product. If those terms were never agreed, then what financial duties do you have to pay for Windows Server 2025?
Zenkin@reddit
I think that this is the KB, and it does not say anything about an upgrade. It doesn't even say anything about a Server OS.
RCTID1975@reddit
My fully updated server 2022 build 20348.2762 is version 21H2.
The fact that says 24H2 should be giving people pause if they're actually reviewing updates before approving them.
Lots of people grasping at straws to avoid any blame here.
Just fix your process
Vel-Crow@reddit
It was not labeled as a 2025 upgrade anywhere. It may be updated now, but it was a security update by name and category, no mention of 2025.
Deviathan@reddit
My understanding was it's miscategorized as a Security Update.
Bright_Arm8782@reddit
You update in rings
An initial ring of a couple of devices just to make sure that the deployment works.
Then a wider ring of more devices that aren't that important.
Then everything else.
Auto deploy for the initial ring and approvals required for the others. 0, 4 and 10 days ought to do it.
These are low delays because I've worked in environments where I've had to deploy within 14 to retain accreditation.
Reynk1@reddit
My work, soon as all the mandatory apps support it be hardened and published for teams to use (backups, security etc.)
Vel-Crow@reddit
Wait 2 weeks after release. Others will tell you their issues.
Test the updates. Build a lab environment tha at least has the OSs in your production environments. Slap it on those first, check the results, then deploy to production.
Cormacolinde@reddit
Only two weeks? You’re mad.
godspeedfx@reddit
You could use something like Automox which is a very cheap patch management and automation solution. Simple agent install on your server, and you can set up a policy to only run updates 2 weeks after they are released. It can be a lot more granular than that, but you get the idea. I use it for 23 servers and it's only about $100 US per month. Price is per server so it would be far cheaper for you. Good reporting, nice dashboard, easy peasy. Highly recommend for small / medium sized environments.
gandraw@reddit
The way I usually do it is to run an ADR in SCCM like 2 days after patch day, then publish those updates to be installed 5 days in the future for test systems, and 12 days for production systems. That's quick enough for most months (unless there is a super nasty 0 day being patched) but long enough to not fall for the major Microsoft fuckups.
godspeedfx@reddit
This is what I do as well.. seen these vendor caused issues come and go for years and never had to remediate.
BlackV@reddit
you don't, not really know for sure
/r/sysadmin here is a good source, waiting 2 or so weeks is a good step
its more of a general browse random IT sites and end of the day hope for the best when you do click that button