SSL certs
Posted by Intrepid_Evidence_59@reddit | sysadmin | View on Reddit | 233 comments
Is it just me or does anyone else hate renewing ssl’s. Like I have done it over and over but every year I get anxious about it. Then once it’s over I pounder why it stresses me out. I’m coming up on a couple of our annual servers and I’ve been dreading this month. Every July, September, and December I do this but yet I am stressed.
Resident-Artichoke85@reddit
Automation or use an internal CA.
For internal-only access where we have control of the client devices (to push our own Root CA and CRLs) we use very long Root CAs (100 years) and very long end-device certs (20-50 years, depending on device). The idea behind this is two-fold: We want to install internal-only servers/apps with a "set it and forget it" certificate that will work even when technology moves on, but yet the server/app won't support newer crypto standards. Second, what danger is there in using long certs so long as we use CRLs and revoke any old certs? Our Root CA is offline/powered down except when we need to issue a new Sub-Root CA. We cycle our Sub-Root CAs every 5 years, but keep them in our certificate store issued to clients so end-device certs will function indefinitely.
N0Zzel@reddit
I tried to get my org to implement ACME but networking wouldn't give us the keys to the DNS records so we could do the DNS challenges
OhioIT@reddit
There's still HTTP challenges with ACME that work fine
N0Zzel@reddit
Those don't work so well if you have a firewall between the acme server and your clients
I work in manufacturing so basically all of our shit is behind a firewall and can't access the internet
Caldazar22@reddit
As a junior, certificate-related tasks bothered me until I spent a few days reading through the mechanics of the underlying algorithms: the X.509 format, Diffie-Hellman, RSA, and SHA; there was no EC at the time. Once it stopped being a black box to me, the anxiety dissipated.
occasional_cynic@reddit
Pray FIPS never comes to your organization.
mmzznnxx@reddit
Everything being inaccessible is technically FIPS-compliant though, right?
mkosmo@reddit
FIPS-validated crypto isn't all bad. It's just a pain when your Windows desktops have to run in FIPS mode.
Cheomesh@reddit
That's always been the case in my environments - only thing I remember not working right is Adobe not being able to use certain older form templates.
mkosmo@reddit
There's enough that doesn't work right with FIPS mode that even DCMA doesn't bat an eye when you say that you don't have FIPS mode explicitly turned on, despite -171 3.13.11.
Cheomesh@reddit
I mainly remember running into issues with it when it was applied as a STIG requirement. That was my first encounter with a technical implementation and it would rear its head in the strangest places.
Cheomesh@reddit
Why's that?
skreak@reddit
It has come to mine and it's nothing but a god damned headache. We've even had to have vendors change database access schemes and send patched software. There are some drivers that we need to recompile from time to time (Mellanox) and the only way to do it is to turn off fips and reboot, recompile with special options for the rpm signing, and then reboot again. Total PITA.
ReputationNo8889@reddit
Same here, if you read up on certs you realize they are not really complicated. Some IT guys still are amazed that i can convert one cert type to another.
JerikkaDawn@reddit
To me that's not the confusing part. Rather it's all the different file extensions and ways these things are packaged.
Caldazar22@reddit
You want the PKCS, specifically PKCS 1 (single cert), PKCS 7 (cert chain), and PKCS 12 (pub cert + associated private key). Don’t rely on file extensions; use openssl or certutil.exe to examine file contents.
Low-Okra7931@reddit
This is a solution to most things in the field. If you focus on understanding the subject a bit more deeply, instead of just solving the problem ASAP you can avoid this type of anxiety.
Lv_InSaNe_vL@reddit
I deal with this all the time with newer techs. They'll talk about how something doesn't make sense and it's dumb and frustrating and they just can't figure out how to make this easier.
"Did you read the documentation?" No, they never have. Give them some pointers and reading materials and then all of a sudden a few days or a week later it makes sense to them and it's not frustrating anymore!
idonthuff@reddit
Look at "certificate lifecycle automation" tools that work for both public facing certs and private (internal) pki.
SortaIT@reddit
i think also the more widely used term "certificate lifecycle management" is good too
some good solutions out there can solve OP's problem years ago
Carlos_Spicy_Weiner6@reddit
I don't mind doing them. Mainly because I charge an hour to do it. Does it take me an hour? Usually not.
What I hate is when people demand that they need one when they really don't.
I'm currently working on a problem that was created by a website guy who is demanding our method for streaming webcams to a website needs to be SSL.
The program itself doesn't allow for it and honestly we're just streaming motion jpegs to a website. He swears up and down that we have to have it cuz it's so hard for him to make one page that isn't SSL certified.
We've explored other options like setting up a dedicated machine with OBS studio to stream to YouTube and then link that over to the website. The problem is if our internet hiccups the system still continues to stream but YouTube stops the stream. So then we have to go into the computer. Stop and restart the stream. Go into YouTube. Get the new URL and embed it into our website. Versus our old way of streaming motion jpegs to a website that was Rock solid for multiple years and if anything ever happened, all we had to do was go to the streaming PC. Push the power button. It would turn itself off and then immediately turn itself back on and boom we were back to the races.
lordmycal@reddit
It's 2025. All http traffic should be retired as it's unsafe and subject to transparent adversary in the middle attacks.
Carlos_Spicy_Weiner6@reddit
Tell that to the people that made netcam studio and demand that they make an easy and reliable way to add an SSL certificate to it. I will back you up 100% and sing it from the highest mountain tops
Dal90@reddit
Put a proxy in front of the webcams that serves the SSL stream.
Browsers have been bitching about non-SSL content by default for the last four years.
narcissisadmin@reddit
This right here. An nginx reverse proxy will happily serve up https traffic from an http source.
Fast-Gear7008@reddit
They put the cart in front of the horse with certs there should have been an auto renew protocol in place before requiring renewals
EmploymentDry1696@reddit
No one is to talk about the SSL Fight Team!
SkyyySi@reddit
For public-facing web servers, using Caddy can make your life much easier. It can set up a fully-functional HTTPS reverse proxy in literally one line.
WDWKamala@reddit
Nobody tell him about the changes to the maximum lifetime of SSL certs.
general-noob@reddit
Shh… we don’t talk about that yet
kezow@reddit
That's future team's problem.
bananajr6000@reddit
What? 90 days from now? Shit! 30? What do you mean 14 days?
Aaauuuggghhhhh!
Sk1rm1sh@reddit
Tre_Fort@reddit
As a member of the CAB forum, I resemble this remark. Made me laugh.
itdweeb@reddit
Just renew it every day. Just at a random offset around a random time. Better safe than sorry.
Normal-Difference230@reddit
that sounds like a problem for Future Ted and Future Marshall
Intrepid_Evidence_59@reddit (OP)
Our forward facing web servers are only good for a year, phone system are good for 3, internal are set to 4 or 5. They all arent synced so no matter what I’m manually doing some of them every year. Majority are automated though.
mixduptransistor@reddit
The point of the comment above is that public certificate lifetimes will be dropping to 200 days in 2026, 100 days in 2027, and 47 days in 2029
Intrepid_Evidence_59@reddit (OP)
When did this happen?
Ruben_NL@reddit
Intrepid_Evidence_59@reddit (OP)
Great. Something to look forward too
Intrepid_Evidence_59@reddit (OP)
Just looked it up and you guys weren’t lying. Looks like I am going to push for automation for these.
ca1v@reddit
Digicert have an API if that’s the vendor you use.
mingepop@reddit
Sorry I’m a bit new to this, but how could you leverage Digicerts API to get this process fully automated?
Intrepid_Evidence_59@reddit (OP)
Digicert and GoDaddy. I’m looking to transfer everything back to digicert possibly if not another vendor that allows automation. From the sounds of it GoDaddy doesn’t. Not only that every year I have issues with GoDaddy.
snebsnek@reddit
The system has worked!
Tulpen20@reddit
If it was good enough for my pappy and his pappy before him, it's good enough for me!
FALSE_PROTAGONIST@reddit
Knightshadow21@reddit
Always make use of a crisis :)
yankdevil@reddit
And this is why it's being done because it should have been automated over a decade ago.
mixduptransistor@reddit
It's been in motion for a long time with browser vendors, mostly Apple, pushing for it for a couple of years. The organization that manages this stuff finally voted and agreed the new rules in April of this year, and will phase in starting next year
uptimefordays@reddit
Google has also pushed for these changes pretty hard.
Longjumping_Gap_9325@reddit
And I still haven't received solid info around what the DCV validity period actually means in terms of OV validated domains with our CA... but with all of our sub-domain certs using the OV validated off of our main, I'm hoping it just means you have a 10 day window to complete the DCV once started and not "The DCV is good for 10 days, and then any cert after that in the 47 day window will be rejected as not having a validated domain" would suck
Scared_Bell3366@reddit
Web browsers are getting super picky about certs. I had to cut my home internal ones down to 2 years. I’m automating them now, only a few left to do.
We can’t automate them at work. They also double as client certs for machine to machine stuff and that just adds to the stress.
PantlessAvenger@reddit
Better automate the web servers also. Every 47 days is gonna suck.
IJustLoggedInToSay-@reddit
Eventually they are gonna rotate in real time like the barcode of a mobile bus pass.
smoike@reddit
I have them on my personal hosting because of email and cloudflare. I've been dreading this coming up as much as I don't like paying a bit extra for cert renewals to happen automatically, those changes are going to make it look far more attractive.
goingslowfast@reddit
Certbot and Let’s Encrypt are a great pair and free.
smoike@reddit
I'm only self hosting the system tunnelled to via cloudflare, everything else is with my hosting co. I found out about Lets Encrypt when I had to set up cloudflare. No idea what I'll do next time I come up with cert renewal.
goingslowfast@reddit
Keep in mind that with Cloudflare tunnels, your data is transiting Cloudflare’s infrastructure unencrypted.
That may be fine for your use case, but consider that reality.
lsumoose@reddit
Well they have to be able to inspect the traffic for the WAF
goingslowfast@reddit
I’m not saying it’s bad or avoidable, just that it isn’t zero knowledge which isn’t often pointed out in setup videos about it.
dustojnikhummer@reddit
Use DNS challenge and an owned domain. You can have a trusted certificate in your LAN without being accessible from the outside.
pertymoose@reddit
*Laughs in 50 years of using the same SSH public key*
Stupid certificates and their stupid "trusted" infrastructure that no one trusts anyway so they have to pull stupid stunts like this
general-noob@reddit
lol, I have been screaming this from the roof tops at work and everyone just ignores me. F all then, you guys are going to get screwed
WDWKamala@reddit
Most things are easily automated, but those damn appliances….
Discipulus96@reddit
No kidding firewalls and network hardware is such a pita. Not all of them can be done via scripting. Thankfully many of them are at least starting to put letsencrypt functions in newer firmware.
snowtax@reddit
Automate your cert renewals now! Don’t wait!
ca1v@reddit
Shhhhh his blood pressure will be through the roof 🤣
Happy_Kale888@reddit
LMAO!
InevitableOk5017@reddit
It’s ridiculous it’s changing.
gorramfrakker@reddit
Shh, get back under the rug.
FullPoet@reddit
Why not automate?
Intrepid_Evidence_59@reddit (OP)
Majority of our environment is. It’s our forwards web facing servers that have to be manually done. Along with a couple of other devices.
SevaraB@reddit
Those are the best candidates for LetsEncrypt- rando web visitor #24601 is way more likely to have LE CA certificates in their trusted root stores than your internal CA cert. There’s no difference in security between them and Digicert when it comes to domain validation (DV) certs, either. You’re literally just paying for the brand name.
narcissisadmin@reddit
This is what I've been trying unsuccessfully to explain to the decision-makers. You go through more scrutiny to get an EV or OV certificate but the traffic is exactly as secure.
SevaraB@reddit
Yep. That’s when you get into cert EKUs and which EKUs are sensitive enough to justify the extra spend (like code signing certs, for example- you WANT to limit those to trusted CAs that you know are doing extra verification).
itsgottabered@reddit
Look down! Look down!
OhioIT@reddit
If your webservers are IIS or Apache, this can be automated for free. There are multiple tools that work with Let'sEncrypt's ACME protocol
Maelefique@reddit
It can be automated for free with nginx too.
Stosstrupphase@reddit
Are there still webservers that do not allow to automate this?
J_de_Silentio@reddit
Yes, we have one specific to our industry. I have to upload the cert/private key, then wait 30 minutes for the services to reboot.
I believe it runs on TOMCAT? Apache? Either way, has to be done through their shitting web GUI.
Stosstrupphase@reddit
That sounds like hot garbage.
narcissisadmin@reddit
Our keycard system had a self-signed certificate created and assigned upon installation with no way whatsoever to change it, outside of messing with the server files offline.
Stosstrupphase@reddit
That sounds even worse.
OhioIT@reddit
I've had to deal with Tomcat before, so I can understand that. To get HTTPS working, I ended up using Apache as the initial frontend, then redirected other folders to the Tomcat instance running on it. Was able to automate it the and it worked great until I retired the server
admiralspark@reddit
This sounds like vmware, specifically the tomcat garbage they had in Horizon.
Or the Tomcat server for that CMDB that utilities use...TOA? iTOA?
Or Futura anything. Lol.
dustojnikhummer@reddit
The underlying webserver can 100% do it, just the app built on top of it won't allow you to do it.
Maelefique@reddit
None of the majors that I'm aware of, there might be some tiny distro that doesn't.
symcbean@reddit
erm, if you can do REALLY BASIC scripting then you can easily do certificate provisioning and renewal across a cluster of apache, nginx, lightspeed and probably lots of other things too (I also do postfix certs this way). Its not rocket science.
dagbrown@reddit
Those are the ones you should be automating. Failure to do so at this point is simple incompetence on your part.
Learn how to do your job instead of whining about how hard it is.
Intrepid_Evidence_59@reddit (OP)
Just so you know from what I’ve read from other comments Godaddy doesn’t offer a way to automate cert renewal. I didn’t check to see if this is 100% true(but am doing so). I am not whining. I was just simple ranting lol. I truly love my job and everything that comes with it. I also mentioned below that I just took over my role and now am able to change the process of how we do things and have my IT directors full backing after a meeting today about switching to a Cert vendor that will allow us to automate the process especially since everyone is switching to basically a monthly renewal in the coming years. I only did this because of what other people in the post talked about. Instead of trying to bring me down they educated and gave me opinions and other options. I think your comment is irrelevant and just plain out ignorant. You are trying to bring another person in the same industry as you down. For what joy or because you have nothing else to do. You are the exact reason I almost got out of IT. Thankfully I ran into a bunch of people who showed me how amazing the community can be.
OhioIT@reddit
Most webhosts let you automate certificate renewal for free and provide an easy method automatically. GoDaddy is one of the very few that doesn't let you AND charges you money for certs
mixduptransistor@reddit
These are precisely the ones that should be automated. The public-facing, critical, disaster-if-they're-down systems should be the FIRST ones you automate so that it isn't a problem. You can't forget to renew, and if you've tested your automation you can't screw it up. (Of course you should still monitor and alert so you know if the automation breaks before the existing certs expire)
Scary_Bus3363@reddit
You cant forget to renew but your automation can break and God help you if you need help fixing it
WackoMcGoose@reddit
Or worse, your automation can be unplugged by a janitor that couldn't be arsed to find a different outlet for their floor buffer...
mixduptransistor@reddit
I mean if you know what you're doing and do it right, it should not take much to fix if it breaks. The key is simplicity
Also, monitoring is very important so you catch failures. Setup the automation to renew at 80% of lifetime so you have the remaining 20% to fix the automation
schmeckendeugler@reddit
Ask VMware!
seuledr6616@reddit
Anyone doing this with multiple sites in IIS? We have some web servers with multiple sites, some needing to be bound to different certs. Haven't looked into a bunch of options yet for automating this via let's encrypt, but the last time I did, options were limited.
narcissisadmin@reddit
Stuff the sites into the SAN.
OhioIT@reddit
Yes. Win-ACME works great for this. I've had it going for probably 5 years now
DueBreadfruit2638@reddit
You can do this easily and for free with win-acme. For web servers, you can just use HTTP validation.
dustojnikhummer@reddit
We use WACS (WinAcme) for this and store certificates for IIS in Certmgr
HelixClipper@reddit
Win-Acme (WACS) don't even look at anything else https://www.win-acme.com/
It's utterly brilliant. What I did at our org is for internal services generate a wildcard cert that gets saved off to pfx to a locked down central share then either use central certs on IIS, or for other services such as RDG and NPS used custom PS scripts to update the cert using the pfx from the share. WACS also includes a bunch of scripts that you can execute directly after renewal (it'll ask you during the first registration run through), or you can use them as examples to create your own which is what I did
For DMZ servers just use WACS directly on them and it'll just renew and update the bindings
In both instances I'm using DNS validation to Azure DNS, as there is a module you can install for automated Azure DNS validation (piece of piss to set up) then just did a CNAME or NS from our DNS provider for the fqdn it checks (can't remember what that is, docs on the wacs website explain the process) so it effectively delegates the request to Azure where WACS will do it's automated TXT record
FmHF2oV@reddit
Certifytheweb works great. Can use a variety of options with it. Central certificate store or use the program directly on machine.
fys4@reddit
Yep, been using CtW for years and well impressed with the service. Tech support is top notch (they're in AU so that might be a problem depending on your TZ) and very reasonably priced for what it does. I've had replies from tech support in the early morning their time and even on a weekend !
I believe it's posh-acme under the hood, but you can also use your own scripts or use predefined tasks to handle any renewal I've come across so far
No links to CtW other than as a happy user !
seuledr6616@reddit
Thanks! I was actually just looking at this after re-googling haha
ashimbo@reddit
Like others have mentioned, there are several pre-built tools that can handle this for you. However, if you're good with PowerShell, you can use the Posh-ACME module to automate the process.
I use PowerShell Universal for automating PowerShell scripts already, and I now have it renewing my certificates on various websites and business applications, too.
Clavisnl@reddit
I use simple-acme for this. Works great. It’s free, Certifytheweb is payed if I’m correct.
mkosmo@reddit
CTW is free for smaller use-cases. But yeah, you can quickly scale to their paid tiers. But there are lots of free tools out there - CTW was just the first to make it all point-and-click.
adsarelies@reddit
Not every application that uses certs can be automated, yet
ViperThunder@reddit
Some ppl just don't like opening port 80 for let's encrypt to do the easy automated renewal
narcissisadmin@reddit
Every single pen test we've had dinged us for having port 80 open at all, even when the only thing it was doing was redirecting to the root page on 443.
Intrepid_Evidence_59@reddit (OP)
Lmao. I think our security audit team would pass out 🤣
paulschreiber@reddit
Why are you still manually renewing certificates? It's 2025. You should be using Let's Encrypt and an ACME client.
Let me guess: you still require passwords to be rotated, too.
narcissisadmin@reddit
It's almost never OP controlling a given policy on this sub.
Intrepid_Evidence_59@reddit (OP)
I just took over a position that can change our process it will come in time. Still getting people use to the idea of not doing it the old way.
schmeckendeugler@reddit
Oh, dude, I despise them worse than printers.
WittyWampus@reddit
Have around 1000 certs combining internal and external in our environment. All get manually created/renewed/retired/revoked by mainly me, then shipped off to app/server owners to install/bind. I think I've become numb to the process at this point. I highly recommend automating if that's something your business allows you to do. Unfortunately, not at a point to do that yet in our org.
narcissisadmin@reddit
My org is pushing back because LetsEncrypt only has domain validation.
sigh
pdp10@reddit
Oh no! Now dozens of staff need to maintain a non-core expertise, and manually do unfamiliar work, even if it's just the installation and not the creation.
This should be automated, TLS should be provided on a reverse proxy outside the domain of the app-owners, or both. Especially with public-cert validity at 13 months and most likely getting shorter.
Longjumping_Gap_9325@reddit
There's no most likely, it is.
200 days March 15, 2026
100 days March 15, 2027
47 days March 15, 2029
The part that has me wondering is the DCVs, which have dropping maximum periods:
200 days March 15, 2026
100 days March 15, 2027
10 days March 15, 2029 <-- this one here, and I'm not sure how that will work with CA's and OV validations, especially of any wildcard domains are required. That pretty much forces DNS, and at least our CA doesn't have a "DNS Agent" that will automated DCV's for our on-prem IPAM/DNS setup, so that's something I'll need to script out and work with our IPAM team on
WittyWampus@reddit
Not really a problem in our org, but yes in general I agree it's not ideal.
Again, I agree, just not up to me. I'd love if our certs were automated as cert management has basically become 95% of my job at this point. It will be getting better though within the next year as we have the right people working on cleaning up the mess that was left for us now. Also, the people above me know we're on a clock due to the diminishing lifespans over the next few years.
derango@reddit
You might want to work on that pretty soon....
WittyWampus@reddit
Yeah unfortunately like I said, I can't make that decision lol. I've brought it up, but all I can do is wait. I'm dreading the next couple years as the lifespans reduce.
gumbrilla@reddit
Mate, you have Snr Sys admin as your flair.... you dont bring it up. You fucking tell them.
derango@reddit
Tell them they need to have money in the budget to hire someone specifically to renew all 1000 certs every 47 days, and make sure they include money for the therapy that person is going to need. Sheesh.
WittyWampus@reddit
The only saving grace is that most of that 1000 is internal certs not public, so the lifespan reductions won't actually matter for those ones. But yeah we're still looking at a few hundred public certs. It's all in the works though, just going to take some time. Hoping within a year we start making some real headway to getting automation as we have the right people in the right places now for cleaning up the mess we were left.
UninvestedCuriosity@reddit
What the fuck, are you okay dude? Does your work have EAP you can use for this? You deserve all the mental health resources!
OnlyWest1@reddit
I have a PS script that changes it server wide for me. So it's not that bad. Just checking it all is kind of annoying.
Lukage@reddit
Don't remind me.
90% of our cert usage can't be automated thanks to the dozens of various applications and formats required (some need SHA1, some SHA256, some need a PFX, some need separate PEM with configuration files pointing to specific local paths for files, some need XML files updated, some need a manual GUI intervention, etc).
Meanwhile management won't approve a 2+ year certificate because that wildcard cert costs X amount a year, but if we got a 2-year cert, it now costs 2X and that's twice as expensive.
Seriously. They won't justify the purchase because its twice as expensive, even if we're only buying it once every other year and halving the labor. They're that stupid.
Intrepid_Evidence_59@reddit (OP)
That’s ridiculous. We purchased 2 years with GoDaddy but still have to redo them each year.
OhioIT@reddit
All my external certs have been automated with LetsEncrypt, so I honestly don't think about them anymore
Intrepid_Evidence_59@reddit (OP)
I’ll check this out. Thank you
OhioIT@reddit
YW. Also, if you have a webhost like GoDaddy that charges for SSL and doesn't let you automate the process, drop them and find a new(better) host.
Intrepid_Evidence_59@reddit (OP)
We switched to go daddy a few years ago and our looking at other vendors.
OhioIT@reddit
They're horrible. Glad you're looking to change already. Most hosts that use Cpanel have certbot built-in for LetsEncrypt
Intrepid_Evidence_59@reddit (OP)
I’ll check them out. Thank you
chuckmilam@reddit
This is the way, especially for those public-facing systems that can easily do an HTTP ACME challenge.
Free_Treacle4168@reddit
Does that involve a coyote?
Silveress_Golden@reddit
beep beep!
uptimefordays@reddit
No, that’s the manual way lol.
madroots2@reddit
You mean you will "check out" Lets Encrypt? Where are you working for gods sake? How do you not know Lets Encrypt or Cloudflare. Are you a Barista or something
Intrepid_Evidence_59@reddit (OP)
We don’t use cloudflare for anything and I’ve never used let’s encrypt. We have an internal CA server and issuing server. For anything web facing which isn’t much we use Digicert and GoDaddy. We also have nothing cloud base everything is on prem and I work for a city.
Intrepid_Evidence_59@reddit (OP)
Let me rephrase that. We have a few SAAS that are in the cloud but I don’t have to do anything for those other than a few firewall rules. I personally don’t have to manage anything in the cloud other than our Microsoft exchange stuff.
BigBobFro@reddit
Automation is your friend
Ninjatron-@reddit
My team lead who just resigned discuss this topic to me, but that task won't be assign to me. I still have a lot to learn being a sysadmin.
spin81@reddit
Since Ctrl-F "eab" doesn't come up with results, I think I have an important addition that I feel doesn't get mentioned a lot in this conversation.
When you google ACME or ask people about ACME, they might tell you that your servers need to be reachable over port 80. But depending on where you get your certs, this is not in fact true.
I know Sectigo does this but there are bound to be others out there that offer it: External Account Binding (EAB for short). It's a challenge like HTTP or DNS but it works with an account and what's essentially a username and password, and the communication to the ACME server is over a REST API, and it's all outgoing. We do it where I work no a problem, and through a proxy at that.
So depending on what sort of machines you want to use ACME with, you might want to go shopping for vendors that can sell you ACME with EAB.
UninvestedCuriosity@reddit
Reverse proxy all the things behind caddy or nginx! Automatic txt updates for internally hosted records. It's so worth the time investment.
Narrow_Card_6143@reddit
Certificates give me PTSD
Technical-Coffee831@reddit
We’ve been using ACME clients to automate much of it. Highly recommend you look into it!
Bill_Guarnere@reddit
Usually in my experience most of the people I found hating certificates management are those who did not understood completely how PKI works, because once you found how to use openssl it's a piece of cake.
Just to be clear, I'm talking about certificates and keys and csr management, I'm not talking about installing certificates in products.
Usually on open source products installing certificates is a piece of cake, but I remember when I worked on IBM and Oracle products, and It was a pain in the ass because those products (WebSphere and Oracle Portal) manage certificates in the most painful way possible.
I don't know exactly on Microsoft products, I tried a couple of times to trust CA certificates on Windows Server and It was a painful procedure, renewing certificates was extremely simple and straightforward, but installing them on Windows was a PITA.
Fortunately I don't work on Windows, and in my company we only have one Windows Server host that will be removed soon.
HorrimCarabal@reddit
Nah, when you only perform a task once a year, you tend to forget. I feel for the small shops with an overworked single IT person juggling daily tasks while having to figure out ACME.
Rouxls__Kaard@reddit
Sooner than later you’re going to need to replace all those manual certs with automated ones or use a proxy like cloudflare.
dadoftheclan@reddit
CertifyTheWeb if you like UIs.
Phyxiis@reddit
That’s what we use to automate ~50 servers. Everyone who doesn’t know yet should know that the likelihood of ssl certs being issued as another has said will be 47 days by 2029 https://www.darkreading.com/cyberattacks-data-breaches/critical-steps-advance-ssl-tls-certificates
Cheomesh@reddit
Yep, never liked it - unfortunately every position I've worked has not really had an automated solution, so it was all generated by hand each time.
sudds65@reddit
Time to buy Venafi lol
naednek@reddit
Considering this was my first year doing after my coworker retired. Yep. Still don't understand why we sometimes issue internally and some from a vendor.
N0vajay05@reddit
Certificates os one of those things many never stop to learn but are extremely important to the environments. I highly recommend taking a deep dive or certificates so they aren't such an issue anymore.
Scary_Bus3363@reddit
I understand certs well. Its the automation part that depends on cloud stuff to work and no doco on my appliance which supposedly supports it that scares me. Understanding certs isnt going to help you when the stupid http site you have to run for letsencrypt goes down or the renewal fails. Knowing why something is not working does not help you fix it. I need a deep dive into ACME maybe and my freaking reverse proxy vendor who is not F5 or Cisco lies and says its supported
Intrepid_Evidence_59@reddit (OP)
It’s not that I don’t understand it. It’s just one of the few routine maintenance things that I get anxious about. No different when I am doing a full disaster recovery check once a month. I’ve done that hundreds of times but I still go slow and steady because once I fucked up so bad that a 1 hour task turned into a week long headache. I think some people are taking this post as if I’m clueless when it comes to certs but really it was just a rant and I see a lot of other people feel the same way as me.
Shot-Document-2904@reddit
Managing certs on Windows workstations, not so bad. Managing certs at scale across Windows Servers, Linux Servers, and dozens of hosted applications, a real pain in the arse. Now let’s make it an offline environment. I automate as much as possible and it’s still pretty labor intensive. All the formats, permissions, and locations…
Scary_Bus3363@reddit
This person does certs
ButternutCheesesteak@reddit
Idk I use PKI to establish trust between our Linux and Windows servers and it's easy.
Shot-Document-2904@reddit
You may have missed to part where I highlighted dozens of hosted apps.
ButternutCheesesteak@reddit
If 2 servers are communicating they either beed creds or a trist relationship.
Shot-Document-2904@reddit
It’s ok. These things come with time. You’re on the right track.
Scary_Bus3363@reddit
Abysmal doco, poor vendor support and super criticality make me terrified of moving forward with the automation options that exist here. I understand certs fairly well but this has a lot of moving parts that could result in severe outages. In time hardware will adapt and support this but that does not help when I am forced to run not quite EOL stuff due to budget,
I think my initial statement is why most people hate certs so much. No consistency. No mans land of support. Clunky tools and so damn important the world stops if it fails. Anyone who thinks certs are easy has not met a Java Keystore.
Being I consider myself pretty advanced with cert knowledge and I am scared of this, I feel for the average Windows click ops admin that gets this dumped on them.
Steve----O@reddit
It will soon be 47 days.
skiitifyoucan@reddit
SSL certs dont.... I have 2000 of them, and like 95% are automated.
Azure fucking app registration secrets that fucking devs have stored anywhere and everywhere but EXCEPT in an Azure keyvault stress me out. I have something like 1000 of those.
Intrepid_Evidence_59@reddit (OP)
Thankfully we only have a few things linked in azure. One being a camera software that only allows you to have a 1 year cert the others are 2 if I’m not mistaken. Most of ours our automated except our phone system, and web facing servers. Those we use digicert or godaddy. After this post I am looking into switching to one vendor that allows me to automate the process. Especially since everyone let me know in a few years everyone is switching to basically a bi monthly cert renewal.
cbass377@reddit
I hate it too, but not stressfully so.
Intrepid_Evidence_59@reddit (OP)
I just push it off until the week before that’s why it stresses me out. I do it to myself lol
cbass377@reddit
Yeah. There is a time pressure if you put it off.
I get the notice, send it to app owner saying get me the csr. Then do the work the next morning. First thing in the day. Move the big rocks/ or do the things you hate first thing, then the day gets easier as it goes.
Intrepid_Evidence_59@reddit (OP)
I get that. I scheduled the meeting with our ERP person so he can verify the software is functioning correctly since this batch of servers are for our ERP system. Thankfully half of our organization has moved to the cloud for this the other half will be moved next year. Then the servers just have to stay alive for 5ish years for audit reasons. It was also partially pushed back due to timing with my ERP guy. We both have been slammed with work especially him since he’s in the middle of a migration. I also have phone certs coming up next week so that took priority. I’m doing those 2 weeks before they lapse just incase of any issues.
dracotrapnet@reddit
Absolutely. I hate the phone system's certs the most. It completely manual and I always miss something somewhere and a suer gets an error signing into the app once the old cert expires. It is hard to confirm that all the nginx services moved to the new cert. I have a walk through document I made for it but I always have to go through it twice. I have been putting off a cert change for the phone system right now - it is due in 4 days. Worst part is it disconnects all clients to update the cert and we always get tickets and complaints when their app doesn't immediately reconnect.
Intrepid_Evidence_59@reddit (OP)
You got this!!
dracotrapnet@reddit
Maybe... I just went through the task, then sent it to do windows updates for August (it is on slow track)
jakesps@reddit
No. I use certbot and other ACME clients with Let's Encypt and ZeroSSL.
TxDuctTape@reddit
The ones I hate are the ones that use damn keystores
zaazz55@reddit
Automate it
x-Mowens-x@reddit
TIL people don't use Letsencrypt.
hitman133295@reddit
Lol wait until you have to migrate your CA server to external providers that's not msft
Studiolx-au@reddit
This thread scares me to see how many people don’t have cert automation in place. Cert renewal is a problem from 5-10 years ago.
Adam_Kearn@reddit
I would recommend automating this as the certificate life time is getting reduced soon.
There are loads of tools out there that can help with this. For web servers I tend to just put these behind Cloudflare. But IIS / Nginx and all the other popular hosting services will also support the automating process.
Exp3r1mentAL@reddit
Yikes!! don't look up abt upcoming tls cert lifetime changes
ButternutCheesesteak@reddit
Never had a problem w/ it, pretty simple for me. Why is this so hard for you? I maintain our web-facing and internal certs. Also it's TLS. SSL was deprecated a while ago.
jake04-20@reddit
Nope, ACME.
TheRealJachra@reddit
Perhaps you should take a look at software like CyberArk Certificate Manager or something like that.
https://www.cyberark.com/products/certificate-manager/
The lifetime of SSL/TSL certificates are going to be changed in the near future. The will be only valid for less days from March 2026 onwards. By March 2029 the lifetime will be 47 days. I would suggest to start planning and start thinking about automation for it.
https://www.thesslstore.com/blog/47-day-ssl-certificate-validity-by-2029/
Unorthodox_3311@reddit
I was bothered by a similar problem and decided to build a simple tool for cert expiring alerts. Eventually, I build it into somewhat working web app called "CertAlert". It was not as useful as I thought it would be, but still better than sheets. Maybe I was just not familiar with similar tools out there.
Constant_Hotel_2279@reddit
I completely automated it with cron jobs.....
XD__XD@reddit
wildcard all the things JK JK dont do that, please dont do that
Gainside@reddit
automation (let’s encrypt + acme clients) helps, but for the stuff that can’t use it, still gives the same pit-in-the-stomach feeling every renewal
OhioIT@reddit
Agreed. Thankfully for internal sites, ACME certificate authorities can be deployed and then use the same tools as LC for internal sites too.
I wish there was automation for specific devices where installing an agent isn't possible
Gainside@reddit
servers are easy enough with acme, but once you get into appliances / legacy gear it’s still a manual circus. some vendors are finally exposing apis for cert push, but for the ones that don’t, it’s still pretty manual
notarealaccount223@reddit
For any that you cannot automate
Write a procedure
Use that procedure every renewal and tweak/adjust it as needed.
We have two systems that need to be manually changed. One is significantly user facing. The procedure means it goes smoothly every time.
Automate anything that can be automated.
dollhousemassacre@reddit
I think I've gone the opposite direction. It used to be this huge thing for me, now it's just a tiny part of the job.
Jawshee_pdx@reddit
I have done so many certs I don't even think about it anymore. I am the cert guy currently so before I finish typing this I bet there will be a cert related task sitting on my desk.
HeligKo@reddit
Certificate management is low hanging fruit. Most systems now support ACME protocol.
Fritzo2162@reddit
Yeah, I hate it too, but I have ours all scheduled out so tickets are automatically created 60 days before expiration. That way there's no surprises.
Intrepid_Evidence_59@reddit (OP)
We monitor them with a software and get alerts at 90, 60, 30, and 7 days.
NSFW_IT_Account@reddit
Probably the worst part about IT for me.
Intrepid_Evidence_59@reddit (OP)
Agreed. It’s not that it’s hard it’s just the paranoia of when you go do it will it go smoothly or will you have to troubleshoot what went wrong. We have our ERP system on this next batch and I am dreading if it goes wrong. It shouldn’t but it’s the what if lol. Doesn’t help we are switching to there cloud right now so half is still on prem and the other half isn’t.
NSFW_IT_Account@reddit
I just had a fun several hours with an on prem exchange server and renewing SSL a couple weeks ago. No one could access email for a little while, and it was a good time all around!
Intrepid_Evidence_59@reddit (OP)
Oof. I just did our AD servers that sync with our Proxy servers that then sync with Exchange and was petrified that this was going to happen to me. Thankfully I found some helpful articles that lead me in the right direction because my coworkers who use to do this left no documentation and left the company. I got major props though from my director for figuring it out and documenting the crap out of it.
Table-Playful@reddit
It is harder than it should / could be
Otto-Korrect@reddit
And now that Entrust is 'Sectigo', owned by private equity, the service will go away while the prices go sky-high.
I have PTSD from renewing our certs every year. The system changes EVERY time so you can' just make notes and do what you did the year before.
Huge_Recognition_691@reddit
An ACME server is your friend.
phunky_1@reddit
It will be even funnier once the maximum validity length will be 47 days in 2027.
You need to automate it, or you will basically have a full time job to rotate certificates depending on how big the environment is.
Intrepid_Evidence_59@reddit (OP)
80 something VMs only 10-15 use public facing certs though.
SikhGamer@reddit
Farm out it to something like AWS ACM. LE is fine, but ACM is next level hands off.
Intrepid_Evidence_59@reddit (OP)
We don’t have any cloud infrastructure at the moment.
joedotdog@reddit
I have a paranoid theory that says that someone had the idea to commercialize the automation of this process and this is the result.
jamesaepp@reddit
Renewing certificates is easy as shit. Rebinding certs is a pain in the ass.
PoolMotosBowling@reddit
Do them all at once, then you only have to do it once a year. (For now, just wait until it's less then 60 days)
kidmock@reddit
I just let ACME handle it and don't worry about it ever again
Hacky_5ack@reddit
They bug me too
davy_crockett_slayer@reddit
Cert renewal should and can be automated. If CertBot from Let's Encrypt doesn't suit your needs, look into Digicert's TLM. It's actually pretty good for cert renewal if you need to deal with legacy on-prem Windows server and routers, etc. https://www.digicert.com/trust-lifecycle-manager
Usual-Chef1734@reddit
It sux, and there are not very many robust solutions for automating it. The ones that can charge a mountain, because they can.
OinkyConfidence@reddit
Real-world SSL certificates are the racket of the IT world. Used to be legit and necessary, now with everything being secured with SSL certs, nothing is secured with SSL certs.
Noc_admin@reddit
Learn about the different challenge types, there are tons of different options to automate cert renewal with certbot/LetsEncrypt. Theres no good reason for anyone to manually rotate certs these days. Also, if its key infra have a failover self signed cert thats a lifetime or 10 year or something that is never used unless there is an issue. Most modern monitoring solutions you can alert when the failover cert is used and will know something broke but no one else will.
riddlerthc@reddit
my wild card came up for renewal so I switched everything to LC this year. Took maybe 4-5 hours to get everything done.
cjcox4@reddit
Microsoft, and others, have been pushing the "you can't trust certs" message for a bit. End goal? Unknown.
pdp10@reddit
De-commodification. Microsoft is also pushing "passphraseless" authentication, which is a real thing but which only Microsoft is in a good position to sell currently.
On the other hand, Microsoft has thrown in the towel on proprietary discovery protocols for the moment. That usually happens when they've lost conclusively, but every once in a while they do it to save money like when Microsoft embraced Chromium for its branded browser.
pdp10@reddit
Script it. Even if it's not end-to-end automatable using a protocol like ACME or SCEP, script it.
Rotate certs early, to vastly reduce stress. Even though the individual public cert validity period is limited by CA/B, commercial cert signers typically value-add by allowing multiple individual certs to be issued during the subscription period.
Validate the new certs quickly after rotation, also using automation/scripts.
Validate the new certs before rotation, if applicable. This ensures they didn't get truncated or have some other simple error.
Rotate certs during the workday.
CatoDomine@reddit
Every public CA should support ACME.
ACME clients are available for pretty much every platform.
Automate your cert issuance, you will be happier.
CG_Kilo@reddit
Letsencrypt is your friend
First-Structure-2407@reddit
Yep yep yep feel exactly the same but my next renewal should be my last
Intrepid_Evidence_59@reddit (OP)
I have 5 or 6 left hopefully.
Top-Anything1383@reddit
If your infrastructure can handle automation, do that! I'm down to two certs which have to be manually updated annually, I'm hoping it'll be down to one by next renewal.
Intrepid_Evidence_59@reddit (OP)
For internal certs they are mainly automated except for any of our web facing servers. Those our done through two venders. Thank you for the advice though.
Dear-Carpet4756@reddit
Check about automation, and make some courses about how SSL certificates are working At the beginning it was the same but when you know all this stuff is working, it’s pretty simple.
Focus on how certificates work (server certificate, client certificate, how CN attribute work, how CA Chain and so one are working)