how do you handle ssl cert rotation for internal services
Posted by Sroni4967@reddit | sysadmin | View on Reddit | 12 comments
Currently using let's encrypt with certbot for everything but the 90 day renewal keeps breaking random internal apps. Thinking about switching to an internal CA but not sure if the overhead is worth it for \~20 services.
8BFF4fpThY@reddit
How does cert rotation break apps? I've never heard of such a thing.
anonpf@reddit
Are the new root ca certs used not installed on the internal system?
Electronic_Tap_3625@reddit
I use Microsoft CA for all internal apps, that are only accessed from inside the network. External apps I use https://certifytheweb.com/ to enroll lets encrypt. It actually works well and has many options like running scripts when the cert is renewed.
poizone68@reddit
Can those services be put behind an internal load balancer and are fine with SSL offloading? If yes, then you should be ok with a reverse proxy solution.
sryan2k1@reddit
99% automated with internal ACME. A few stubborn things get them done by hand from our normal PKI.
techretort@reddit
Poorly!
omglazrgunpewpew@reddit
Whelp, cert rotation is one of those things that seems small at 20 services right up until one random internal app decides trust chains are a personal attack.
I’d separate the issue into two buckets:
Public/internal services that already work cleanly with ACME? Keep letting Let’s Encrypt do its thing if it’s behaving. Sure, the 90-day cert lifetime is annoying, but the automation model is good.
Internal only weird apps that break every renewal because they need a Java keystore, a Windows cert store import, a service restart, a bundled PEM path, the first full moon of the month, etc.? Those probably need a real internal process.
For \~20 services, an internal CA can be worth it if you do it stupidly boringly:
The important bit is not internal CA vs Let’s Encrypt. It’s “do you have a repeatable deployment path per app?” Certbot renewing the file is only half the job. The app also has to reload it, import it, bind it, or whatever dance that particular service requires.
I’d probably keep Let’s Encrypt/ACME where it’s smooth, and move the delicate internal only stuff behind an internal CA or a reverse proxy/load balancer that owns TLS centrally. Centralizing TLS is the biggest quality of life upgrade if the apps themselves are allergic to cert rotation.
Just don’t switch because 90 days is annoying. Switch/centralize because your services don’t consume renewed certs reliably. Expiring certs are bad, but “the cert renewed successfully and the app is still using the old one” is where your eyes start accidentally drifting in different directions.
Visible_Spare2251@reddit
We've automated certbot on our internal systems to post records to our external DNS for validation.
Prestigious-Bath8022@reddit
90 day renewals sound nice until you actually have to babysit them.
dchit2@reddit
Internal? Yea make yourself a 1000 year CA and live the dream
agent-squirrel@reddit
What apps are breaking and why? Are they pinning certs?
SimpleSysadmin@reddit
Why is certs updating breaking internal apps?