My preferred procedure is to use DNS-01 validation and have no publicly accessible "A" or "AAAA" record for internal services.
Or even a more extreme example: https://crt.sh/?id=27555237869 (sorry for any possible crt.sh downtime) - the domain name in question never existed in public or private DNS by itself. It is used only for a WPA3-Enterprise network, as the CN that WiFi clients expect to be present in the RADIUS server certificate, but never resolve. In the public DNS, only the "_acme-challenge" TXT record exists.
The relative proximity of the words "done right" and "split-horizon DNS" makes my insides hurt a little bit.
Use DNS validation to allow these internal services to pull ACME certs. There's so much less headache, long-term.
Split-horizon DNS (and the tedious make-work it can create when you start needing to mirror public-accessibly records in the private DNS) has always been something to aspire to move away from in my experience.
I use the acme dns-1 challenge on my public domain. That gives you certificates you can use as you see fit, without needing to expose anything else to the public internet.
I also use Tailscale so I configure my DNS to use my Tailscale IP addresses. If you don’t want to expose them on a public DNS server you can add them only to an internal DNS server.
So if "example.com" is control by Corporate IT, and they don't want 'random' folks fiddling with it, then you can create a "dnsauth.example.com" and point the dns-1 challenge record from "…foo.example.com" to "foo.dnsauth.example.com" (or a completely different domain, like "…example.net").
There are DNS servers written strictly focused on this use case:
Personally, I hate split horizon DNS. I prefer the "BeyondCorp" model. I MUCH prefer putting an mTLS cert in my trusted devices over relying on VPNs in same devices. I've yet to see a "clever" DNS setup not cause annoyances.
Specifically grafana is nice to be able to see on the phone, and split horizon DNS and corp VPN is a hassle, to say the least, on phones.
This is crazy. If you have a home network with a few internal services, or some sort of network where you don't control the endpoints, just use DNS validation. That's why it exists.
But on hosts you control, you should absolutely provision them with an identity and join the local CA. You're going to need it for a multitude of other reasons.
The real answer here is that configuring HTTPS clients to trust a self-signed cert (or signed by an internal CA) shouldn't be as difficult as it is. I find it extremely annoying that every programming language has it's own idea of where certificates should live instead of just checking the os trust store.
Arguably it's 1/2. You can put public certs on proxy then give proxy private CA to backend services. Then you don't need public certs for all the private stuff nor need to trust the private CA on all your devices.
Why not just map the domain to an internal IP and call it a day? Then the only way it can be accessed is through a VPN. Then use a wildcard so none of leaks into cert transparency logs
> Then use a wildcard so none of leaks into cert transparency logs
You now also have to build infrastructure to distribute the wildcard from (presumably) central place where you generate it to all the different places where it is desired.
And hope the wildcard's private key does not leak from one of myriad of places it now lives.
I've documented how to securely set up TLS certificates for internal services without creating TLS issues for http clients downstream. All thanks to split-horizon DNS, WAF and ACME protocol. All for free!
I use a registered domain with DNS validation and then CNAMEs that I resolve locally. Basically:
1. Register a domain ("server.com") and put it on some public DNS that can do DNS validation with acme.sh.
2. Use DNS validation to get a certificate on your domain from Let's Encrypt. You can just grab a wildcard one ("*.server.com").
3. CNAME all of your services on a public DNS to an internal address ("email.server.com" → "server.internal", "plex.server.com" → "server.internal").
4. Resolve your internal address on a local DNS server with an A record ("server.internal" → 192.168.0.123). This can often just be done on your router.
Since you use DNS validation, you just API keys for your public DNS service that acme.sh can use. No need to have any VPN network interfaces for getting your certificate. Your wildcard certificate also doesn't leak any details about your services.
My preferred procedure is to use DNS-01 validation and have no publicly accessible "A" or "AAAA" record for internal services.
Or even a more extreme example: https://crt.sh/?id=27555237869 (sorry for any possible crt.sh downtime) - the domain name in question never existed in public or private DNS by itself. It is used only for a WPA3-Enterprise network, as the CN that WiFi clients expect to be present in the RADIUS server certificate, but never resolve. In the public DNS, only the "_acme-challenge" TXT record exists.
The relative proximity of the words "done right" and "split-horizon DNS" makes my insides hurt a little bit.
Use DNS validation to allow these internal services to pull ACME certs. There's so much less headache, long-term.
Split-horizon DNS (and the tedious make-work it can create when you start needing to mirror public-accessibly records in the private DNS) has always been something to aspire to move away from in my experience.
I use the acme dns-1 challenge on my public domain. That gives you certificates you can use as you see fit, without needing to expose anything else to the public internet.
I also use Tailscale so I configure my DNS to use my Tailscale IP addresses. If you don’t want to expose them on a public DNS server you can add them only to an internal DNS server.
> I use the acme dns-1 challenge on my public domain.
See also perhaps DNS aliasing in case you are not able to dynamically update your 'primary' domain, but can update a secondary or sub-domain:
* https://github.com/acmesh-official/acme.sh/wiki/DNS-alias-mo...
So if "example.com" is control by Corporate IT, and they don't want 'random' folks fiddling with it, then you can create a "dnsauth.example.com" and point the dns-1 challenge record from "…foo.example.com" to "foo.dnsauth.example.com" (or a completely different domain, like "…example.net").
There are DNS servers written strictly focused on this use case:
* https://github.com/acme-dns/acme-dns
Also code that handles a bunch of DNS provider APIs so you don't have to roll your own for ACME client hooks:
* https://github.com/dns-lexicon/dns-lexicon
Moreover you can delegate domain to improve security.
https://www.eff.org/deeplinks/2018/02/technical-deep-dive-se...
Personally, I hate split horizon DNS. I prefer the "BeyondCorp" model. I MUCH prefer putting an mTLS cert in my trusted devices over relying on VPNs in same devices. I've yet to see a "clever" DNS setup not cause annoyances.
Specifically grafana is nice to be able to see on the phone, and split horizon DNS and corp VPN is a hassle, to say the least, on phones.
I bet you can do it with HA-Proxy, but I use https://github.com/ThomasHabets/sni-router
This is crazy. If you have a home network with a few internal services, or some sort of network where you don't control the endpoints, just use DNS validation. That's why it exists.
But on hosts you control, you should absolutely provision them with an identity and join the local CA. You're going to need it for a multitude of other reasons.
The real answer here is that configuring HTTPS clients to trust a self-signed cert (or signed by an internal CA) shouldn't be as difficult as it is. I find it extremely annoying that every programming language has it's own idea of where certificates should live instead of just checking the os trust store.
> TLS certificates for internal services* done right
* "internal services" = on a single server that is publicly routable
I don't agree that tunneling everything through some external facing proxy is "TLS certificates for internal services done right".
Arguably it's 1/2. You can put public certs on proxy then give proxy private CA to backend services. Then you don't need public certs for all the private stuff nor need to trust the private CA on all your devices.
I am looking forward to finally using DNS-PERSIST-01 for validation. No more dynamic DNS updates, DNS credentials or forwarding necessary.
Why not just map the domain to an internal IP and call it a day? Then the only way it can be accessed is through a VPN. Then use a wildcard so none of leaks into cert transparency logs
> Then use a wildcard so none of leaks into cert transparency logs
You now also have to build infrastructure to distribute the wildcard from (presumably) central place where you generate it to all the different places where it is desired.
And hope the wildcard's private key does not leak from one of myriad of places it now lives.
I've documented how to securely set up TLS certificates for internal services without creating TLS issues for http clients downstream. All thanks to split-horizon DNS, WAF and ACME protocol. All for free!
Uhm....source?
this is all fine and good, if you are okay broadcasting your internal hostnames. I suppose it's a trade off some might make.
There's one way around that which is requesting a wildcard cert, but then that has its own rammifications
I use a registered domain with DNS validation and then CNAMEs that I resolve locally. Basically:
Since you use DNS validation, you just API keys for your public DNS service that acme.sh can use. No need to have any VPN network interfaces for getting your certificate. Your wildcard certificate also doesn't leak any details about your services.