*.local.domain.com -> has its own cert but the * can be anything and the same cert can be used for anything in place of the star as many times as you want and therefore doesn’t need to be internet accessible to verify. That way vaultwarden.local.domain.com remains local only.
You are completely correct…for normal certs. Internal domains require a wild card cert with DNS challenge.
This video explains how to set it up with traefik
https://youtu.be/liV3c9m_OX8
I’d bet caddy can do something similar.
Basically you have: