

I tell it to point docmost.example.com to 192.168.1.80, […] but actually nothing happens. I get a connection timed out error.
I suggest to collect more info about this “nothing”.
ping from your PC on the inside to the name docmost.example.com
ping from your PC on the inside to the number 192.168.1.80
traceroute from your PC on the inside to the name docmost.example.com
traceroute from your PC on the inside to the number 192.168.1.80
Then do the same in the reverse direction: from that docker container to your PC.
Maybe traceroute shows you some stations on the route. Then do the same from this station.
Write down the results thoroughly.
Now you know that the problem is not your DNS.
It is either your routing or firewalling.