My network is built using a good separation of zones. It is set up so that I do most things myself and rely on third parties like an ISP only for the connection itself.
So I have servers on a seprate network, including a caching DNS server and a proxy. There is a LAN where our PC's and phones connect. Traffic is managed by the awesome Pfsense firewall.
Most surfing we do is via the proxy, but outbound http(s) from LAN is not prohibited. So recently I found that Youtube only works through the proxy and not directly. I checked all the usual suspects, did traceroutes etc and everything turned out ok.
Until I went to my terminal and tried some things:
$ http youtube.com HTTP/1.1 301 Moved Permanently Alternate-Protocol: 80:quic,p=0 Content-Length: 0 Content-Type: text/html Date: Fri, 31 Jul 2015 07:12:35 GMT Location: http://www.youtube.com/ Server: gwiseguy/2.0 X-XSS-Protection: 1; mode=block
Looks good, so we need to go to www.youtube.com
$ http www.youtube.com http: error: ConnectionError: HTTPConnectionPool(host='www.youtube.com', port=80): Max retries exceeded with url: / (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)
So, a DNS problem while we did find youtube.com?
$ host www.youtube.com ;; Truncated, retrying in TCP mode. ;; connection timed out; no servers could be reached
What caught my eye is 'Truncated, retrying in TCP mode', so doing the host command from the server network:
$ www.youtube.com ;; Truncated, retrying in TCP mode. www.youtube.com is an alias for youtube-ui.l.google.com. youtube-ui.l.google.com has address 22.214.171.124 youtube-ui.l.google.com has address 126.96.36.199 youtube-ui.l.google.com has address 188.8.131.52 youtube-ui.l.google.com has address 184.108.40.206 youtube-ui.l.google.com has address 220.127.116.11 youtube-ui.l.google.com has address 18.104.22.168 youtube-ui.l.google.com has address 22.214.171.124 youtube-ui.l.google.com has address 126.96.36.199 youtube-ui.l.google.com has address 188.8.131.52 youtube-ui.l.google.com has address 184.108.40.206 youtube-ui.l.google.com has address 220.127.116.11 youtube-ui.l.google.com has address 18.104.22.168 youtube-ui.l.google.com has address 22.214.171.124 youtube-ui.l.google.com has address 126.96.36.199 youtube-ui.l.google.com has address 188.8.131.52 youtube-ui.l.google.com has IPv6 address 2a00:1450:400c:c02::5b
Same line but it does work,.. in TCP mode.
So back to the firewall and I see that traffic from LAN to SERVERS is allowed on port 53, UDP only. So I set it to allow UDP+TCP and bingo, youtube works on LAN without proxy.
The explanation is rather simple, the response for www.youtube.com is simply to big for UDP, so the client library falls back to TCP which I did not explicitly allow on my network, hence youtube was 'broken'
For a long time I've looked at DNS as a UDP based protocol that you could use over TCP if you wanted. This problem however shows that DNS relies on TCP to work, so we as system or network admins have to take that into consideration.
So, go forth and open TCP in your firewalls for port 53!