T-Mobile hijacks Google and Level3 DNS servers

Posted: Oct 28, 2014

UPDATE: It looks like they fixed it. Let me know if you continue to see intercepted DNS traffic. -Eli, 11/5/14
When you type an address that doesn't exist into your browser, T-Mobile US hijacks the "Server Does Not Exist" error message and instead redirects you to a cheesy T-Mobile branded web search page (example). This is called DNS Hijacking and they're not the only ones doing it, but it's still not cool. (T-Mobile claims to offer a way to opt-out of this service, but it uses cookies and doesn't really work in a meaningful way.) However T-Mobile takes this a significant step further: they actually intercept DNS queries to popular public servers like 8.8.8.8 and 4.2.2.2 . So people who go out of their way to avoid T-Mobile broken default server still get a broken experience. Network security people would call this a "Man in the middle" attack. Here's an example of a bogus search page being returned when making a query for a nonexistent domain with the default T-Mobile DNS: 
$ nslookup bogus.fake.esd.io
Server: 192.168.42.129
Address: 192.168.42.129#53

Name: bogus.fake.esd.io
Address: 198.105.254.104
Name: bogus.fake.esd.io
Address: 198.105.244.104
Those IP addresses returned (198.105.254.104 & 198.105.244.104) belong to tmob.search-help.net. Typically users who don't want this sort of annoying search page can avoid it by manually changing their DNS to another server not controlled by the ISP. A popular semi-public choice of DNS server is 4.2.2.2. Except with T-Mobile internet, even switching servers results in the same problem! 
$ nslookup bogus.fake.esd.io 4.2.2.2
Server: 4.2.2.2
Address: 4.2.2.2#53

Name: bogus.fake.esd.io
Address: 198.105.254.104
Name: bogus.fake.esd.io
Address: 198.105.244.104
But! Here's what happens when you make the same request for the previous server's neighbor: 4.2.2.1. T-Mobile's network does not intercept requests to this server. 
$ nslookup bogus.fake.esd.io 4.2.2.1
Server:     4.2.2.1
Address:    4.2.2.1#53

** server can't find bogus.fake.esd.io: NXDOMAIN
That works! NXDOMAIN is the result we want when we query a domain that doesn't exist. They are messing with 4.2.2.2 but not with 4.2.2.1. Further evidence: Compare the traceroutes. 
$ traceroute 4.2.2.1
traceroute to 4.2.2.1 (4.2.2.1), 64 hops max, 52 byte packets
 1  192.168.42.129 (192.168.42.129)  0.778 ms  0.214 ms  0.352 ms
 2  10.170.226.192 (10.170.226.192)  3571.048 ms  37.735 ms  39.922 ms
 3  10.170.201.10 (10.170.201.10)  55.323 ms  37.650 ms  51.597 ms
 4  10.177.30.209 (10.177.30.209)  92.900 ms  71.239 ms  110.118 ms
 5  10.177.24.10 (10.177.24.10)  71.566 ms  77.048 ms  80.365 ms
 6  xe-8-2-0.edge3.washington1.level3.net (4.59.144.13)  61.665 ms  67.321 ms  58.015 ms
 7  ae-104-3504.edge1.washington12.level3.net (4.69.158.29)  61.814 ms
    ae-101-3501.edge1.washington12.level3.net (4.69.158.17)  76.690 ms
    ae-203-3603.edge1.washington12.level3.net (4.69.158.41)  111.538 ms
 8  ae-201-3601.edge1.washington12.level3.net (4.69.158.33)  57.449 ms
    ae-102-3502.edge1.washington12.level3.net (4.69.158.21)  59.410 ms
    ae-202-3602.edge1.washington12.level3.net (4.69.158.37)  98.926 ms
 9  a.resolvers.level3.net (4.2.2.1)  57.297 ms  55.422 ms  74.387 ms

 $ traceroute 4.2.2.2
 traceroute to 4.2.2.2 (4.2.2.2), 64 hops max, 52 byte packets
 1  192.168.42.129 (192.168.42.129)  0.684 ms  0.363 ms  0.245 ms
 2  \* \* \*
 3  \* \* \*
 4  \* \* \* 
 5  \* \* \*
4.2.2.1 works, 4.2.2.2 (the more common one) is getting eaten by T-Mobile.

Questions for T-Mobile

  1. What is the purpose of intercepting/altering traffic to 3rd party DNS servers?
  2. What's the full list of addresses being intercepted?
  3. Why is NXDOMAIN being hijacked in the first place? Is it to generate revenue from ads on the search page, or is there some other technical reason?