Find the perfect gift for the grad in your life with Graduation gifts that connects us from AT&T.
Get superfast AT&T Fiber internet
randrews_01's profile

Tutor

 • 

13 Messages

Sunday, April 21st, 2024 8:25 PM

ATT datacenter router problem

I noticed that last night my internet connection was terrible, with dropped packets and what not.   Based on some tracerroutes, it looks like ATT's data center is having a problem both in IPV4 and IPV6.  I used to have latency in the 10-12ms range for hitting google.com.  Now I have 30-50ms and dropped packets.   Is there a method to reach a competent network engineer to share these results with?  I can't bear to deal with L1/L2 front line support where they always want to blame the enduser.

  2     2 ms     4 ms     2 ms  71.149.77.250
  3     *        *        *     Request timed out.
  4    35 ms    36 ms    38 ms  32.130.17.87
  5     *        *       40 ms  108.170.231.42
  6    36 ms    36 ms    36 ms  142.251.71.113
  7    37 ms    35 ms    36 ms  dns.google [8.8.8.8]

Tracing route to ipv6.l.google.com [2607:f8b0:4007:817::200e]
over a maximum of 30 hops:

  1     1 ms     2 ms     2 ms  2001:506:6000:100:76:253:217:49
  2     2 ms     5 ms     4 ms  2001:506:6000:100:76:253:217:50
  3    10 ms     *        *     2001:1890:f6:779::2
  4    10 ms     8 ms     8 ms  2001:1890:fff:f6e:12:255:10:124
  5    10 ms    10 ms     8 ms  2607:f8b0:832b::1
  6     *        *        *     Request timed out.
  7     *        9 ms     *     2001:4860:0:1::2032
  8    11 ms     *        *     2001:4860::c:4001:e559
  9     *        *        *     Request timed out.
 10    39 ms    38 ms    38 ms  2001:4860::9:4003:11cc
 11    39 ms     *       40 ms  2001:4860::12:0:aba3
 12    37 ms    37 ms    37 ms  2001:4860:0:1::56fb
 13    37 ms    37 ms    37 ms  lax31s16-in-x0e.1e100.net [2607:f8b0:4007:817::200e]

2     4 ms     2 ms     5 ms  71.149.77.250
  3     *        *        *     Request timed out.
  4     *       43 ms     *     32.130.16.9
  5     *       35 ms    39 ms  gar25.dlstx.ip.att.net [12.122.85.233]
  6    37 ms    37 ms    38 ms  192.205.36.222
  7    39 ms    40 ms    38 ms  be2764.ccr32.dfw01.atlas.cogentco.com [154.54.47.213]
  8    47 ms     *       48 ms  be2433.ccr22.mci01.atlas.cogentco.com [154.54.3.214]
  9    59 ms    56 ms    57 ms  be2832.ccr42.ord01.atlas.cogentco.com [154.54.44.170]
 10    58 ms    62 ms    61 ms  be2125.rcr21.ord04.atlas.cogentco.com [154.54.82.202]
 11     *        *        *     Request timed out.
 12    58 ms    58 ms     *     vl202.chi-cs1-dist-1.cdn77.com [138.199.0.235]
 13    63 ms     *       59 ms  dns.adguard.com [94.140.15.15]

 2     2 ms     5 ms     4 ms  2001:506:6000:100:76:253:217:50
  3    10 ms     *        *     2001:1890:f6:779::2
  4    10 ms     8 ms     8 ms  2001:1890:fff:f6e:12:255:10:124
  5    10 ms    10 ms     8 ms  2607:f8b0:832b::1
  6     *        *        *     Request timed out.
  7     *        9 ms     *     2001:4860:0:1::2032
  8    11 ms     *        *     2001:4860::c:4001:e559
  9     *        *        *     Request timed out.
 10    39 ms    38 ms    38 ms  2001:4860::9:4003:11cc
 11    39 ms     *       40 ms  2001:4860::12:0:aba3
 12    37 ms    37 ms    37 ms  2001:4860:0:1::56fb
 13    37 ms    37 ms    37 ms  lax31s16-in-x0e.1e100.net [2607:f8b0:4007:817::200e]

1 Message

1 month ago

I can see we're both following similar routing at early hops. I'm in Round Rock, TX and experiencing the same thing. I think we're back to this from a few months ago (I was one of the ones who ended up making the FCC complaint that eventually got AT&T to acknowledge and resolve).

(edited)

Tutor

 • 

13 Messages

1 month ago

Visuals help

42 Messages

1 month ago

Came here to say the same thing! Back with my fellow central TX friends and a terrible internet connection, all upstream.

My AT&T account does say there is an outage in my area and it will be resolved soon, but it says that pretty much every time I log in, so I don't put much faith in it.

(edited)

42 Messages

1 month ago

And since we love traceroutes, we go from 3 ms to 400 ms (and timeouts) all within AT&T's networks:

$ traceroute -w 1 www.google.com 
traceroute: Warning: www.google.com has multiple addresses; using 142.250.115.106
traceroute to www.google.com (142.250.115.106), 64 hops max, 40 byte packets
 1  192.168.1.254 (192.168.1.254)  0.37 ms  0.268 ms  0.471 ms
 2  99-67-236-1.lightspeed.austtx.sbcglobal.net (99.67.236.1)  1.633 ms  1.401 ms  1.481 ms
 3  71.149.77.250 (71.149.77.250)  2.955 ms  2.987 ms  2.997 ms
 4  * * *
 5  32.130.17.87 (32.130.17.87)  397.904 ms  400.336 ms *
 6  * * *
 7  142.251.60.52 (142.251.60.52)  821.111 ms 142.251.60.136 (142.251.60.136)  814.717 ms *
 8  * 142.251.237.112 (142.251.237.112)  274.556 ms *
 9  108.170.228.81 (108.170.228.81)  426.896 ms 108.170.228.86 (108.170.228.86)  40.08 ms 108.170.228.103 (108.170.228.103)  789.763 ms
10  108.170.231.7 (108.170.231.7)  755.521 ms  760.666 ms *
11  216.239.63.121 (216.239.63.121)  769.92 ms 216.239.62.132 (216.239.62.132)  812.223 ms 216.239.43.144 (216.239.43.144)  780.984 ms
12  142.250.224.25 (142.250.224.25)  785.052 ms 142.250.224.27 (142.250.224.27)  829.763 ms 209.85.241.167 (209.85.241.167)  273.643 ms
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * rq-in-f106.1e100.net (142.250.115.106)  610.026 ms  612.685 ms

Tutor

 • 

13 Messages

1 month ago

That 32.130.x.x block from ATT is just BROKEN.   I've started looking more and more at it and they need to deal with whatever broken hardware is at that location.

(edited)

Tutor

 • 

13 Messages

1 month ago

I noticed that last night my internet connection was terrible, with dropped packets and what not.   Based on some tracerroutes, it looks like ATT's data center is having a problem both in IPV4 and IPV6.  I used to have latency in the 10-12ms range for hitting google.com.  Now I have 30-50ms and dropped packets.   Is there a method to reach a competent network engineer to share these results with?  I can't bear to deal with L1/L2 front line support where they always want to blame the enduser.

  2     2 ms     4 ms     2 ms  71.149.77.250
  3     *        *        *     Request timed out.
  4    35 ms    36 ms    38 ms  32.130.17.87
  5     *        *       40 ms  108.170.231.42
  6    36 ms    36 ms    36 ms  142.251.71.113
  7    37 ms    35 ms    36 ms  dns.google [8.8.8.8]

Tracing route to ipv6.l.google.com [2607:f8b0:4007:817::200e]
over a maximum of 30 hops:

  1     1 ms     2 ms     2 ms  2001:506:6000:100:76:253:217:49
  2     2 ms     5 ms     4 ms  2001:506:6000:100:76:253:217:50
  3    10 ms     *        *     2001:1890:f6:779::2
  4    10 ms     8 ms     8 ms  2001:1890:fff:f6e:12:255:10:124
  5    10 ms    10 ms     8 ms  2607:f8b0:832b::1
  6     *        *        *     Request timed out.
  7     *        9 ms     *     2001:4860:0:1::2032
  8    11 ms     *        *     2001:4860::c:4001:e559
  9     *        *        *     Request timed out.
 10    39 ms    38 ms    38 ms  2001:4860::9:4003:11cc
 11    39 ms     *       40 ms  2001:4860::12:0:aba3
 12    37 ms    37 ms    37 ms  2001:4860:0:1::56fb
 13    37 ms    37 ms    37 ms  lax31s16-in-x0e.1e100.net [2607:f8b0:4007:817::200e]

2     4 ms     2 ms     5 ms  71.149.77.250
  3     *        *        *     Request timed out.
  4     *       43 ms     *     32.130.16.9
  5     *       35 ms    39 ms  gar25.dlstx.ip.att.net [12.122.85.233]
  6    37 ms    37 ms    38 ms  192.205.36.222
  7    39 ms    40 ms    38 ms  be2764.ccr32.dfw01.atlas.cogentco.com [154.54.47.213]
  8    47 ms     *       48 ms  be2433.ccr22.mci01.atlas.cogentco.com [154.54.3.214]
  9    59 ms    56 ms    57 ms  be2832.ccr42.ord01.atlas.cogentco.com [154.54.44.170]
 10    58 ms    62 ms    61 ms  be2125.rcr21.ord04.atlas.cogentco.com [154.54.82.202]
 11     *        *        *     Request timed out.
 12    58 ms    58 ms     *     vl202.chi-cs1-dist-1.cdn77.com [138.199.0.235]
 13    63 ms     *       59 ms  dns.adguard.com [94.140.15.15]

 2     2 ms     5 ms     4 ms  2001:506:6000:100:76:253:217:50
  3    10 ms     *        *     2001:1890:f6:779::2
  4    10 ms     8 ms     8 ms  2001:1890:fff:f6e:12:255:10:124
  5    10 ms    10 ms     8 ms  2607:f8b0:832b::1
  6     *        *        *     Request timed out.
  7     *        9 ms     *     2001:4860:0:1::2032
  8    11 ms     *        *     2001:4860::c:4001:e559
  9     *        *        *     Request timed out.
 10    39 ms    38 ms    38 ms  2001:4860::9:4003:11cc
 11    39 ms     *       40 ms  2001:4860::12:0:aba3
 12    37 ms    37 ms    37 ms  2001:4860:0:1::56fb
 13    37 ms    37 ms    37 ms  lax31s16-in-x0e.1e100.net [2607:f8b0:4007:817::200e]


Their network is just broken.

tracert ipv6.google.com

Tracing route to ipv6.l.google.com [2607:f8b0:4007:819::200e]
over a maximum of 30 hops:

  1     1 ms     1 ms     2 ms  2001:506:6000:100:76:253:217:49
  2     4 ms     5 ms     6 ms  2001:506:6000:100:76:253:217:50
  3   487 ms   496 ms   578 ms  2001:1890:f6:779::2
  4  1035 ms  1105 ms   957 ms  2001:1890:fff:f6e:12:255:10:124
  5   613 ms   627 ms   863 ms  2607:f8b0:8326::1
  6   740 ms   695 ms   701 ms  2001:4860:0:1::5692
  7     *        *        *     Request timed out.
  8     *        *        *     Request timed out.
  9     *        *        *     Request timed out.
 10   576 ms   610 ms   681 ms  2001:4860::9:4003:11cd
 11   637 ms   664 ms   620 ms  2001:4860:0:1110::1
 12   518 ms   504 ms   607 ms  2001:4860:0:1::6519
 13   616 ms   774 ms   610 ms  lax17s55-in-x0e.1e100.net [2607:f8b0:4007:819::200e]

Trace complete.

ping ipv6.google.com

Pinging ipv6.l.google.com [2607:f8b0:4007:819::200e] with 32 bytes of data:
Reply from 2607:f8b0:4007:819::200e: time=676ms
Reply from 2607:f8b0:4007:819::200e: time=732ms
Reply from 2607:f8b0:4007:819::200e: time=596ms
Reply from 2607:f8b0:4007:819::200e: time=629ms

Ping statistics for 2607:f8b0:4007:819::200e:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 596ms, Maximum = 732ms, Average = 658ms

I noticed that last night my internet connection was terrible, with dropped packets and what not.   Based on some tracerroutes, it looks like ATT's data center is having a problem both in IPV4 and IPV6.  I used to have latency in the 10-12ms range for hitting google.com.  Now I have 30-50ms and dropped packets.   Is there a method to reach a competent network engineer to share these results with?  I can't bear to deal with L1/L2 front line support where they always want to blame the enduser.

  2     2 ms     4 ms     2 ms  71.149.77.250
  3     *        *        *     Request timed out.
  4    35 ms    36 ms    38 ms  32.130.17.87
  5     *        *       40 ms  108.170.231.42
  6    36 ms    36 ms    36 ms  142.251.71.113
  7    37 ms    35 ms    36 ms  dns.google [8.8.8.8]

Tracing route to ipv6.l.google.com [2607:f8b0:4007:817::200e]
over a maximum of 30 hops:

  1     1 ms     2 ms     2 ms  2001:506:6000:100:76:253:217:49
  2     2 ms     5 ms     4 ms  2001:506:6000:100:76:253:217:50
  3    10 ms     *        *     2001:1890:f6:779::2
  4    10 ms     8 ms     8 ms  2001:1890:fff:f6e:12:255:10:124
  5    10 ms    10 ms     8 ms  2607:f8b0:832b::1
  6     *        *        *     Request timed out.
  7     *        9 ms     *     2001:4860:0:1::2032
  8    11 ms     *        *     2001:4860::c:4001:e559
  9     *        *        *     Request timed out.
 10    39 ms    38 ms    38 ms  2001:4860::9:4003:11cc
 11    39 ms     *       40 ms  2001:4860::12:0:aba3
 12    37 ms    37 ms    37 ms  2001:4860:0:1::56fb
 13    37 ms    37 ms    37 ms  lax31s16-in-x0e.1e100.net [2607:f8b0:4007:817::200e]

2     4 ms     2 ms     5 ms  71.149.77.250
  3     *        *        *     Request timed out.
  4     *       43 ms     *     32.130.16.9
  5     *       35 ms    39 ms  gar25.dlstx.ip.att.net [12.122.85.233]
  6    37 ms    37 ms    38 ms  192.205.36.222
  7    39 ms    40 ms    38 ms  be2764.ccr32.dfw01.atlas.cogentco.com [154.54.47.213]
  8    47 ms     *       48 ms  be2433.ccr22.mci01.atlas.cogentco.com [154.54.3.214]
  9    59 ms    56 ms    57 ms  be2832.ccr42.ord01.atlas.cogentco.com [154.54.44.170]
 10    58 ms    62 ms    61 ms  be2125.rcr21.ord04.atlas.cogentco.com [154.54.82.202]
 11     *        *        *     Request timed out.
 12    58 ms    58 ms     *     vl202.chi-cs1-dist-1.cdn77.com [138.199.0.235]
 13    63 ms     *       59 ms  dns.adguard.com [94.140.15.15]

 2     2 ms     5 ms     4 ms  2001:506:6000:100:76:253:217:50
  3    10 ms     *        *     2001:1890:f6:779::2
  4    10 ms     8 ms     8 ms  2001:1890:fff:f6e:12:255:10:124
  5    10 ms    10 ms     8 ms  2607:f8b0:832b::1
  6     *        *        *     Request timed out.
  7     *        9 ms     *     2001:4860:0:1::2032
  8    11 ms     *        *     2001:4860::c:4001:e559
  9     *        *        *     Request timed out.
 10    39 ms    38 ms    38 ms  2001:4860::9:4003:11cc
 11    39 ms     *       40 ms  2001:4860::12:0:aba3
 12    37 ms    37 ms    37 ms  2001:4860:0:1::56fb
 13    37 ms    37 ms    37 ms  lax31s16-in-x0e.1e100.net [2607:f8b0:4007:817::200e]

6 Messages

1 month ago

I have my PC directly connected to the BGW210 gateway via ethernet and have been getting about 1-5 Mbps down, 400 up since 2:30 central time.  Similar thing happened last night but seemed to resolve itself after 30 minutes.  ONT has solid green power and pon lights with flashing yellow data light.  I'm really hoping this isn't another recurring regional issue like what happened in January :(.  Anybody else in the area getting this as well?

Note: This comment was created from a merged conversation originally titled Slow Fiber 1000 in Austin, TX area

42 Messages

Same thing in Pflugerville! It feels like a repeat of this event from January all over again:

https://forums.att.com/conversations/att-fiber-equipment/horrible-evening-connectivity-and-timeouts-in-the-austindallas-areas-now-on-its-11th-day/65ac791e46076f4da64722b0

My AT&T account does say there is an outage in my area and it will be resolved soon, but it says that pretty much every time I log in, so I don't put much faith in it.

42 Messages

And since we love traceroutes, we go from 3 ms to 400 ms (and timeouts) all within AT&T's network:

$ traceroute -w 1 www.google.com 
traceroute: Warning: www.google.com has multiple addresses; using 142.250.115.106
traceroute to www.google.com (142.250.115.106), 64 hops max, 40 byte packets
 1  192.168.1.254 (192.168.1.254)  0.37 ms  0.268 ms  0.471 ms
 2  99-67-236-1.lightspeed.austtx.sbcglobal.net (99.67.236.1)  1.633 ms  1.401 ms  1.481 ms
 3  71.149.77.250 (71.149.77.250)  2.955 ms  2.987 ms  2.997 ms
 4  * * *
 5  32.130.17.87 (32.130.17.87)  397.904 ms  400.336 ms *
 6  * * *
 7  142.251.60.52 (142.251.60.52)  821.111 ms 142.251.60.136 (142.251.60.136)  814.717 ms *
 8  * 142.251.237.112 (142.251.237.112)  274.556 ms *
 9  108.170.228.81 (108.170.228.81)  426.896 ms 108.170.228.86 (108.170.228.86)  40.08 ms 108.170.228.103 (108.170.228.103)  789.763 ms
10  108.170.231.7 (108.170.231.7)  755.521 ms  760.666 ms *
11  216.239.63.121 (216.239.63.121)  769.92 ms 216.239.62.132 (216.239.62.132)  812.223 ms 216.239.43.144 (216.239.43.144)  780.984 ms
12  142.250.224.25 (142.250.224.25)  785.052 ms 142.250.224.27 (142.250.224.27)  829.763 ms 209.85.241.167 (209.85.241.167)  273.643 ms
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * rq-in-f106.1e100.net (142.250.115.106)  610.026 ms  612.685 ms

42 Messages

1 month ago

Latency and packet loss returned to normal at 8:29 PM CDT last night. I will continue logging in case there is a recurrence this afternoon/evening.

My AT&T account page claims there is no longer an outage in my area, but we'll see if there is any correlation.

Tutor

 • 

13 Messages

1 month ago

Yes, I see the same thing on my side.  Performance and packet transmission have returned. 

42 Messages

1 month ago

No issues this evening. Hopefully it was a one-time thing.

Not finding what you're looking for?
New to AT&T Community?
New to the AT&T Community? Start by visiting the Community How-To.
New to the AT&T Community?
Visit the Community How-To.