Ask a question
Search in U-verse Forums

U-verse Forums

Reply
Posted Nov 23, 2008
8:20:52 PM
How To Measure Your Internet Usage Using VLANs and MRTG

With all the talk of the possible upcoming internet usage caps, I was wondering if there was a way that I could measure my own internet usage.  Although it looks like AT&T will provide us a usage meter that we can check to see the total usage on our own accounts, I wanted to see if I could measure my own usage myself as a cross-check on AT&T.

 

Some people have suggested downloading and installing a personal internet usage meter on your computer.  While this works well in single-computer environments, it doesn't solve the problem when there is more than one computer on your network.  You need a measurement system that can measure all internet traffic from all computers and internet devices.

 

A while back, I came up with a method that could measure the RG's total bandwidth usage using MRTG.  (See this thread over at UverseUsers).  However, this only showed total bandwidth, without regard to what bandwidth was being consumed by TV and what was being used by computers.  Since the proposed internet usage limits would be on computer use only, I needed a method to split the bandwidth usage statistics into IPTV and Internet.

 

To do this, I split up my network using VLAN-capable switches, and then used MRTG to measure only the computer bandwidth portion.

The switches I'm using are Netgear GS-108T switches, which can be had for only $79.00 each for an 8-port Gigabit managed switch -- a really good bargain.  The following is a diagram of my home network -- all connections throughout my network are Cat5e (no coax is used):

 

 

The trick is that the main switch (Switch 1) is uplinked to the RG twice.  Port 1 is a member of VLAN 1, which is carrying all Internet traffic, and port 2 is a member of VLAN 2, carrying IPTV traffic.  This enables MRTG to query port 1 of the switch to get traffic counters for only Internet traffic.  All IPTV traffic travels to/from the RG through port 2, so that traffic is not included in the port 1 counters.

 

The green lines on the diagram are Cat5e links that are carrying 802.1q VLAN-tagged frames.  These Ethernet frames have an additional header on them that tells the switch what VLAN they belong to.  When they get to the switch on the other end, that switch strips off the VLAN tag, and switches the packet only to the ports allowed by that VLAN.  What this essentially means is that the only place in the network where packets can go from VLAN 1 to VLAN 2 (or vice versa) is through the RG.  So, if Desktop 1 tries to ping the Office STB, the ping packet will travel all the way up to the RG and back to reach the Office STB.

 

The reason that switch 1 is uplinked to the RG twice is because the RG is not VLAN-aware.  To the RG, it looks like all 3 STBs are on port 2, and all computer hardware is on port 1.  The RG doesn't know that the network is actually mixed together.

 

The computer labeled as the video server actually runs MRTG.  It queries both the RG (for total VDSL statistics, error counts, and line sync rate/capacity) and switch 1 (to get traffic counters for IPTV and Internet off ports 1 and 2).  These parameters are graphed on the MRTG summary page (available here).

 

The final graph on that page, which shows the total Internet data transferred for the last 30 days, is generated on the fly from the MRTG log files.  If anyone is interested and wants to set this up themselves, I'll provide the source code for that (it's VBscript/ASP).

 

Here's some other examples of how data flows through the network for various functions:

 

1. Watching a live TV program on the Office STB.  Note that this traffic doesn't get counted as Internet traffic because it goes through port 2 of switch 1 (only traffic through port 1 of switch 1 gets counted by MRTG):

 

 

2. Watching a DVD from the Video Server on the Living Room XBox Media Center.  Note that this traffic doesn't get counted anywhere by MRTG because it doesn't traverse port 1 of switch 1:

 

 

3. Downloading a file from the Internet to Desktop 2.  This gets counted as Internet traffic since it flows through switch 1 port 1:

 

 

4. Now the really cool one. :smileyhappy:  Watching a recorded program from the DVR, using THDVR, from a remote location using the Slingbox.  This will get counted as Internet traffic in the upload direction as it traverses switch 1 port 1:

 

 

Now, here are some problems you may run into with the above setup:

 

1. Coax devices cannot get their traffic counted.  Now, this isn't a problem if you're only interested in counting Internet traffic.  However, if you have used the trick where you have an STB connected via coax, and then have used the Cat5e port on the STB as a bridge to a computing device, you won't be able to count the traffic from that computing device.

 

2. You cannot use the RG's built-in wireless if you want to count that traffic.  You must use an external 802.11g/n wireless access point connected to the Internet VLAN to get wireless devices' traffic counted.  This is why I have shown a wireless access point in my diagrams (I haven't installed that yet).

 

3. MRTG runs as a service, so it's required to be installed on a computer that runs 24/7. 

 

I know this setup goes beyond what many people may want to do, but I hope it gives you some ideas of what you might want to try with your network.

 

Message Edited by SomeJoe7777 on 11-24-2008 11:27 AM

With all the talk of the possible upcoming internet usage caps, I was wondering if there was a way that I could measure my own internet usage.  Although it looks like AT&T will provide us a usage meter that we can check to see the total usage on our own accounts, I wanted to see if I could measure my own usage myself as a cross-check on AT&T.

 

Some people have suggested downloading and installing a personal internet usage meter on your computer.  While this works well in single-computer environments, it doesn't solve the problem when there is more than one computer on your network.  You need a measurement system that can measure all internet traffic from all computers and internet devices.

 

A while back, I came up with a method that could measure the RG's total bandwidth usage using MRTG.  (See this thread over at UverseUsers).  However, this only showed total bandwidth, without regard to what bandwidth was being consumed by TV and what was being used by computers.  Since the proposed internet usage limits would be on computer use only, I needed a method to split the bandwidth usage statistics into IPTV and Internet.

 

To do this, I split up my network using VLAN-capable switches, and then used MRTG to measure only the computer bandwidth portion.

The switches I'm using are Netgear GS-108T switches, which can be had for only $79.00 each for an 8-port Gigabit managed switch -- a really good bargain.  The following is a diagram of my home network -- all connections throughout my network are Cat5e (no coax is used):

 

 

The trick is that the main switch (Switch 1) is uplinked to the RG twice.  Port 1 is a member of VLAN 1, which is carrying all Internet traffic, and port 2 is a member of VLAN 2, carrying IPTV traffic.  This enables MRTG to query port 1 of the switch to get traffic counters for only Internet traffic.  All IPTV traffic travels to/from the RG through port 2, so that traffic is not included in the port 1 counters.

 

The green lines on the diagram are Cat5e links that are carrying 802.1q VLAN-tagged frames.  These Ethernet frames have an additional header on them that tells the switch what VLAN they belong to.  When they get to the switch on the other end, that switch strips off the VLAN tag, and switches the packet only to the ports allowed by that VLAN.  What this essentially means is that the only place in the network where packets can go from VLAN 1 to VLAN 2 (or vice versa) is through the RG.  So, if Desktop 1 tries to ping the Office STB, the ping packet will travel all the way up to the RG and back to reach the Office STB.

 

The reason that switch 1 is uplinked to the RG twice is because the RG is not VLAN-aware.  To the RG, it looks like all 3 STBs are on port 2, and all computer hardware is on port 1.  The RG doesn't know that the network is actually mixed together.

 

The computer labeled as the video server actually runs MRTG.  It queries both the RG (for total VDSL statistics, error counts, and line sync rate/capacity) and switch 1 (to get traffic counters for IPTV and Internet off ports 1 and 2).  These parameters are graphed on the MRTG summary page (available here).

 

The final graph on that page, which shows the total Internet data transferred for the last 30 days, is generated on the fly from the MRTG log files.  If anyone is interested and wants to set this up themselves, I'll provide the source code for that (it's VBscript/ASP).

 

Here's some other examples of how data flows through the network for various functions:

 

1. Watching a live TV program on the Office STB.  Note that this traffic doesn't get counted as Internet traffic because it goes through port 2 of switch 1 (only traffic through port 1 of switch 1 gets counted by MRTG):

 

 

2. Watching a DVD from the Video Server on the Living Room XBox Media Center.  Note that this traffic doesn't get counted anywhere by MRTG because it doesn't traverse port 1 of switch 1:

 

 

3. Downloading a file from the Internet to Desktop 2.  This gets counted as Internet traffic since it flows through switch 1 port 1:

 

 

4. Now the really cool one. :smileyhappy:  Watching a recorded program from the DVR, using THDVR, from a remote location using the Slingbox.  This will get counted as Internet traffic in the upload direction as it traverses switch 1 port 1:

 

 

Now, here are some problems you may run into with the above setup:

 

1. Coax devices cannot get their traffic counted.  Now, this isn't a problem if you're only interested in counting Internet traffic.  However, if you have used the trick where you have an STB connected via coax, and then have used the Cat5e port on the STB as a bridge to a computing device, you won't be able to count the traffic from that computing device.

 

2. You cannot use the RG's built-in wireless if you want to count that traffic.  You must use an external 802.11g/n wireless access point connected to the Internet VLAN to get wireless devices' traffic counted.  This is why I have shown a wireless access point in my diagrams (I haven't installed that yet).

 

3. MRTG runs as a service, so it's required to be installed on a computer that runs 24/7. 

 

I know this setup goes beyond what many people may want to do, but I hope it gives you some ideas of what you might want to try with your network.

 

Message Edited by SomeJoe7777 on 11-24-2008 11:27 AM
0
(0)
  • Rate this reply
View profile
Solved
Mar 18, 2011 9:17:42 AM
0
(0)
Expert

The graphics from the first post were lost in the forum move a few months ago, so I'm reposting them here.

 

 

Master diagram:

 

 

1. Watching a live TV program on the Office STB.  Note that this traffic doesn't get counted as Internet traffic because it goes through port 2 of switch 1 (only traffic through port 1 of switch 1 gets counted by MRTG):

 

 

 

2. Watching a DVD from the Video Server on the Living Room XBox Media Center.  Note that this traffic doesn't get counted anywhere by MRTG because it doesn't traverse port 1 of switch 1:

 

 

 

3. Downloading a file from the Internet to Desktop 2.  This gets counted as Internet traffic since it flows through switch 1 port 1:

 

 

 

4. Now the really cool one. Smiley Happy  Watching a recorded program from the DVR, using THDVR, from a remote location using the Slingbox.  This will get counted as Internet traffic in the upload direction as it traverses switch 1 port 1:

 

 

 

Accepted Solution

How To Measure Your Internet Usage Using VLANs and MRTG

16,905 views
32 replies
(0) Me too
(0) Me too
Reply
View all replies
(32)
0
(0)
  • Rate this reply
View profile
Jan 17, 2012 7:14:04 AM
0
(0)
Expert
You cannot use only one connection because the RG does not support VLAN tagging. There would be no way to differentiate what traffic coming from the RG belonged to which VLAN.
You cannot use only one connection because the RG does not support VLAN tagging. There would be no way to differentiate what traffic coming from the RG belonged to which VLAN.

Re: How To Measure Your Internet Usage Using VLANs and MRTG

31 of 33 (1,927 Views)
0
(0)
  • Rate this reply
View profile
Jun 11, 2012 10:27:03 PM
0
(0)
Tutor

Dan,

 

Do you have details of how you configured your Netgear gs108t switches?  I'm trying to replicate what you have and think I have it right however I'm still getting very choppy TV playback.  Is IGMP snooping supposed to be enabled?  How about jumbo frames?  Any further details on the RG and Netgear configs would be appreciated.

Dan,

 

Do you have details of how you configured your Netgear gs108t switches?  I'm trying to replicate what you have and think I have it right however I'm still getting very choppy TV playback.  Is IGMP snooping supposed to be enabled?  How about jumbo frames?  Any further details on the RG and Netgear configs would be appreciated.

Re: How To Measure Your Internet Usage Using VLANs and MRTG

32 of 33 (1,806 Views)
Highlighted
0
(0)
  • Rate this reply
View profile
Jun 12, 2012 4:16:48 PM
0
(0)
Expert

No, you should have neither IGMP Snooping nor Jumbo Frames enabled.

Do you have the right kind of switches?  This works only with the GS108T switches, not the GS108E switches,

 

Basically, all you should have to do is define 2 VLANs, one for Internet and one for IPTV.  On your main switch near the RG, make port 1 = Internet and port 2 = IPTV.  Uplink BOTH ports to the RG (separately, using 2 different Ethernet cables).

 

Now configure ports on the switch based on what type of device is plugged in.  If it's a computer, assign the port to the Internet VLAN.  If it's a DVR/STB, assign it to the IPTV VLAN.  If it's another GS108T switch, configure the port for Tagged Frames on both VLANs (you will have to do this for the ports on both ends of the connection).

 

Remember when you configure a port that it needs to be assigned as "untagged" (U) on the VLAN it's supposed to be a member of, AND you need to change the PVID of the port to the corresponding VLAN number.  If VLAN 1 is the Internet VLAN and VLAN 2 is the IPTV VLAN, then to configure a port to be a member of the IPTV VLAN, it needs 3 separate settings:

 

Membership on VLAN 1 should be blank (not a member)

Membership on VLAN 2 should be untagged (U)

PVID of the port has to be 2

 

To do the tagged ports for downstream GS108T switches, do 3 settings:

 

Membership on VLAN 1 should be tagged (T)

Membership on VLAN 2 should be tagged (T)

PVID of the port should be 1 (also can be 2, it doesn't matter when running only tagged frames on the link)

 

Once you've got all ports configured, then the multicast traffic on the IPTV VLAN should not reach the Internet VLAN.

 

No, you should have neither IGMP Snooping nor Jumbo Frames enabled.

Do you have the right kind of switches?  This works only with the GS108T switches, not the GS108E switches,

 

Basically, all you should have to do is define 2 VLANs, one for Internet and one for IPTV.  On your main switch near the RG, make port 1 = Internet and port 2 = IPTV.  Uplink BOTH ports to the RG (separately, using 2 different Ethernet cables).

 

Now configure ports on the switch based on what type of device is plugged in.  If it's a computer, assign the port to the Internet VLAN.  If it's a DVR/STB, assign it to the IPTV VLAN.  If it's another GS108T switch, configure the port for Tagged Frames on both VLANs (you will have to do this for the ports on both ends of the connection).

 

Remember when you configure a port that it needs to be assigned as "untagged" (U) on the VLAN it's supposed to be a member of, AND you need to change the PVID of the port to the corresponding VLAN number.  If VLAN 1 is the Internet VLAN and VLAN 2 is the IPTV VLAN, then to configure a port to be a member of the IPTV VLAN, it needs 3 separate settings:

 

Membership on VLAN 1 should be blank (not a member)

Membership on VLAN 2 should be untagged (U)

PVID of the port has to be 2

 

To do the tagged ports for downstream GS108T switches, do 3 settings:

 

Membership on VLAN 1 should be tagged (T)

Membership on VLAN 2 should be tagged (T)

PVID of the port should be 1 (also can be 2, it doesn't matter when running only tagged frames on the link)

 

Once you've got all ports configured, then the multicast traffic on the IPTV VLAN should not reach the Internet VLAN.

 

Re: How To Measure Your Internet Usage Using VLANs and MRTG

33 of 33 (1,797 Views)
Advanced
You must be signed in to add attachments
Share this post
Share this post