My home router (Linux box) is configured to shape upstream traffic to just below the link rate to avoid Bufferbloat – this greatly improves interactive performance under load. Recently I’ve experimented with various packet sizes. The charts below show the effect of small packets.
Effect of per-packet overhead on VDSL2?
- Between 0-6 seconds the link is idle.
- From 6-14 seconds the upstream link is flooded with 1,400 byte packets (10Mb/sec of traffic trying to get through a 6.2Mb link)
- At 17 seconds the upstream link is flooded with 64 byte packets (10Mb/sec as well)
Notice how much higher the latency and jitter are with small packets.
Confusingly, these results were gathered with the bandwidth shaper configured for 53 bytes of overhead [1] which is my current understanding of the per-packet overhead on VDSL2 [53 is a coincidence with the ATM cell size].
Per-packet overhead for VDSL2 (without ATM) and PPPoE:
- 5 Bytes for PTM
- 40 bytes for 802.3
- 8 bytes for PPPoE
Either the above overhead numbers or wrong or there is something else going on.
[1] – overhead argument to tc.
Pingback: Per packet overhead on VDSL2 – 2 | Dan Siemon
Pingback: Per packet overhead on VDSL – 3 | Dan Siemon
Your DSL line is not subject to the IP MTU. So, depending on your reference point, there really isn’t 53 bytes of “overhead”, if any at all. PTM adds a minimum of 5 bytes but technically it does not have to be taken into account for the local MTU . So your native MTU can remain at 1500 bytes. If your provider is using PPPoE instead of 802.1x for authorization, then you will see 8 bytes of overhead for that encapsulation reducing the MTU to 1492. That should be it for overhead.
In the case of PTM, it uses and HDLC encapsulation and my understanding is that the local ethernet frame is stripped and only the IP packet is encapsulated. I’m not sure how you are calculating the 802.3 overhead at 40 bytes but it should be irrelevant. 40 bytes is more like the IP packet overhead which shouldn’t be considered either. The 802.3 frame overhead is above and beyond the IP MTU but is typically 18 bytes, anyway.
The big problem with PTM over xDSL is that you run into the serialization delay problem the slower the link is. It takes 12msecs on a 1Meg link to serialize a 1500 byte packet.
The best doc I’ve found explaining this is a comparison between ATM, STM, and PTM for xDSL transport. Funny enough, this article’s intent is to show that PTM may not be the best choice for last mile transport, at least on links with speeds under 10Meg.
http://www.ethernetinthefirstmile.com/articles/WTC2002.pdf
Thank for the information. I’ll read that paper. My understanding is that the 802.3 frame headers are transmitted over the wire. If that’s not the case then my calculation is wrong.
Also, this post is about overhead from the perspective of serialization time in order to stay below the link bandwidth. I’m not proposing the MTU is adjusted other than the 8 bytes for the PPPoE header.
53 bytes overhead for ATM is a misunderstanding. Each ATM cell is 53 bytes, 48 bytes of payload plus 5 byte header, so the usable throughput is 48/53. A 1500-byte datagram sent over ATM would require 32 cells, so the overhead in this case is 32×5 = 160 bytes, a little over 10% wasted.
On my home VDSL line, the modem says it trains to 5,398,000 bps upstream(*), and speedtest.net shows 4.97 Mbps usable. So the overhead is about 8%, not high enough to be ATM; I believe it must be using Packet Transfer Mode (PTM).
OTOH, this overhead seems high just for TCP. With 1500 byte MTU(**), the IP header is 20 bytes and the TCP header 28 bytes (with timestamp option), so payload is 1452.
I don’t know the PTM framing, but a guess of 1520 total frame size would imply 4.5% overhead. Maybe Forward Error Correction accounts for the rest?
(*) The modem is Draytek Vigor 130, which reports the info via web and telnet
(**) The modem and router both support baby jumbo ethernet frames (1508 bytes), so I get a 1500-byte IP MTU even with PPPoE
Correction: I underestimated the ATM overhead. 1500 bytes does indeed require 32 cells, but the cells are of fixed size, meaning the last cell is padded. The overhead is therefore (53×32 – 1500) = 196 bytes, or about 13%.
I just found this recently Dan and realised how late to the party I am in thinking about upstream traffic shaping and bufferbloat.
I think the VDSL2 overhead should be: 8 PPPoE + 22 Ethernet (VLAN tagged) + 5 PTM = 34 bytes. There is also a 64/65-octet encapsulation used with PTM, in the UK at least, which can be accounted for by reducing your target upstream rate by 1?65.