Net100's TCP Vegas

We have done considerable experimentation with TCP Vegas under ns, and we have an implementation in our TCP-over-UDP (more info). We have also done some exeprimenting with CalTech's FAST but lacking access to the FAST source, we proceeded to port an earlier Linux implementation into the Net100 Linux kernel.

On firebird, the Net100 TCP Vegas can be turned on by the WAD on a per-flow basis. There are WAD variables for the Vegas alpha, beta, and gamma, and there is Web100 instrumentation to count Vegas cwnd increases, decreases, and the Vegas delta value. The Net100 Vegas uses a hi-res kernel timer (do_gettimeofday) by default, but with a sysctl (net.ipv4.tcp_vegas_timres ) you can disable the hires timer. The mixing of hires timestamps with standard timestamps when vegas is first enabled by WAD is still a matter of "interest".

Vegas tries to sense queueing delays by observing changes in throughput each RTT and modulating cwnd accordingly to avoid loss. This version of Vegas does NOT use gamma during the initial slow-start -- we're considering it. We are also considering supporting the virtual MSS or Floyd's AI during Vegas. Since Vegas operates when TCP is not in congestion avoidance, it is possible to also enable the various Net100 acclerated recovery options (Floyd's HS TCP, Scalable TCP, virtual MSS, or other AIMD settings). To compete with standard TCP, larger values of alpha/beta are required for high-speed links.

Vegas can be enabled on a per flow basis with the WAD, and there are systcl variables if you want to set things for the whole kernel. The WAD overrides the sysctl settings. WAD_Gamma is the slow-start threshold, set this to a big number (999999) if you don't want Vegas to mess with slow-start when Vegas is enabled. Instrumentation variables report Vegas cwnd increments/decrements or "no change" based on the Vegasdiff compared to alpha/beta. You can also view Vegasdiff. WAD_VegasGTGamma counts number times Vegas forced slow-start exit. Veg

WAD variables Vegas control WAD_Vegas WAD_VegasAlpha WAD_VegasBeta WAD_VegasGamma if these values are not set by the WAD, they default to values in the sysctl variables. Vegas instrumentation WAD_VegasNoChange WAD_VegasGTBeta WAD_VegasLTAlpha WAD_VegasGTGamma VegasDiff Vegas sysctl (WAD overrides these) net.ipv4.tcp_vegas_timres = 1 net.ipv4.tcp_vegas_gamma = 2 net.ipv4.tcp_vegas_beta = 3 net.ipv4.tcp_vegas_alpha = 1 net.ipv4.tcp_vegas_cong_avoid = 0 The WAD's config file variables are # wad_vegas 1 (1=enable vegas, 0=disable vegas) # gamma 2 vegas values for gamma, beta, alpha-- # beta 3 used when vegas is enabled (above) # alpha 1

A vegas-capable WAD is available at ~ffowler/new/wad-vegas/ on firebird

Experimental Results (preliminary)

We have experimented on a 100T machine over NISTnet and the widearea net. With sendstall's "enabled", Vegas clearly improves TCP's performance by reducing sendstall events. If the send buffer is bigger than required by the bandwidth-delay product, Vegas limits cwnd to the bandwidth-delay product as predicted.

The following graph shows 4 different tests from ORNL to LBL, with standard Linux TCP (which gets a loss), and with WAD-tuned Vegas. Buffers are set at 8 MB. Gamma is 200, which is what cuts off the slow-start. One case is with alpha/beta 6/8, the other with 60/80 plus one with the hires timer option (which performs slightly better).


RTT for the Vegas runs stays around 70 ms, but for the standard Linux run, RTT goes to 80 ms then 90 ms before the loss, then returns to 70 ms during recovery.

The following plot is for the same path with a 120 mbs UDP background, both Vegas and standard Linux TCP run at about 300 mbs, but standard TCP oversubsribes the send buffer for the available bandwidth on the path (OC12).

See download page for experimental ORNL Vegas patches. See the many Vegas references and our atou vegas experiments early '03.


Last Modified Monday, 29-Mar-2004 13:46:39 EST thd@ornl.gov (touches: 189368 )
back to Tom Dunigan's page or the Net100 page or the ORNL home page