Covad Blogger Relations - Baselining to Start
I mentioned several days ago that as part of the Covad Blogger Relations program, I now have a Covad T1 installed in my home office.
I've been wanting to do some baseline comparisons before I start testing other things, and I want to share them here. This post has a rather extensive collection of screenshots in small sizes. All are linked to the originals in my Covad Blogger Relations Program set on Flickr. Please note that to keep this post reasonable in length, I have not posted all the screenshot comparisons and information here. There are screenshots in the Flickr set with more information.
For testing consistency, http://myvoipspeed.visualware.com/ was used for all the testing in this post. The test were all completed within a 15 minute window around 6pm. Let's take a look at how things baseline across the three connections.
Speed
First, let's take a look at raw speed comparisons.
Covad T1 Speed
Interesting marks for a T1. 1.29 Mbps download is close enough to T1 speed. 757 kbps upload speed tells me that I am not getting symmetric bandwidth at the moment. I did when I looked at this the other day. I've gone back and rerun the test and am now getting 1.29 Mbps down and 1.09 Mbps up. There may have been an anomaly there, but I'm using the original test for the moment.
Qwest DSL Speed
To be honest, the Qwest DSL speed surprised me. When it first went in two years ago, it was pretty fast and snappy, but over time, with more users, it's always felt slow and sluggish in operation. Measurement shows it on par with the T1 at 1.16 Mbps downstream, but a pitiful 104 kbps upstream.
Comcast Cable Speed
The cable connection brought another surprise. It has always "felt" faster than the DSL connection. Yet it measures a miserable 318 kbps downstream, but beats the DSL with 198 kbps up. Clearly Comcast is doing some oversubscription or under heavy load at the time.
It is noteworthy that both DSL and cable modem are shared network topologies. We should expect degraded service during peak usage periods, and the 6pm test definitely hits a peak period. Kids are home from school gaming and surfing and parents are home. It's a busy hour on the Net.
We should also expect the Covad T1 to be rock solid consistent in throughput performance. Interesting that it wasn't. I'll test it daily for a week and see if there's an issue there. For those interested, the T1 is running on Frame Relay using PPPoA and Internetworking (FRF.8) over an ATM infrastructure.
VoIP Support
Now let's look at how well they each might support VoIP services. That's one of the reasons for the T1 in the first place, and I expect to be doing quite a bit of VoIP testing and comparison across these three services.
Covad T1 VoIP
3.2 ms average jitter and 0% packet loss is looking pretty good.
Qwest DSL VoIP
At 2.8ms average jitter and 0% packet loss, the Qwest DSL actually beat the Covad T1 here.
Comcast Cable VoIP
As we might have expected, the Comcast cable comes in sucking the hindquarter with an absolutely deplorable 1112.8 ms average jitter. Yes, over a full second average jitter. That's essentially comparable to two tin cans and a string. Add 12.4% packet loss and what we have is a crappy circuit that isn't much good. Comcast will be hearing from me real soon, with some documented performance testing that I'll go public with once done. I'm setting up an analyzer to do 24X7 monitoring for a week to see just how badly they're ripping off customers.
Consolidated Test Results (text form)
Here are some more detailed test results that were available in text format for comparison across the three. These are based on more comprehensive analysis shown in other screenshots or behind the scenes.
Covad - T1 service
Speed test statistics
---------------------
Download speed: 1299720 bps
Upload speed: 757200 bps
Quality of service: 98 %
Download test type: socket
Upload test type: HTTP
Maximum download pause: 67 ms
Average download pause: 9 ms
Minimum round trip time to server: 92 ms
Average round trip time to server: 94 ms
VoIP test statistics
--------------------
Jitter: you --> server: 3.2 ms
Jitter: server --> you: 3.6 ms
Packet loss: you --> server: 0.0 %
Packet loss: server --> you: 0.0 %
Packet discards: 0.0 %
Packets out of order: 0.0 %
Number of supported VoIP lines: 12
Estimated MOS score: 4.0
Qwest - Consumer DSL service
Speed test statistics
---------------------
Download speed: 1166680 bps
Upload speed: 104960 bps
Quality of service: 77 %
Download test type: socket
Upload test type: HTTP
Maximum download pause: 52 ms
Average download pause: 10 ms
Minimum round trip time to server: 97 ms
Average round trip time to server: 100 ms
VoIP test statistics
--------------------
Jitter: you --> server: 2.8 ms
Jitter: server --> you: 3.5 ms
Packet loss: you --> server: 0.0 %
Packet loss: server --> you: 0.0 %
Packet discards: 0.0 %
Packets out of order: 0.0 %
Number of supported VoIP lines: 1
Estimated MOS score: 4.0
Comcast - Cable Modem service
Speed test statistics
---------------------
Download speed: 318768 bps
Upload speed: 198416 bps
Quality of service: 15 %
Download test type: socket
Upload test type: HTTP
Maximum download pause: 1174 ms
Average download pause: 37 ms
Minimum round trip time to server: -- ms
Average round trip time to server: -- ms
VoIP test statistics
--------------------
Jitter: you --> server: 1112.8 ms
Jitter: server --> you: 42.4 ms
Packet loss: you --> server: 12.4 %
Packet loss: server --> you: 0.0 %
Packet discards: 0.5 %
Packets out of order: 0.0 %
Number of supported VoIP lines: 3
Estimated MOS score: 3.0
If we just compare some minimal basics, here's what we get:
Quality of service: Covad T1- 98%, Qwest DSL - 77%, Comcast cable - 15%
VoIP Lines Supported: Covad T1 - 12, Qwest DSL - 1, Comcast cable - 3
MOS Scores - Covad T1 - 4.0, Qwest DSL - 4.0, Comcast cable - 3.0
Clearly web-based measurement tools leave a lot to be desired. A T1 that will support only 12 voice lines would have killed a telco. The results are just the sort of disjointed findings any of the consumer analytics web sites will provide. But they do indicate that the Covad T1 is the strongest and most reliably consistent. The Qwest DSL suprised me by edging out the Comcast cable significantly.
And I was frankly appalled and the measureably inferior service I'm getting from Comcast. More on that later, I'm sure.
I'll keep posting and sharing findings as testing continues.
Technorati Tags: Covad, Qwest, Comcast, baseline performance. performance comparisons
Email This!
Digg it!
Sphere it!
Del.icio.us
Reddit!
Newsvine
Comments
It's not really accurate to say that "both DSL and cable modem are shared network topologies", suggesting (by default) that Covad's T1s don't suffer any kind of sharing effect.
A DSL customer's local loop from the DSLAM to their home is point-to-point and not shared. There may be an oversubscribed uplink from the DSLAM into the service provider's network (if they're using one or more T1s or a DS3), but if it's fiber-based transport the sharing may not occur until the traffic hits the local or regional backhaul to the service provider's core. So sharing is not at the local loop DSL service.
With cable modem service a node may serve several hundred cable modems and the MSO may use one or more 6 MHz wide frequencies to serve the customers on that node. In this case the sharing is on the local drop, and could also be on the transport side from the CMTS to the service provider's core.
In regards to a T1, it's a direct point-to-point link from the subscriber to a service provider's router. The transport of that T1 may run over multiple carriers' networks, but it's bandwidth that 'pinned up' and always available. Available, that is, until it hits the service provider's router, after which it also has a potential for bandwidth over-subscription.
In each case, a service provider's link/connection to the internet is also shared. So when someone tests against a public speed test site there's almost always bandwidth contention and customer's local performance also depends on how much congestion and latency there is between the service provider and their target site. With a T1 the 'pinned up' bandwidth likely enters the service provider's network core deeper than a DSLAM or CMTS, avoiding some of that sharing that DSL and cable modem service may have on the transport. The latencies of a T1 are lower, too, because the modulation, interleaving, and error correction of DSL and DOCSIS (which are in place to deal with that 'lousy' copper) add up.
Frank
Posted by: Frank | June 24, 2007 10:12 AM
Actually, having taught both technologies for 10+ years, I'll stand by that both are absolutely shared, but shared differently. It's true the local loop is dedicated, but a crock that that actually means anything. Telcos all typically oversubscribe the DSLAM hi-cap line on the back end, providing dismal quality. DSL, ADSL and ADLS2 are highly capable technologies. But what's sold by telco carriers is oversubscribed smoke and mirrors nine times out of ten. I say this from assissting those carrier design engineer's with their technical problems. I sort of know whereof I speak in this area. A DS3 on the bacnkend is typically a fractional, and just plain oversubscribed. It's a common problem and in telco COs all over the US.
Cable is shared no matter how you slice it. it's different sharing, but I'm sorry, the concept of houses passed with the cable signal still holds true in Internet delivery for the propoderence of MSOs. In my view @Home Networks, who I worked with closely for their first 3 years in business, was the only cable provider who really delivered in solid engineering in the earlier implementations. Some MSOs do well, other do poorly, bvut it is, at it's core, a shared environment.
I don't think we really disagree Frank. They're all great technologies, but each has to either be engineered for maximum performance, or sold for what it is. Too often DSL and cable are sold as more than they're designed to deliver.
Posted by: Ken Camp | June 24, 2007 8:28 PM