Over the past 2 months we've used Amazon's Mechanical Turk to pay users with high speed, mostly residential Internet connections, to run our cloud speedtest. To date, about 1100 unique users in 115 different countries have run the speedtest. This speedtest tracks the amount of time required to download a 1MB test file from 26 different cloud services (servers and content delivery networks), and records the transfer rate for each test and service run. We use MaxMind's geoip database to track where the user is downloading from (country + state/province for US/Canada). The order of the tests is always random and we enforce a time limit such that users with dial up or slow connections are excluded. We also limited individual users to running the speedtest only once per day. To date, about 1300 tests have been performed. About 60% of the tests were performed from US-based connections.
The purpose of these tests is to compare bandwidth speed from various cloud services to high-speed Internet connections. Businesses can use this data to determine which cloud services will provide the best throughput for their customer base.
We got a fairly diverse group of users to run the speedtest from US-based connections covering all 50 states. Voxel, IBM and EC2 (US East) were the top 3 in this category. IBM's service is still in beta and not intended for production use. The bottom 3 are located in Europe, explaining their slower bandwidth.
Non-US Speedtests to Cloud Servers
Most of the non-US speedtests were run by Canadian, European and Indian users. Linode's London data center performed very well in this category followed by EC2 (EU West) and Flexiscale.
Global Speedtests to Cloud ServersThis table shows the aggregate results from all speedtests (both US and non-US). The top 3 are the same as the US-based results, IBM, Voxel, and EC2 (US East).
US Speedtests to CDNs
The only major CDN we were unable to test is Akamai. This is because, unlike other CDNs, Akamai is still strongly opposed to a paygo model and the new realities of CDN pricing, and we are unable to commit to a $200/Mo, 50GB plan ($4GB!). Update: VPS.NET has announced they will be making Akamai CDN available on a zero commit paygo plan. We will add Akamai to the speedtest as soon as we are able to get setup with an account. More info is available here: http://bit.ly/98IQkC
As with our previous pingdom-based tests, CacheFly performed the best in all of our tests. Edgecast also performed very well. Edgecast is available with a zero-commit paygo plan through GoGrid through which it is very good from a price/features/performance perspective. Surprisingly, the #2 CDN (marketshare-wise) Limelight (via RackspaceCloud CDN) performed very poorly in all of our tests.
Non-US Speedtests to CDNs
Outside of the US, both CacheFly and Edgecast performed very well. Amazon's CloudFront came in 3rd, about 18% slower than CacheFly.
Global Speedtests to CDNs
Globally, CacheFly and Edgecast were neck-to-neck, followed by CloudFront at about 15% slower.
Summary
Bandwidth is certainly not the only consideration when choosing a cloud service vendor. However, it is one factor we believe businesses should consider in combination with quality of support, pricing, features and reliability. The combination of these factors should allow users to make a fairly objective decision about which vendor will provide the best overall service for their business.
On a side note, if you intend to use cloud services for internal-facing applications (e.g. virtual private clouds), bandwidth throughput to your business locations should be a very significant factor in your decision making process.
Our speedtest is available publicly in beta release at http://cloudharmony.com/speedtest
