Today, Mozilla is using SeaMicro’s older 512-core machines to handle downloads of its Firefox browser
Can anyone comment on the accuracy of this? "Handling downloads" is pretty vague, but still it seems like a pretty braindead job, and I know Mozilla is leveraging lots of mirrors and probably CDNs for this.
"Today, when an end user running Firefox wants to download an update, addon or persona, the browser sends a URL request to Mozilla’s download cluster. The servers running Linux, Apache, and PHP run a query to the backend database/memcache cluster and return an HTTP Redirect URL (also called an HTTP 302 return) back to the browser so that Firefox can fetch the desired file from the network of donated download resources."
Let me get this strait, it's basically (request)>(redirect) for a few MB's of data. That seems like the kind of thing where a single CPU could handle ~100,000 requests per second. I know the Firefox team is not exactly well know for writing fast code, but there must be more to it than that.
This could be journalistic laziness and muddling of facts, but there is also something to be said for understanding the sub-millisecond performance considerations of a large professional effort, which may need an assured throughput even with known possible failures, and instantaneous hand off of each packet in as near real time as tuned with other devices, so as to not gum up the works subtly both down and up stream.
and instantaneous hand off of each packet in as near real time as tuned with other devices, so as to not gum up the works subtly both down and up stream.
Wow. You make static HTTP-downloads sound like rocket science.
Honestly, this is as trivial as it gets in terms of scaling. No "512 core machine" needed whatsoever, unless they've started to fold proteins along with their downloads...
You have less than 1kb requests and less than 1kb responses on average.
Let's see how many requests a 1gb net card can do? Well it's bit's not bites, but it's also a full duplex so you have 1gbit up and download. 1gigabyte / (8 kiliobit) = 1,000,000 / 8 = 125,000 requests per second. Granted full TCP/IP has a fair amount of overhead, but if you use UDP you can get vary close to this.
Yet, they don't need to get anywhere near that performance, for every 100 million clients updating every four hours using a random second to connect is around 10k connections a second.
Note that semiconductor sells Atom cards which are about 10% of a Xeon core in power and energy. So 512 core isn't 43 2x6core Mac Pros, it is more like 4.
Comments
Today, Mozilla is using SeaMicro’s older 512-core machines to handle downloads of its Firefox browser
Can anyone comment on the accuracy of this? "Handling downloads" is pretty vague, but still it seems like a pretty braindead job, and I know Mozilla is leveraging lots of mirrors and probably CDNs for this.
"Today, when an end user running Firefox wants to download an update, addon or persona, the browser sends a URL request to Mozilla’s download cluster. The servers running Linux, Apache, and PHP run a query to the backend database/memcache cluster and return an HTTP Redirect URL (also called an HTTP 302 return) back to the browser so that Firefox can fetch the desired file from the network of donated download resources."
http://www.seamicro.com/sites/default/files/MozillaCaseStudy...
Let me get this strait, it's basically (request)>(redirect) for a few MB's of data. That seems like the kind of thing where a single CPU could handle ~100,000 requests per second. I know the Firefox team is not exactly well know for writing fast code, but there must be more to it than that.
This could be journalistic laziness and muddling of facts, but there is also something to be said for understanding the sub-millisecond performance considerations of a large professional effort, which may need an assured throughput even with known possible failures, and instantaneous hand off of each packet in as near real time as tuned with other devices, so as to not gum up the works subtly both down and up stream.
and instantaneous hand off of each packet in as near real time as tuned with other devices, so as to not gum up the works subtly both down and up stream.
Wow. You make static HTTP-downloads sound like rocket science.
Honestly, this is as trivial as it gets in terms of scaling. No "512 core machine" needed whatsoever, unless they've started to fold proteins along with their downloads...
So you're saying Amazon S3 runs on a single CPU? :D I think we're talking about different things.
Network cards and storage adapters the limiting factors, not CPU, that is the point.
You have less than 1kb requests and less than 1kb responses on average.
Let's see how many requests a 1gb net card can do? Well it's bit's not bites, but it's also a full duplex so you have 1gbit up and download. 1gigabyte / (8 kiliobit) = 1,000,000 / 8 = 125,000 requests per second. Granted full TCP/IP has a fair amount of overhead, but if you use UDP you can get vary close to this.
Yet, they don't need to get anywhere near that performance, for every 100 million clients updating every four hours using a random second to connect is around 10k connections a second.
Note that semiconductor sells Atom cards which are about 10% of a Xeon core in power and energy. So 512 core isn't 43 2x6core Mac Pros, it is more like 4.