People surprised by the 100 days requirement have presumably never seen the type of infrastructure required by a top-100 website or had to deal with user-time issues.
Having an independent clock means turning off caching, which is a big deal when you're serving millions of page requests.
Not to mention having to deal with timezone issues, daylight saving, latency, etc. This stuff is hard.
Just add some javascript to load the time from a different server after the cached version is rendered. Build a simple service in node.js, ruby or java to return a json datetime, use that as the endpoint to load the time from. This is not rocket science and should not take 3 months to build, even with all the caching involved.
Edit: I just checked the headers that are sent along with any request to bbc.co.uk, and the response already includes the current datetime. So basically they don't even need to implement a time server anymore, but just do an Ajax request to document.location, and read the Date header on the result, to get the accurate time. They can just upscale their servers a bit to handle the extra requests (or implement an empty-response URL). Why would this take 100 days again?
This is the BBC, not some startup in a garage with a couple of EC2 instances. It's the busiest website in europe, and it's publicly funded. If they just threw up external node services whenever they felt like it there would be chaos.
For example, whenever there's a major news story they see traffic spikes that would make you weep.
Internally on the other hand, where the stakes aren't so high, they make heavy use of rails, node, go and all that cool stuff. They know what they're doing.
Check out this list of some of their machines, from part of their monitoring system which is for some reason left open: http://support.bbc.co.uk/support/rrd
EDIT: you're right that they could probably implement this quite easily, but the point is they can't just throw up an agile solution at the scale of the BBC, even if it seems safe.
The point is that this could be solved by a solution that is completely external to their current infrastructure, meaning they don't even have to mess with any integration or legacy code. Agile in this case seems perfectly suitable because they can easily have a fallback when the server doesn't respond to a "time"-request, that falls back to the current state (i.e. use client-side time). So we've got the perfect storm of no integration with current systems, tiny overhead on requests (it's basically an ECHO server), unimportant feature and easy fallback.
In fact I have an enterprise plan to sell them that does exactly this if they would like to externalise this problem.
Because your OPs and deployment team are going to know exactly how to deploy and support something you cobbled together in a completely arbitrary way.
You obviously thought through the integration with monitoring, scaling and multiple redundancy systems so in the event of traffic spike or disaster it'll keep working without manual intervention.
That's not even touching the business logic complexity of issues like DST or leap seconds where you have to be able to arbitrarily change the clock forward or backward.
Here ya go mate, caching, fallback, all in a 15 lines of javascript. If you want to extract jquery out of there that should be easy enough.
No need for extra servers, the OPs guys don't even need to know it's there. Take an extra performance hit that is equal to loading another image per page (yeah image requests, while probably served from CDN, also include an accurate Date response header as I checked), cached for 7 days (remove the cookie during periods of daylight saving times etc). Just have to make sure the server time is reasonably accurate. I'm sure a few seconds inaccuracy wouldn't matter.
Doesn't work if there's a proxy server caching old versions between the user and the server. Not to mention that according to the RFC standard intermediary servers are allowed to manipulate/remove/alter any non-restricted header (including Date).
Yes you can come up with a hacky solution that works 80% of the time quickly, but if you want a 99% solution you need to put the work in to do it right.
Actually, the Date content header is quite well defined in HTTP/1.1 [0], and should not be manipulated nilly-willy by intermediates. If you're having problems with proxies caching your requests, even if you're asking non-cached content [1], then you're never going to be able to get an accurate time from anywhere since any request might be cached, and no solution would work anyway.
It's not quite a "hacky" solution, I would guesstimate it would work 95% of the time rather than 80%, but even 80% is a great solution for the amount of work involved, the tradeoffs, and the fact that we're talking about a clock displayed on a website.
Or, you know, you could spend £15.000 taxpayers' money over 3 months for something I'm not even sure would be more flexible/maintainable/performant.
If you want that, you can just remove the caching, and it'll always accurately reflect the server time. However who changes their local clock? I think we could definitely get away with caching it for 7 days.
Same thing with DST (I even mentioned it). Also once you have an accurate time, the javascript time object includes the DST logic itself I think. We're only dealing with an inaccurate local clock, we don't have to reimplement whatever ISO standard stands for Time. As soon as we have the delta between local time and server time, we're golden.
Well, I haven't worked there since about 2006, so numbers may have changed, but it was at least 10000 servers of various types (redhat, solaris, windows were the usual suspects I had to build out) of which I was aware (between maidenhead, the various colo's i had some responsibility for, tv center, white-city).
software? i know there was a lot of perl, ruby was gaining some traction, a few different CMS's.. etc, i've (thankfully) forgotten most of the details, but diverse ecosystem is an appropriate phrase here.
undoubtedly things have changed between then and now, but i'd guess it's something similar hardware-wise.
My penultimate day of work was racking up ~300 servers for the i-player farm (i was a system engineer) at red-bus in docklands. My back has only really just started to recover. :/
Comments
People surprised by the 100 days requirement have presumably never seen the type of infrastructure required by a top-100 website or had to deal with user-time issues.
Having an independent clock means turning off caching, which is a big deal when you're serving millions of page requests.
Not to mention having to deal with timezone issues, daylight saving, latency, etc. This stuff is hard.
Just add some javascript to load the time from a different server after the cached version is rendered. Build a simple service in node.js, ruby or java to return a json datetime, use that as the endpoint to load the time from. This is not rocket science and should not take 3 months to build, even with all the caching involved.
Edit: I just checked the headers that are sent along with any request to bbc.co.uk, and the response already includes the current datetime. So basically they don't even need to implement a time server anymore, but just do an Ajax request to document.location, and read the Date header on the result, to get the accurate time. They can just upscale their servers a bit to handle the extra requests (or implement an empty-response URL). Why would this take 100 days again?
Edit2: Sample code here: https://news.ycombinator.com/item?id=5825319
This is the BBC, not some startup in a garage with a couple of EC2 instances. It's the busiest website in europe, and it's publicly funded. If they just threw up external node services whenever they felt like it there would be chaos.
For example, whenever there's a major news story they see traffic spikes that would make you weep.
Internally on the other hand, where the stakes aren't so high, they make heavy use of rails, node, go and all that cool stuff. They know what they're doing.
Check out this list of some of their machines, from part of their monitoring system which is for some reason left open: http://support.bbc.co.uk/support/rrd
Here's is some data about their peering: http://support.bbc.co.uk/support/peering/
Yeah that's right. Peering.
EDIT: you're right that they could probably implement this quite easily, but the point is they can't just throw up an agile solution at the scale of the BBC, even if it seems safe.
The point is that this could be solved by a solution that is completely external to their current infrastructure, meaning they don't even have to mess with any integration or legacy code. Agile in this case seems perfectly suitable because they can easily have a fallback when the server doesn't respond to a "time"-request, that falls back to the current state (i.e. use client-side time). So we've got the perfect storm of no integration with current systems, tiny overhead on requests (it's basically an ECHO server), unimportant feature and easy fallback.
In fact I have an enterprise plan to sell them that does exactly this if they would like to externalise this problem.
Because your OPs and deployment team are going to know exactly how to deploy and support something you cobbled together in a completely arbitrary way.
You obviously thought through the integration with monitoring, scaling and multiple redundancy systems so in the event of traffic spike or disaster it'll keep working without manual intervention.
That's not even touching the business logic complexity of issues like DST or leap seconds where you have to be able to arbitrarily change the clock forward or backward.
Here ya go mate, caching, fallback, all in a 15 lines of javascript. If you want to extract jquery out of there that should be easy enough.
No need for extra servers, the OPs guys don't even need to know it's there. Take an extra performance hit that is equal to loading another image per page (yeah image requests, while probably served from CDN, also include an accurate Date response header as I checked), cached for 7 days (remove the cookie during periods of daylight saving times etc). Just have to make sure the server time is reasonably accurate. I'm sure a few seconds inaccuracy wouldn't matter.
Doesn't work if there's a proxy server caching old versions between the user and the server. Not to mention that according to the RFC standard intermediary servers are allowed to manipulate/remove/alter any non-restricted header (including Date).
Yes you can come up with a hacky solution that works 80% of the time quickly, but if you want a 99% solution you need to put the work in to do it right.
Actually, the Date content header is quite well defined in HTTP/1.1 [0], and should not be manipulated nilly-willy by intermediates. If you're having problems with proxies caching your requests, even if you're asking non-cached content [1], then you're never going to be able to get an accurate time from anywhere since any request might be cached, and no solution would work anyway.
It's not quite a "hacky" solution, I would guesstimate it would work 95% of the time rather than 80%, but even 80% is a great solution for the amount of work involved, the tradeoffs, and the fact that we're talking about a clock displayed on a website.
Or, you know, you could spend £15.000 taxpayers' money over 3 months for something I'm not even sure would be more flexible/maintainable/performant.
[0]: http://tools.ietf.org/html/rfc2616#section-14.18
[1]: E.g. URL's with random number in the query
Have you tested it over a daylight savings period or if the user changed their local clock?
If you want that, you can just remove the caching, and it'll always accurately reflect the server time. However who changes their local clock? I think we could definitely get away with caching it for 7 days.
Same thing with DST (I even mentioned it). Also once you have an accurate time, the javascript time object includes the DST logic itself I think. We're only dealing with an inaccurate local clock, we don't have to reimplement whatever ISO standard stands for Time. As soon as we have the delta between local time and server time, we're golden.
i agree, i've actually worked at the bbc and the infrastructure that goes into these things is unbelievable and necessary.
same here. It's absolutely not a couple of servers and a bit of node, kids!
Then what is it?
Well, I haven't worked there since about 2006, so numbers may have changed, but it was at least 10000 servers of various types (redhat, solaris, windows were the usual suspects I had to build out) of which I was aware (between maidenhead, the various colo's i had some responsibility for, tv center, white-city).
software? i know there was a lot of perl, ruby was gaining some traction, a few different CMS's.. etc, i've (thankfully) forgotten most of the details, but diverse ecosystem is an appropriate phrase here.
undoubtedly things have changed between then and now, but i'd guess it's something similar hardware-wise.
My penultimate day of work was racking up ~300 servers for the i-player farm (i was a system engineer) at red-bus in docklands. My back has only really just started to recover. :/
I just hope they're not going to get rid of caching, but instead, implement a time server and sync through JS, or something alike.
" timezone issues, daylight saving, latency, etc"
Didn't it say "time in the UK"?
We do have daylight savings so timezones do change.
uh, no. Applying DST or not does not change the time zone I am in.
In Britain, British Summer Time is considered a different timezone.