When I do stuff like this I purposefully ignore the results for the time I've set for the experiment. It's very easy to fall prey to thinking you have a result that will not change in the longer term. Things like daily and weekly cycles for instance can really throw off your analysis.
The only danger is having a 'hidden variable' influence your results and averaging over the longer term masks that influence. For example, if you are not geo-targeting your content, you could conclude after a long run of testing that a certain page performs better than another, only to throw away the averaged out effect of having the different pages up on different times of the day, one of them performing significantly better for one audience and vv.
So you should keep all your data in order to figure out if such masking is happening and giving you results that are good but that could be even better.
Comments
When I do stuff like this I purposefully ignore the results for the time I've set for the experiment. It's very easy to fall prey to thinking you have a result that will not change in the longer term. Things like daily and weekly cycles for instance can really throw off your analysis.
The only danger is having a 'hidden variable' influence your results and averaging over the longer term masks that influence. For example, if you are not geo-targeting your content, you could conclude after a long run of testing that a certain page performs better than another, only to throw away the averaged out effect of having the different pages up on different times of the day, one of them performing significantly better for one audience and vv.
So you should keep all your data in order to figure out if such masking is happening and giving you results that are good but that could be even better.