"Figure 4 shows the result of an experiment we ran to test rendering with JavaScript for better web performance. . . [The test failed and] even after we turned off the experiment, it took almost a month for pages in the enabled group to recover..."
This is the incredibly frustrating part of White Hat SEO w/Google because:
(a) At nearly every SEO conference, by Google's own actions in releasing Page Speed Insights, and by Matt Cutts engaging and talking about page speed being an important indicator it seems like speed is pretty !@$!@% important to Google and worth pouring resources into. (1) (2)
(b) As a result a well equipped and well connected organization like Pinterest launches tests designed to improve this important signal. I'm going to assume they're organizationally smart enough to not damage or ignore other important Google signal ranks like usability, time on site, etc that you have to balance w/the JS page speed test.(3)
(c) Google penalizes them.
WTF!
My frustration as a customer acquisition guy - encompassing CRO / SEM / SEO / etc - is that I try to discuss and push best practices for my own projects, for clients, and for public facing blogs / presentations / etc.
I get that they don't want people gaming / pushing - but when they push out a "best practices" methodology like page speed, and then execute a penalty as described by Pinterest, I just want to throw my hands up.
(3) I'm going to add the disclaimer of not having seen the JavaScript that pinterest used and perhaps they're not properly weighting / aware of other important SEO signals that GOOG penalizes when using Javascript, but I'm sure they are. Happy to answer more on that directly via my profile or this thread.
One of my clients has a site that gets about 1k visitors per day, heavily dependant on SEO, nothing massive, but big enough to see the effects of certain changes.
After I refactored one of their pages to go from 1.5 sec initial/8 sec total page load to 0.04 sec/4 sec load times there was no discernible change in rankings or traffic.
Page load as SEO is totally over egged for the most part as it's one of the things developers can easily measure and point a finger at, along with number of requests, CSS file size, etc. while everything actually meaningful for SEO is absolutely black box.
Every time I get handed a "SEO site report" by a client I can immediately tell if the SEO guy is any good or not if the report actually contains anything else other than recommendations to improve page loads, reduce requests and all those practically useless, but very expensive in developer time, actions. Oh, and the obligatory spreadsheet with all the external inbound links domains. Because that's really insightful.
If one improves a factor on their own site, it doesn't mean that will make a lick of difference in rankings because:
1. That may not be the factor holding a site back, and
2. It may be the site is performing as good as possible because users do not want that type of site in the SERPs.
SERP construction - what goes where - has a bigger impact on rankings than any one other factor, e.g. maps vs images versus news, and that has less to do with one's own site, and more to do with user feedback.
While I too couldn't comment on the actual JavaScript they put in place for the test and how it might have impacted other factors I just wanted to make a quick comment on the idea of a page speed penalty.
1. I have never in my life seen evidence of this happening before (enterprise consultant in this field). I am positive in my mind that it was not a case of "page is faster = drop in rankings".
I would be willing to bet that it had a lot more to do with the "rendering content in JS" which has traditionally been a huge issue for search engines (despite what they claim).
But every single time I have seen someone claim this or something like this there has been many other factors at play.
2. It is highly likely that the amount of impact that any given variable (in this case pagespeed) is not the same across the board. In fact it is much more aligned with the kinds of industries you are in and the keywords relating to that search.
As an example page speed is MUCH more likely to be a bigger factor for an e-commerce website and keywords showing any kind of transactional intent than it would be for someone looking for detailed information on a medical condition for example.*
*This isn't a confirmed fact that I know of at all but it seems to be a relatively well established theory in some more advanced SEO circles I believe.
The main issue with JS and SEO is that most JS Frameworks we are talking about are used to create SPAs (Single Page Applications). Usability-wise, these are great; everybody hates page loads. SEO-wise, there is a problem because the search engines can’t see all the HTML because everything is being rendered client side.
Basically, with a traditional web page written in PHP, the HTML is constructed from a template. The template spits out HTML from the server to the client. Every time you click a new link to load a new page, it’s delivered to your client via a GET request (EDIT: Your client/browser "asks" for the file from the server, the server sends the requested document back to the client).
With a SPA, JS is doing some DOM manipulation, and you’re not making round-trips to the server to display new content. For example, if you were to look at the source of a SPA written in Angular, you might see some <div ng-view> </div> elements, but almost no actual html. The web crawlers would see something similar.
There are several tactics for circumventing this issue, and Im curious if the Pintrest team considered them during this experiment. Anybody on the team here?
I agree with you that this definitely a rendering content in JS problem.
Most of the SEO experiments with increasing page speed from bad to average or great have shown a very small increase in rankings. We're talking one small signal in the whole algorithm. In this case, trying to go from average or good page speed to great page speed just for rankings is quite foolish.
If they want page speed to be the best it can be, and they absolutely must render the whole page in JS, they should have at least tried to use PhantomJS or something else to render the page properly for googlebot.
There's always the traditional options that can increase speed: minimize the amount of JS, increase the quality of JS written, less HTTP requests, proper DB calls, image sizing, proper caching, etc.
I work at Google in web search, and I have a few comments about this discussion.
Firstly, I think this whole discussion about page speed is the wrong way to approach it. The primary motivation for page speed should be user happiness, which affects the key metrics you care about like user acquisition, conversion, and revenue. The fact it's a (small) ranking signal is a nice benefit, a cherry on top. Here is a nice case study about page speed and user metrics from Lonely Planet:
The conversion rate graph on slide 9 is what pretty much every study that looks at performance and user engagement finds. And here is one from Google search about the effect of page speed on searchers:
Secondly, there could be other issues with how the experiment was conducted on a technical level:
1. The experiment was about using JavaScript. Was Googlebot allowed to crawl the JS files? If robots.txt blocked crawling, that would have translated to less content visible to Googlebot, and so less content to index, which can easily result in a loss of ranking.
Note that the JS file itself may have been crawlable but it may have made an API call that was blocked. Same end result in terms of indexing.
2. Related to (1), we only started rendering documents as part of our indexing process a few months ago. When was this experiment conducted? If it was before full rendering was the norm, it's very likely we didn't index JS-inserted content that we could now, which, again, may have resulted in lower ranking.
For both of these, using the Fetch and Render feature in Webmaster Tools gives you the definitive view of how our indexing system sees your content. Before running any such experiement, it's worth running a few tests using Fetch and Render.
Rude: A site that optimizes for speed to gain an SEO boost, does not deserve this SEO boost. A site like Pinterest should optimize for speed, regardless of what Google says.
If you turn back an experiment meant to enhance usability and engagement, you show that you care more for search engines than for search engine users. You are of course free to place your priorities so that your business survives, but then you can't make a claim to the woes of a whitehat SEO and you will drown in a turmoil of mystification and algo update-fears.
I do not understand why they even experiment with javascript rendering to increase the page-speed when they do not even currently defer loading of javascript to </body>. There are a lot of requests that could be combined into a single request and they could save over 50%(!) of the resource size by minifying html and lossless compression of the pinned images on pinimg.com. The mobile site gets a pagespeed score of 58 / 100. This could be 90+ / 100. It should be 90+ / 100 before you even think of javascript rendering. And if you go down that route, better study WAI-ARIA, before you make your site inaccessible to some. Good SEO follows from good design, good development, good content strategy, good UX, good accessibility. All of these things can be optimized in isolation and will benefit SEO.
If by javascript rendering they meant one of those fancy endless scrolls, then it is likely they did not implement this correctly. You can expect a lot from a bot that is able to parse javascript, but not that it spends 2 minutes scrolling down with the mouse till it reaches the end of a board and has seen all the content.
I don't think Google penalizes speed improvement. They penalize use of JavaScript for html rendering. Google bot probably cannot render all page content with JS, I'm sure in most cases it does not make all requests for static assets, it just makes GET to your url and parses html that is in response. If your initial response to first HTTP GET contains blank <body> that is filled dynamically by JavaScript Google bot will only see this blank body.
I'm pretty sure it would be completely inefficient for Google to measure page load times by making hundreds and hundreds of requsts to create full DOM representation.
For some businesses I've worked with, SEO is their lifeblood. Risking potential revenue to "test" whether certain SEO strategies will yield positive results is not possible. As always, Google picks the winners and losers.
I do this for a living and I couldn't emphasise enough how much of a bad idea it is to build a business around SEO.
Is it a really important and potentially profitable thing to get right and do well at? Without any doubt the answer to that is yes in almost all cases.
Is it also at the same time where you should spend all of your marketing time and spend? No, not unless you want to spend the rest of your days worrying that the next major update will potentially wipe you out.
I find the advice of "don't rely too much on one channel" to be a bit insincere. Yes, that's good advice, but avoiding that trap, for many businesses, can be very hard. And when you're constantly trying to grow, increasing reliance on channels that provide growth is either difficult or stupid to avoid.
For channels where you don't have strong control, which are many -- SEO, social, etc. -- it's best to get as much overlap in channels you do control. So do things like convert organic and social traffic to emails as efficiently as possible. It's still going to hurt when you take hits, but you should be in a better position to rebound.
Also, when Genius got their spanking from Google last year, that advice was thrown around a lot. "Why are you so reliant on organic search?" For Genius, it makes 100% sense to be reliant on organic search. I search for lyrics a lot but almost never go to Genius directly. Why? It's not because they're bad, it's because it's easier to type in the Google bar "$song_title lyrics". And most of the time, Genius is right up there and that's who I click on because, in my opinion, they offer the best lyrics experience. And for people not familiar with Genius, they will always be most likely to become familiar with them from search results for "$song_title lyrics". It's a perfect fit.
and perhaps they're not properly weighting / aware of other important SEO signals that GOOG penalizes when using Javascript, but I'm sure they are. Happy to answer more on that directly via my profile or this thread.
I for one would very much like to see this discussed in this thread.
I've avoided SEO all my life (not determinate enough for my brain to handle) but I have one client (car hire) for whom SEO is their lifeblood, and I need to get over my discomfort and understand more.
As a result a well equipped and well connected organization like Pinterest launches tests designed to improve this important signal.
One may think so, but I have worked at an organization where SEO was well known about but due to the lack of attention their recommendations were given by engineering teams, many of them left. The site subsequently suffered some penalties.
Google can not see JS-rendered stuff. They only do simple HTTP-requests. Googlebot doesn't even load the .js files ... Try searching your http log for \.js.*Googlebot
Comments
"Figure 4 shows the result of an experiment we ran to test rendering with JavaScript for better web performance. . . [The test failed and] even after we turned off the experiment, it took almost a month for pages in the enabled group to recover..."
This is the incredibly frustrating part of White Hat SEO w/Google because:
(a) At nearly every SEO conference, by Google's own actions in releasing Page Speed Insights, and by Matt Cutts engaging and talking about page speed being an important indicator it seems like speed is pretty !@$!@% important to Google and worth pouring resources into. (1) (2)
(b) As a result a well equipped and well connected organization like Pinterest launches tests designed to improve this important signal. I'm going to assume they're organizationally smart enough to not damage or ignore other important Google signal ranks like usability, time on site, etc that you have to balance w/the JS page speed test.(3)
(c) Google penalizes them.
WTF!
My frustration as a customer acquisition guy - encompassing CRO / SEM / SEO / etc - is that I try to discuss and push best practices for my own projects, for clients, and for public facing blogs / presentations / etc.
I get that they don't want people gaming / pushing - but when they push out a "best practices" methodology like page speed, and then execute a penalty as described by Pinterest, I just want to throw my hands up.
(1) https://developers.google.com/speed/pagespeed/insights/
(2) http://www.webpronews.com/today-on-the-matt-cutts-show-page-...
(3) I'm going to add the disclaimer of not having seen the JavaScript that pinterest used and perhaps they're not properly weighting / aware of other important SEO signals that GOOG penalizes when using Javascript, but I'm sure they are. Happy to answer more on that directly via my profile or this thread.
One of my clients has a site that gets about 1k visitors per day, heavily dependant on SEO, nothing massive, but big enough to see the effects of certain changes.
After I refactored one of their pages to go from 1.5 sec initial/8 sec total page load to 0.04 sec/4 sec load times there was no discernible change in rankings or traffic.
Page load as SEO is totally over egged for the most part as it's one of the things developers can easily measure and point a finger at, along with number of requests, CSS file size, etc. while everything actually meaningful for SEO is absolutely black box.
Every time I get handed a "SEO site report" by a client I can immediately tell if the SEO guy is any good or not if the report actually contains anything else other than recommendations to improve page loads, reduce requests and all those practically useless, but very expensive in developer time, actions. Oh, and the obligatory spreadsheet with all the external inbound links domains. Because that's really insightful.
Everyone - repeat after me:
SEO DOES NOT HAPPEN IN ISOLATION!
I repeat:
SEO DOES NOT HAPPEN IN ISOLATION!
If one improves a factor on their own site, it doesn't mean that will make a lick of difference in rankings because:
1. That may not be the factor holding a site back, and
2. It may be the site is performing as good as possible because users do not want that type of site in the SERPs.
SERP construction - what goes where - has a bigger impact on rankings than any one other factor, e.g. maps vs images versus news, and that has less to do with one's own site, and more to do with user feedback.
While I too couldn't comment on the actual JavaScript they put in place for the test and how it might have impacted other factors I just wanted to make a quick comment on the idea of a page speed penalty.
1. I have never in my life seen evidence of this happening before (enterprise consultant in this field). I am positive in my mind that it was not a case of "page is faster = drop in rankings".
I would be willing to bet that it had a lot more to do with the "rendering content in JS" which has traditionally been a huge issue for search engines (despite what they claim).
But every single time I have seen someone claim this or something like this there has been many other factors at play.
2. It is highly likely that the amount of impact that any given variable (in this case pagespeed) is not the same across the board. In fact it is much more aligned with the kinds of industries you are in and the keywords relating to that search.
As an example page speed is MUCH more likely to be a bigger factor for an e-commerce website and keywords showing any kind of transactional intent than it would be for someone looking for detailed information on a medical condition for example.*
*This isn't a confirmed fact that I know of at all but it seems to be a relatively well established theory in some more advanced SEO circles I believe.
The main issue with JS and SEO is that most JS Frameworks we are talking about are used to create SPAs (Single Page Applications). Usability-wise, these are great; everybody hates page loads. SEO-wise, there is a problem because the search engines can’t see all the HTML because everything is being rendered client side.
Basically, with a traditional web page written in PHP, the HTML is constructed from a template. The template spits out HTML from the server to the client. Every time you click a new link to load a new page, it’s delivered to your client via a GET request (EDIT: Your client/browser "asks" for the file from the server, the server sends the requested document back to the client).
With a SPA, JS is doing some DOM manipulation, and you’re not making round-trips to the server to display new content. For example, if you were to look at the source of a SPA written in Angular, you might see some <div ng-view> </div> elements, but almost no actual html. The web crawlers would see something similar.
There are several tactics for circumventing this issue, and Im curious if the Pintrest team considered them during this experiment. Anybody on the team here?
Angular.js, courtesy of Google.
I agree with you that this definitely a rendering content in JS problem.
Most of the SEO experiments with increasing page speed from bad to average or great have shown a very small increase in rankings. We're talking one small signal in the whole algorithm. In this case, trying to go from average or good page speed to great page speed just for rankings is quite foolish.
If they want page speed to be the best it can be, and they absolutely must render the whole page in JS, they should have at least tried to use PhantomJS or something else to render the page properly for googlebot.
There's always the traditional options that can increase speed: minimize the amount of JS, increase the quality of JS written, less HTTP requests, proper DB calls, image sizing, proper caching, etc.
I work at Google in web search, and I have a few comments about this discussion.
Firstly, I think this whole discussion about page speed is the wrong way to approach it. The primary motivation for page speed should be user happiness, which affects the key metrics you care about like user acquisition, conversion, and revenue. The fact it's a (small) ranking signal is a nice benefit, a cherry on top. Here is a nice case study about page speed and user metrics from Lonely Planet:
http://cdn.oreillystatic.com/en/assets/1/event/88/Performanc...
The conversion rate graph on slide 9 is what pretty much every study that looks at performance and user engagement finds. And here is one from Google search about the effect of page speed on searchers:
http://googleresearch.blogspot.co.uk/2009/06/speed-matters.h...
Secondly, there could be other issues with how the experiment was conducted on a technical level:
1. The experiment was about using JavaScript. Was Googlebot allowed to crawl the JS files? If robots.txt blocked crawling, that would have translated to less content visible to Googlebot, and so less content to index, which can easily result in a loss of ranking.
Note that the JS file itself may have been crawlable but it may have made an API call that was blocked. Same end result in terms of indexing.
2. Related to (1), we only started rendering documents as part of our indexing process a few months ago. When was this experiment conducted? If it was before full rendering was the norm, it's very likely we didn't index JS-inserted content that we could now, which, again, may have resulted in lower ranking.
For both of these, using the Fetch and Render feature in Webmaster Tools gives you the definitive view of how our indexing system sees your content. Before running any such experiement, it's worth running a few tests using Fetch and Render.
Rude: A site that optimizes for speed to gain an SEO boost, does not deserve this SEO boost. A site like Pinterest should optimize for speed, regardless of what Google says.
If you turn back an experiment meant to enhance usability and engagement, you show that you care more for search engines than for search engine users. You are of course free to place your priorities so that your business survives, but then you can't make a claim to the woes of a whitehat SEO and you will drown in a turmoil of mystification and algo update-fears.
I do not understand why they even experiment with javascript rendering to increase the page-speed when they do not even currently defer loading of javascript to </body>. There are a lot of requests that could be combined into a single request and they could save over 50%(!) of the resource size by minifying html and lossless compression of the pinned images on pinimg.com. The mobile site gets a pagespeed score of 58 / 100. This could be 90+ / 100. It should be 90+ / 100 before you even think of javascript rendering. And if you go down that route, better study WAI-ARIA, before you make your site inaccessible to some. Good SEO follows from good design, good development, good content strategy, good UX, good accessibility. All of these things can be optimized in isolation and will benefit SEO.
If by javascript rendering they meant one of those fancy endless scrolls, then it is likely they did not implement this correctly. You can expect a lot from a bot that is able to parse javascript, but not that it spends 2 minutes scrolling down with the mouse till it reaches the end of a board and has seen all the content.
I don't think Google penalizes speed improvement. They penalize use of JavaScript for html rendering. Google bot probably cannot render all page content with JS, I'm sure in most cases it does not make all requests for static assets, it just makes GET to your url and parses html that is in response. If your initial response to first HTTP GET contains blank <body> that is filled dynamically by JavaScript Google bot will only see this blank body.
I'm pretty sure it would be completely inefficient for Google to measure page load times by making hundreds and hundreds of requsts to create full DOM representation.
For some businesses I've worked with, SEO is their lifeblood. Risking potential revenue to "test" whether certain SEO strategies will yield positive results is not possible. As always, Google picks the winners and losers.
I do this for a living and I couldn't emphasise enough how much of a bad idea it is to build a business around SEO.
Is it a really important and potentially profitable thing to get right and do well at? Without any doubt the answer to that is yes in almost all cases.
Is it also at the same time where you should spend all of your marketing time and spend? No, not unless you want to spend the rest of your days worrying that the next major update will potentially wipe you out.
I find the advice of "don't rely too much on one channel" to be a bit insincere. Yes, that's good advice, but avoiding that trap, for many businesses, can be very hard. And when you're constantly trying to grow, increasing reliance on channels that provide growth is either difficult or stupid to avoid.
For channels where you don't have strong control, which are many -- SEO, social, etc. -- it's best to get as much overlap in channels you do control. So do things like convert organic and social traffic to emails as efficiently as possible. It's still going to hurt when you take hits, but you should be in a better position to rebound.
Also, when Genius got their spanking from Google last year, that advice was thrown around a lot. "Why are you so reliant on organic search?" For Genius, it makes 100% sense to be reliant on organic search. I search for lyrics a lot but almost never go to Genius directly. Why? It's not because they're bad, it's because it's easier to type in the Google bar "$song_title lyrics". And most of the time, Genius is right up there and that's who I click on because, in my opinion, they offer the best lyrics experience. And for people not familiar with Genius, they will always be most likely to become familiar with them from search results for "$song_title lyrics". It's a perfect fit.
I for one would very much like to see this discussed in this thread.
I've avoided SEO all my life (not determinate enough for my brain to handle) but I have one client (car hire) for whom SEO is their lifeblood, and I need to get over my discomfort and understand more.
One may think so, but I have worked at an organization where SEO was well known about but due to the lack of attention their recommendations were given by engineering teams, many of them left. The site subsequently suffered some penalties.
Solid fundamentals will help everything long term, but short term dealing with the BS is super frustrating.
Google can not see JS-rendered stuff. They only do simple HTTP-requests. Googlebot doesn't even load the .js files ... Try searching your http log for \.js.*Googlebot