Skip to content

Comment on Stack Exchange Engineering: How We Built Our Blog

Comments

They lost a simple interface designed to edit blog posts and now their marketing team has to use the Github interface and learn Markdown.

They lost the comment system and had to move the data to an external provider, which instantly bit them.

They had to write Python scripts to munge the data.

But now they have the performance of static html. I guess just changing two lines in the nginx configuration did not came to mind.

I think this was a very poor investment and will never pay off.

They've also lost COMPLEXITY, and this always pays off, even if in roundabout ways that you cannot measure, and even if it's only for the positive effects on the mental-health of the engineers working on this!

Really... modern WordPress is a very complicated beast and keeping all sorts of details about how you need to optimize it, server, caching, db in you head is just no worth it. Yeah, "changing two lines in the nginx configuration" could have worked, but imho simply the cost of keeping in your head the information about what two lines to change is just too large. As it grows, it simply drives you insane by how many technical trivialities you need to keep in your head to keep it working smoothly, when you could instead use that brainpower so solve the really interesting problems.

An the fact that "their marketing team has to use the Github interface and learn Markdown" is also a hidden benefit, if you think about all the subtle frontend bugs that happen when somebody uses the WYSIWYG editor of WP and they end up with a combination of styles that breaks the layout. For any project above a certain size, the "dumb WIM" (What I Mean) interface of Markdown always trumps the WYSIWYG.

Also, the benefit of mostly NOT HAVING TO THINK ABOUT SECURITY AT ALL is freaking HUGE! Developers can simply say "it's the server guys' problem" now. Forget about having to manually audit the code of 3rd party plugins if you really care about security. And forget about updating and realizing something breaks and you have to fix it.

For any fast growing company, keep this in mind: REDUCE COMPLEXITY. ALWAYS. NO MATTER HOW LARGE THE SHORT TERM COST.

Just as an aside to WordPress optimization, so there are two ways to do that. One of them is to analyze content flow (how often are things updated) and traffic flow and then come up with a reasonable strategy. Things like use memcaching for sidebars, separate those out depending whether you need to, then work on caching plugins. The caching plugins need work because they tend to be programmed to allow robots to spider uncached content, which makes no sense to me. Robot traffic on high volume sites is just insane. With a hundred blogs or so on a multisite installation you could easily spend a month doing postmortems and hunting down the least common denominator (oh hello switch_to_blog, why do you use 36 MB of RAM on each call?)

Or, you can slap some ram sticks in the machine, install varnish-cache, write a few rules for it and go play minecraft the other 29 days.

As it grows, it simply drives you insane by how many technical trivialities you need to keep in your head to keep it working smoothly

For everything else there's Ansible.

We've taken the approach to not type a single command into a live server's console and use CM tools to document and automate. This, along with well written operating documentation can be invaluable.

Hi! I'm the Director of Product Marketing at Stack Exchange. It might sound like more work for us to use Github and markdown, but I see it as a huge plus. For non technical employees, this is a great opportunity to learn a new skill and collaborate with the engineering organization. Jon trained a few of us already... so far, so good!

Thanks for writing this...too many of us engineers treat non-technical employees as morons/infants, as if you didn't spend 12+ years of your life learning math and English and aren't able to cope with new text/symbolic systems...Learning Markdown is as easy/difficult as learning a rich-text editor...why do people assume that pressing a button with a cute icon on it is inherently easier than a Markdown convention? At least by learning the few Markdown conventions there are, you get some hands on experience with understanding how literal computer languages have to be...rather than forever treating rich text editors as a magical machine.

God, the hours I've seen co-workers waste, trying to eliminate some errant space in a rich-text-editor-generated hyperlink, by pressing a random sequence of rich-text buttons. A lot of times, they just go straight to the raw HTML to fix things...and if you're already doing that, you're capable of learning Markdown (let's skip the discussion of why editing raw HTML can be even more disastrous).

Kudos to you for being eager to learn new skills...I'm an engineer and a self-taught web dev I wish someone had forced me to learn Markdown much earlier than I had.

Hi! While the change surely was not a huge investment I just do not believe that the benefits justify the costs. Since you wrote a blog post about it, I felt free to question your decisions.

Nevertheless, now that it is done, I hope the new system works well for you. Maybe your team gets used to git for version control in other areas of work, too, which would be a huge win and ultimately prove me wrong ;-)

I hope it works well too! Maybe I'll write a follow-up blog post in a few months and report back on our progress. In general, I think that marketers today should acquire some basic front-end knowledge and that knowing git and markdown is a good step towards getting there. Coding things like email templates and landing pages without the help of a designer or developer will be huge for us.

I'm the author of the post and the lead on this project. Many of the costs you mention aren't so bad. As mentioned earlier, we actually consider learning GH and Markdown a plus for non-devs. As for comments and writing Python, some pain here, but were ultimately solvable.

Performance also wasn't the only plus here. Closing major security holes, making more of our content and technology more open, and moving to a platform that our devs liked working in are just some of the other wins.

It's too early to say definitively now, but we think the change is probably a good one.

You haven't broken the podcast RSS feed, thank you. Another large podcast network recently did a redesign and broke all of their feeds, apparently on purpose. Crazy.

However, the podcast feed is impossible to find now if you didn't have it before. Previously I think it was on every podcast post as an RSS link. Now the individual tag RSS links all point to the main feed, rather than to a per-tag feed.

e.g. http://blog.stackoverflow.com/category/podcasts/feed/ still works if you know it, but going to https://blog.stackexchange.com/tags/podcasts/ gives you an RSS link to just "/feed/".

This is probably not a good place for bug reports :-S

Did anyone evaluate ghost? If so, I'm curious what the verdict was.

https://ghost.org/

We did consider Ghost, I love it and it would've given us uploading images to S3 for free, but static files and being able to use any markdown editor you want won over.

Last year, I took two days to build a mostly static blog to replace WordPress. It pays off everytime there is a WordPress vulnerability I don't have to worry about, and everytime we post an interesting post and I don't have to worry about if WordPress is going to fall over (yes, maybe there's some caching thing we could have used). It pays off when I have multiple data centers running, because WordPress doesn't have the ability to read from a local MySQL slave, so perf on the datacenter far from the master is terrible (the i18n plugin we were using didn't help).

I imagine the payoff is similar for the stack overflow team as well.

EDIT: I don't understand why they are using disqus for comments, when they have a self hosted comment platform, in my case we don't accept comments.

Simple caching would have solved all of your issues. Classic case of not profiling before blaming the whole system.

Simple caching wouldn't have solved the (numerous) security issues.

You could still export/dump the WordPress pages as static html and still keep the editor. (and WordPress not accessible from the outside).

Stack Exchange uses Markdown for everything on their Q&A sites already. If they expect their users to be able to use it, I think they can expect the same from their employees. It's likely not a wasted effort to learn it as they might need it somewhere else as well.

The Stack Exchange blog was hacked a few years ago, I suspect that was part of the motivation to switch to something they know better.

My employer originally had a Wordpress-based blogging product.

They spent more than $2M writing their own blogging system, and it still was missing many features that come with Wordpress.

They ditched it and switched back to Wordpress.

TWO MILLION DOLLARS on a blogging system.

I'm speechless.

It might surprise you even more to know that the creators of the most popular blogging system have raise $317M.

10 silicon Valley software engineers on something goes through 2 million in under a year when you consider total cost of employment.

If it takes 10 engineers a year to build a blog then something is very wrong.

I would like to see a study where one would give the same project with a solid deadline to three different teams of 3, 15, 30 engineers and see how long it takes each them to finish it.

The marketing team can use http://prose.io without learning Markdown.

Using using an external comment provider promotes some popular users to comment such as Grimlock whom otherwise wouldn't.

Someone else also mentioned prose.io - something I didn't come across when building this. Going to explore.

Asana did something similar: https://eng.asana.com/2014/02/scaling-asana-com/

There is an argument to be made that a ubiquitous, developer-first system translates into more instant changes. Seeing changes quickly makes it easier to iterate, which is better over time than the initial investment to learn Markdown.

i dislike wordpress quite a bit, but i also agree that this was a head-scratching investment.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.