This ignores different realities in how programmers and writers work.
If you're hiring a writer, you cam ask to see his previous work. Unless he worked for CIA, he can point to books he's written, articles he published etc. You can then read them and that's all you need to make a decision if he's competent enough (you might still interview for cultural fit).
If you're hiring a senior engineer that claims he spent last 5 years coding Big Table at Google, you have to take his word for it. It's not that you wouldn't like to see all his past checkins, you just can't in most cases.
Not being able to see past work, we do the next best thing: interviews, coding questions etc.
The simple explanation is: we don't hire programmers like writers because we can't, not because people hiring programmers are inexplicably so much more incompetent than people hiring writers (despite running much more profitable businesses).
This is changing a bit due to open-source in that more and more people are hired like writers i.e. based on their publicly available past work, but it'll never be the case that every programmer will be able to show his past code.
If you're hiring a senior engineer that claims he spent last 5 years coding Big Table at Google, you have to take his word for it.
I find this assertion to be virtually nonsensical. I've never met a programmer who hasn't written some code that they could show you, if for no other reason than programmers who are passionate about what they do, usually write at least little programs just for fun. And if I were interviewing a programmer, I'd think twice about hiring one who has so little passion for programming that they'd never written anything just for the pleasure of it.
But let's assume for the moment that there are tons of great programmers who've never taken a class for which they could show their code, nor written any code just to write it: Where I work we have a small programming exercise that we ask applicants to submit for code review before we interview them. This process is much more natural than having people code on a whiteboard, because they can do it at their own rate and with their own tools, without having the stress of having someone look over their shoulder. They can also pay proper attention to issues of style and maintainability of their code. The assignments are also something kind of like the work that one might do in the real world.
I write lots of code just for the sheer pleasure of it. None of it is publicly available though nor would I like to share it. Shipping code is a lot of work. The other day I wrote a quick app at my end to analyze and make sense of some parts of a project that I am about to take on. It does what it needs to but if I had to ship it, I would want to proof read it and describe it's purpose to a general audience and make it at least general enough to have some utility outside my immediate needs. That's a lot of work. I write a lot of code for fun and profit. Writing code and publishing/sharing code are two very different things.
Okay, well Google doesn't want to hire you if you can't write efficient and elegant code on a whiteboard for a problem they've specified while they look over your shoulder. Me, I only don't want to hire you if you can't take a few pages of code you've written for fun and make it presentable. Who would you rather interview with? (Assuming, of course, that I was offering jobs as nice as the jobs at Google.)
I might be an exception here but I much prefer the whiteboard. I have one in my room and I enjoy problem solving. Pressure environments really do not impact me much. And if you have a whiteboard in your office and you do use it for actual work and not just interviews, this gives me an excellent opportunity to check your quality as a future problem solving accomplice. It provides me a real if brief window into what working with you would be like and I like that.
Having said that, market realities are what they are and I will probably take a month off when I can afford it and put some substantial code out in the wild. Can't hurt to have both bases covered. I will still like to make the point in here that I will be doing this because of the perceived business climate not because I really want to.
"And if you have a whiteboard in your office and you do use it for actual work and not just interviews, this gives me an excellent opportunity to check your quality as a future problem solving accomplice."
Except that when preparing for an interview, most places will clean the whiteboards before you arrive to make things more presentable, and to avoid accidentally divulging company secrets.
I write code every day, but I don't save it and its been 20 years since I wrote any code for 'pleasure'. For pleasure I have sex, go to a symphony or swim in the ocean. Coding is something I do because I'm good at it and people will pay me for it.
Further, I don't save code. I sell it. If I saved it then I would have to organize it, keep track of it and back it up. Why do that? I'm a programmer. Code I have written for fun or for some small experimental reason has no value to me because I can do it again and again at any time, like breathing. Sure, non-programmers value my code and they save it for posterity but I don't. It has no value to me once I've been paid for it.
Why would I go to all the trouble to keep a complete record of something I could do again at any time? Am I supposed to save what I'm typing right now to prove I can communicate? Do I save lawn clippings to prove I can operate a lawn mower? Do I save my waste to prove I can defecate?
You save code + accompanying documents in order to build up a portfolio to demonstrate to a potential employer how you work, how well you know your tools, and what your idea of a shippable product is. During the interview, you can talk about why you wrote the software, what designs you tried, what kinds of challenges you faced while doing so, and how you surmounted them.
At my company, we offer coding challenges for people who don't have a portfolio already. But really, why not just write software of your own choosing so that you don't need to do coding challenges to prove your ability? It's a far more efficient approach.
> If I saved it then I would have to organize it, keep track of it and back it up.
I really find this hard to swallow. It fails for me on so many counts.
Firstly, code is small compared to just about anything else you might keep on your computer. All you have to do to preserve it is drop it in a directory you keep for that purpose. The code doesn't have to be organized in order to preserve it. If you back up your computer, then your code is backed up. You back up your computer don't you?
Sure, it may be hard to find some useful piece of code later if you don't organize it, but preserving it, at least, is easy.
More importantly, you never write reusable code? You never write a library that can be applied to future problems? If not, then, you may not be the type of programmer that I'd want to work with anyway. I firmly believe that the way to increased productivity for any team is to factor out useful components and make them into more general purpose libraries. To do otherwise, is to end up with a codebase that is bloated with boilerplate.
Thirdly, if coding for you is as natural as breathing, surely you can whip up a couple of pages of code as a "code portfolio" in no time. You could have done that instead of typing in the above for your pleasure.
Not against you personally, but I'm fascinated by how clear people are about aspects of candidates they don't want to hire. There are so many dealbreakers in this thread (in the main), it's like a dating site full of people who have a lot of difficulty finding partners (hint: all dating sites are full of people with dealbreakers).
"I find this assertion to be virtually nonsensical. I've never met a programmer who hasn't written some code that they could show you"...
I've met plenty. Interviewed 6 candidates a few years ago - only one even brought anything beyond his resume. He brought a full physical portfolio of work he'd done. Most of it was web screenshots and such - not much 'code' to speak of, but he'd made the effort. No one else did.
I've been in plenty of places where people say "I can't show my work, cause it's for my previous employer". And they don't code when they go home. Or on weekends.
I brought code samples to an interview once, and the interviewer wouldn't look at it because "it wouldn't be fair to other people who don't bring code samples".
No doubt many people can, but you may be living in a bit of a self-selection bubble where everyone you know and mix with has loads of spare code to demonstrate. Many people don't. Now... if you want to use that as a signal for that person's interest level or ability, feel free to. You may be missing out on some otherwise competent or talented folks. But we all have to use signals to filter out candidates, and that may be a valid signal for you to use.
If you didn't ask candidates to bring code samples, it should come as no surprise that most don't, even though they might be able.
Where I work, we don't ask candidates to bring in a random code sample because instead we have them submit a coding exercise that they can do at home at their leisure. Our entire team then code reviews the code. Given the quality of most of the code submissions, it may be true that most programmers never program for fun, but if so, it shows this fact shows in the quality of their work.
All of the people who've given us nice submissions have also been very enthusiastic about what they do. I know this because we ask candidates if they read any programming journals or books just for their own edification, and the good ones always do. They always show some motivation beyond just completing assigned tasks.
We're a Scala shop, though, so we need people who aren't just programming because it's their job and for no other reason. Such people would have no motivation to become skilled in a complex programming language that is not yet mainstream.
I was doing PHP in 1996. If you were doing PHP in 1996, it was generally because you wanted to do it, learn it, etc (arguably web stuff in general then a bit too). There weren't many "PHP jobs" as such back then, so when you found other PHP developers, you knew they were enthusiasts of some sort.
That's where Scala's been, and almost any 'newish' language. If you find people adopting them, they're probably enthusiastic about it, and enjoy living on the cutting edge. :)
A senior engineer that has spent 5 years coding Big Table at Google would certainly (if genuinely interested in being hired) be able to take a day and throw up some code on github.
Not Big Table code of course, just some interesting code examples tailored to the position being negotiated.
Worried about people passing off copy/pasted code on github? That's what the interview is for. It's great to be able to discuss code that someone has written after you've taken some time to review it. They should be able to have a discussion about the code, their design choices, what they would do differently given a twist in the requirements, etc. If they can't do that: then even if they genuinely wrote the code they don't understand it enough.
I feel I should stress that the code doesn't have to be the next big open source project forked/watched by thousands.
Not comfortable putting code up on github because people might see it? I get that, it is scary to put your code out in public. In that case, a simple Dropbox (or whatever) shared folder would suffice.
In addition I would tend to ask a few important questions of developers:
1) Tell me about one case where you came back to code you had previously written and decided you didn't like the style. What was the issue? Did you rewrite it to fix it? Why or why not?
2) Tell me about one case where you departed from general design best practices where it worked. What did you do? Why did it work well?
3) Tell me about your commenting style. Why and how do you comment your code?
Oh and asking to see code samples is a good thing :-)
These sound like the sort of thing an HR person would ask, not a developer. Commenting style. Seriously? What in all honesty do you expect someone to say, you must know you're going to get spoon fed a perfect answer just for asking such a dumb question.
Why do you say that? As a developer the comments are talking to me, not to the HR department and not to the computer. I want to know someone has thought about this area regarding the way bad comments get in the way of collaboration and the way good comments facilitate it.
The point is that answers to these questions show the extent of introspection a programmer has regarding code. I am looking for thoughtful answers, not right answers. If you think they are stupid questions, I don't want to put you in charge of doing the basic creation of new frameworks.
Edit: One really good follow-up question is something like, 'So you are reading code and you see a comment that says "This is broken." Is that a good comment or a bad one? Why?' It's a good question because right or wrong isn't in "good" or "bad" but rather the reasons they might give.
Edit2: If it's such an easy question what's your answer? ;-)
It's a trivial question, comments are a chore that developers do because they provide someone who is later looking at and/or modifying their code with the knowledge about how it works in a way that isn't obvious from the code itself. Comments have morphed into more structured form with annotations that the IDE can make use of in some languages, but asking about comments in an interview is asking to get spoon fed what you want to hear.
If you are explaining how your code works, shouldn't you be rewriting it so it is obvious rather than adding comments?
I don't know about you but I go after people in open source projects I help manage who write comments like that. Such comments add no value collaboration and indeed they get in the way of debugging more than they help. Folks debugging shouldn't be debugging based on comments but rather based on code.
But with clear code, comments are still helpful if they are used to describe difficult design decisions, or flag issues the developer sees down the road. "This is a kludge" is a much more helpful comment than "looping through an array of invoice lines so we know what to print." The first draws the developer's attention to the fact that an area of code may be unusually troublesome, while the second may be out of date, may be wrong, or may otherwise distract from debugging.
Of course there are function header comments, etc, but those don't really need to be under discussion.
You are right about the code needing to talk to you, but if you see comments as a way of explaining what your code does, then it's both a distraction to the next guy and a crutch for you.
Edit: I think this shows the point of the question and why the trivial spoon-fed answers are likely to be ones I would find inadequate.
Edit2: So the way I read your answer is "programmer just wants to get things done and doesn't mind writing non-clear code to do it, explaining it with comments. Suitable for a maintenance programming role but may in time grow into others."
Here's a good example btw of a comment that I found very helpful.
One day a number of years ago, I was reading through some part of the Linux source code (somewhere in the 2.4 series I think), and I was looking at a section of the TCP stack and came across a comment to the effect of "This is the max timeout value as specified in the RFC. This means FTP to Mars will never work"
Perfect comment. Besides making me laugh, it explains why a specific line is the way it is and explains that this may cause some issues where latency in the connection is high enough.
More importantly, I think it ignores the reality in how the output is consumed. Writing is generally meant to be read; that is its use. When evaluating a text, the qualities it has when read by a human are the primary criteria.
When we evaluate code by reading it, our largest concern is how it will be interpreted by a machine. Clarity matters in comments, but can be justifiably sacrificed in code for an elegant or efficient solution.
It might be more appropriate to compare the requirements of composing legal documents to that of coding. Testing for understanding of methods, patterns and common concepts is far more applicable in that case. Asking if a candidate knows the parameters for a work to be classified as work-for-hire or what kinds of indemnifications and disclaimers should be in a contract are arguable useful indicators of suitability.
Where is the fine line between coding tests and spec work?
I'm not a programmer, but I've seen startups co-founded by engineers try hiring designers in a similar fashion to I assume programmers as they test designers with a variety of unpaid projects. I've found it to be pretty annoying given that I've already shown a portfolio of work and it's not unlike spec work. I get that some people may have fabricated their history, but that's why some companies do a trial period before committing to a full-time gig.
If I ask someone to implement a linked list during 1hr interview, it's clear that it's not because I need a linked list implementation in my company.
If I ask someone to stay for a week to work on the company's code base, that's clearly a work that should be paid for at a market rate.
Similarly, I'm pretty sure that you, as a designer, know without a doubt when company's request crosses the line from "I need to make sure you know what you're doing" to "I need you to do free work for me".
I don't like all their advice, but one thing that 37Signals does in this situation is just pay the person whatever rate they ask for. In the greater scheme of things, it's not a huge expense, and if you're to the point in the interview process where this actually makes sense, then you have a decent idea that this person is a candidate, and so you can invest a little in the process for everybody's benefit. It's a step beyond the whiteboarding "jump, monkey!" coding interview.
I was once asked a fairly involved technical question in an hour long interview - I later interviewed for a different group in the same organization and they mentioned the problem the other group had been having.
I talked about the solution I had proposed and was told - "oh yes, some consultant they had in suggested that and we tried it last week".
Comments
This ignores different realities in how programmers and writers work.
If you're hiring a writer, you cam ask to see his previous work. Unless he worked for CIA, he can point to books he's written, articles he published etc. You can then read them and that's all you need to make a decision if he's competent enough (you might still interview for cultural fit).
If you're hiring a senior engineer that claims he spent last 5 years coding Big Table at Google, you have to take his word for it. It's not that you wouldn't like to see all his past checkins, you just can't in most cases.
Not being able to see past work, we do the next best thing: interviews, coding questions etc.
The simple explanation is: we don't hire programmers like writers because we can't, not because people hiring programmers are inexplicably so much more incompetent than people hiring writers (despite running much more profitable businesses).
This is changing a bit due to open-source in that more and more people are hired like writers i.e. based on their publicly available past work, but it'll never be the case that every programmer will be able to show his past code.
If you're hiring a senior engineer that claims he spent last 5 years coding Big Table at Google, you have to take his word for it.
I find this assertion to be virtually nonsensical. I've never met a programmer who hasn't written some code that they could show you, if for no other reason than programmers who are passionate about what they do, usually write at least little programs just for fun. And if I were interviewing a programmer, I'd think twice about hiring one who has so little passion for programming that they'd never written anything just for the pleasure of it.
But let's assume for the moment that there are tons of great programmers who've never taken a class for which they could show their code, nor written any code just to write it: Where I work we have a small programming exercise that we ask applicants to submit for code review before we interview them. This process is much more natural than having people code on a whiteboard, because they can do it at their own rate and with their own tools, without having the stress of having someone look over their shoulder. They can also pay proper attention to issues of style and maintainability of their code. The assignments are also something kind of like the work that one might do in the real world.
I write lots of code just for the sheer pleasure of it. None of it is publicly available though nor would I like to share it. Shipping code is a lot of work. The other day I wrote a quick app at my end to analyze and make sense of some parts of a project that I am about to take on. It does what it needs to but if I had to ship it, I would want to proof read it and describe it's purpose to a general audience and make it at least general enough to have some utility outside my immediate needs. That's a lot of work. I write a lot of code for fun and profit. Writing code and publishing/sharing code are two very different things.
Okay, well Google doesn't want to hire you if you can't write efficient and elegant code on a whiteboard for a problem they've specified while they look over your shoulder. Me, I only don't want to hire you if you can't take a few pages of code you've written for fun and make it presentable. Who would you rather interview with? (Assuming, of course, that I was offering jobs as nice as the jobs at Google.)
I might be an exception here but I much prefer the whiteboard. I have one in my room and I enjoy problem solving. Pressure environments really do not impact me much. And if you have a whiteboard in your office and you do use it for actual work and not just interviews, this gives me an excellent opportunity to check your quality as a future problem solving accomplice. It provides me a real if brief window into what working with you would be like and I like that.
Having said that, market realities are what they are and I will probably take a month off when I can afford it and put some substantial code out in the wild. Can't hurt to have both bases covered. I will still like to make the point in here that I will be doing this because of the perceived business climate not because I really want to.
"And if you have a whiteboard in your office and you do use it for actual work and not just interviews, this gives me an excellent opportunity to check your quality as a future problem solving accomplice."
Except that when preparing for an interview, most places will clean the whiteboards before you arrive to make things more presentable, and to avoid accidentally divulging company secrets.
I write code every day, but I don't save it and its been 20 years since I wrote any code for 'pleasure'. For pleasure I have sex, go to a symphony or swim in the ocean. Coding is something I do because I'm good at it and people will pay me for it.
Further, I don't save code. I sell it. If I saved it then I would have to organize it, keep track of it and back it up. Why do that? I'm a programmer. Code I have written for fun or for some small experimental reason has no value to me because I can do it again and again at any time, like breathing. Sure, non-programmers value my code and they save it for posterity but I don't. It has no value to me once I've been paid for it.
Why would I go to all the trouble to keep a complete record of something I could do again at any time? Am I supposed to save what I'm typing right now to prove I can communicate? Do I save lawn clippings to prove I can operate a lawn mower? Do I save my waste to prove I can defecate?
You save code + accompanying documents in order to build up a portfolio to demonstrate to a potential employer how you work, how well you know your tools, and what your idea of a shippable product is. During the interview, you can talk about why you wrote the software, what designs you tried, what kinds of challenges you faced while doing so, and how you surmounted them.
At my company, we offer coding challenges for people who don't have a portfolio already. But really, why not just write software of your own choosing so that you don't need to do coding challenges to prove your ability? It's a far more efficient approach.
> If I saved it then I would have to organize it, keep track of it and back it up.
I really find this hard to swallow. It fails for me on so many counts.
Firstly, code is small compared to just about anything else you might keep on your computer. All you have to do to preserve it is drop it in a directory you keep for that purpose. The code doesn't have to be organized in order to preserve it. If you back up your computer, then your code is backed up. You back up your computer don't you?
Sure, it may be hard to find some useful piece of code later if you don't organize it, but preserving it, at least, is easy.
More importantly, you never write reusable code? You never write a library that can be applied to future problems? If not, then, you may not be the type of programmer that I'd want to work with anyway. I firmly believe that the way to increased productivity for any team is to factor out useful components and make them into more general purpose libraries. To do otherwise, is to end up with a codebase that is bloated with boilerplate.
Thirdly, if coding for you is as natural as breathing, surely you can whip up a couple of pages of code as a "code portfolio" in no time. You could have done that instead of typing in the above for your pleasure.
Not against you personally, but I'm fascinated by how clear people are about aspects of candidates they don't want to hire. There are so many dealbreakers in this thread (in the main), it's like a dating site full of people who have a lot of difficulty finding partners (hint: all dating sites are full of people with dealbreakers).
Github is an amazing product.
"I find this assertion to be virtually nonsensical. I've never met a programmer who hasn't written some code that they could show you"...
I've met plenty. Interviewed 6 candidates a few years ago - only one even brought anything beyond his resume. He brought a full physical portfolio of work he'd done. Most of it was web screenshots and such - not much 'code' to speak of, but he'd made the effort. No one else did.
I've been in plenty of places where people say "I can't show my work, cause it's for my previous employer". And they don't code when they go home. Or on weekends.
I brought code samples to an interview once, and the interviewer wouldn't look at it because "it wouldn't be fair to other people who don't bring code samples".
No doubt many people can, but you may be living in a bit of a self-selection bubble where everyone you know and mix with has loads of spare code to demonstrate. Many people don't. Now... if you want to use that as a signal for that person's interest level or ability, feel free to. You may be missing out on some otherwise competent or talented folks. But we all have to use signals to filter out candidates, and that may be a valid signal for you to use.
If you didn't ask candidates to bring code samples, it should come as no surprise that most don't, even though they might be able.
Where I work, we don't ask candidates to bring in a random code sample because instead we have them submit a coding exercise that they can do at home at their leisure. Our entire team then code reviews the code. Given the quality of most of the code submissions, it may be true that most programmers never program for fun, but if so, it shows this fact shows in the quality of their work.
All of the people who've given us nice submissions have also been very enthusiastic about what they do. I know this because we ask candidates if they read any programming journals or books just for their own edification, and the good ones always do. They always show some motivation beyond just completing assigned tasks.
We're a Scala shop, though, so we need people who aren't just programming because it's their job and for no other reason. Such people would have no motivation to become skilled in a complex programming language that is not yet mainstream.
I was doing PHP in 1996. If you were doing PHP in 1996, it was generally because you wanted to do it, learn it, etc (arguably web stuff in general then a bit too). There weren't many "PHP jobs" as such back then, so when you found other PHP developers, you knew they were enthusiasts of some sort.
That's where Scala's been, and almost any 'newish' language. If you find people adopting them, they're probably enthusiastic about it, and enjoy living on the cutting edge. :)
A senior engineer that has spent 5 years coding Big Table at Google would certainly (if genuinely interested in being hired) be able to take a day and throw up some code on github.
Not Big Table code of course, just some interesting code examples tailored to the position being negotiated.
Worried about people passing off copy/pasted code on github? That's what the interview is for. It's great to be able to discuss code that someone has written after you've taken some time to review it. They should be able to have a discussion about the code, their design choices, what they would do differently given a twist in the requirements, etc. If they can't do that: then even if they genuinely wrote the code they don't understand it enough.
I feel I should stress that the code doesn't have to be the next big open source project forked/watched by thousands.
Not comfortable putting code up on github because people might see it? I get that, it is scary to put your code out in public. In that case, a simple Dropbox (or whatever) shared folder would suffice.
In addition I would tend to ask a few important questions of developers:
1) Tell me about one case where you came back to code you had previously written and decided you didn't like the style. What was the issue? Did you rewrite it to fix it? Why or why not?
2) Tell me about one case where you departed from general design best practices where it worked. What did you do? Why did it work well?
3) Tell me about your commenting style. Why and how do you comment your code?
Oh and asking to see code samples is a good thing :-)
These sound like the sort of thing an HR person would ask, not a developer. Commenting style. Seriously? What in all honesty do you expect someone to say, you must know you're going to get spoon fed a perfect answer just for asking such a dumb question.
Why do you say that? As a developer the comments are talking to me, not to the HR department and not to the computer. I want to know someone has thought about this area regarding the way bad comments get in the way of collaboration and the way good comments facilitate it.
The point is that answers to these questions show the extent of introspection a programmer has regarding code. I am looking for thoughtful answers, not right answers. If you think they are stupid questions, I don't want to put you in charge of doing the basic creation of new frameworks.
Edit: One really good follow-up question is something like, 'So you are reading code and you see a comment that says "This is broken." Is that a good comment or a bad one? Why?' It's a good question because right or wrong isn't in "good" or "bad" but rather the reasons they might give.
Edit2: If it's such an easy question what's your answer? ;-)
"As a developer the comments are talking to me"
-- As a developer the code is talking to me.
It's a trivial question, comments are a chore that developers do because they provide someone who is later looking at and/or modifying their code with the knowledge about how it works in a way that isn't obvious from the code itself. Comments have morphed into more structured form with annotations that the IDE can make use of in some languages, but asking about comments in an interview is asking to get spoon fed what you want to hear.
If you are explaining how your code works, shouldn't you be rewriting it so it is obvious rather than adding comments?
I don't know about you but I go after people in open source projects I help manage who write comments like that. Such comments add no value collaboration and indeed they get in the way of debugging more than they help. Folks debugging shouldn't be debugging based on comments but rather based on code.
But with clear code, comments are still helpful if they are used to describe difficult design decisions, or flag issues the developer sees down the road. "This is a kludge" is a much more helpful comment than "looping through an array of invoice lines so we know what to print." The first draws the developer's attention to the fact that an area of code may be unusually troublesome, while the second may be out of date, may be wrong, or may otherwise distract from debugging.
Of course there are function header comments, etc, but those don't really need to be under discussion.
You are right about the code needing to talk to you, but if you see comments as a way of explaining what your code does, then it's both a distraction to the next guy and a crutch for you.
Edit: I think this shows the point of the question and why the trivial spoon-fed answers are likely to be ones I would find inadequate.
Edit2: So the way I read your answer is "programmer just wants to get things done and doesn't mind writing non-clear code to do it, explaining it with comments. Suitable for a maintenance programming role but may in time grow into others."
Here's a good example btw of a comment that I found very helpful.
One day a number of years ago, I was reading through some part of the Linux source code (somewhere in the 2.4 series I think), and I was looking at a section of the TCP stack and came across a comment to the effect of "This is the max timeout value as specified in the RFC. This means FTP to Mars will never work"
Perfect comment. Besides making me laugh, it explains why a specific line is the way it is and explains that this may cause some issues where latency in the connection is high enough.
More importantly, I think it ignores the reality in how the output is consumed. Writing is generally meant to be read; that is its use. When evaluating a text, the qualities it has when read by a human are the primary criteria.
When we evaluate code by reading it, our largest concern is how it will be interpreted by a machine. Clarity matters in comments, but can be justifiably sacrificed in code for an elegant or efficient solution.
It might be more appropriate to compare the requirements of composing legal documents to that of coding. Testing for understanding of methods, patterns and common concepts is far more applicable in that case. Asking if a candidate knows the parameters for a work to be classified as work-for-hire or what kinds of indemnifications and disclaimers should be in a contract are arguable useful indicators of suitability.
If you're writing code in any environment but an early-stage startup, the qualities it has when read by a human are ALSO the primary criteria.
Where is the fine line between coding tests and spec work?
I'm not a programmer, but I've seen startups co-founded by engineers try hiring designers in a similar fashion to I assume programmers as they test designers with a variety of unpaid projects. I've found it to be pretty annoying given that I've already shown a portfolio of work and it's not unlike spec work. I get that some people may have fabricated their history, but that's why some companies do a trial period before committing to a full-time gig.
I don't think the line is fine.
If I ask someone to implement a linked list during 1hr interview, it's clear that it's not because I need a linked list implementation in my company.
If I ask someone to stay for a week to work on the company's code base, that's clearly a work that should be paid for at a market rate.
Similarly, I'm pretty sure that you, as a designer, know without a doubt when company's request crosses the line from "I need to make sure you know what you're doing" to "I need you to do free work for me".
I don't like all their advice, but one thing that 37Signals does in this situation is just pay the person whatever rate they ask for. In the greater scheme of things, it's not a huge expense, and if you're to the point in the interview process where this actually makes sense, then you have a decent idea that this person is a candidate, and so you can invest a little in the process for everybody's benefit. It's a step beyond the whiteboarding "jump, monkey!" coding interview.
I see no problem with trying to hire developers from the contributor pool of the open source projects my company has helped found.
I was once asked a fairly involved technical question in an hour long interview - I later interviewed for a different group in the same organization and they mentioned the problem the other group had been having.
I talked about the solution I had proposed and was told - "oh yes, some consultant they had in suggested that and we tried it last week".