Companies seem to be adding more screening steps to try to reduce their false positive rate -- the rate at which they interview people who aren't hireable.
But most don't seem to understand that mathematically, there's a tradeoff for a higher false negative rate.
Screening false negatives are people who would have done well in an interview, but don't make it to that stage. These are more hidden to the company, but quite expensive for the hiring process, and painful for people who are wrongly rejected. If we put this in probabilistic terms, I hope we can have a deeper conversation about what's happening and how this issue impacts engineers.
I couldn't agree more. I think the illusion that hiring is a deterministic and rational process, combined with the fear of firing, leads to adding more and more filters, which just makes things worse.
It's a really expensive and inefficient system, and tends to worsen as companies grow. It would be much better to hire fast and fire fast.
As I recently blogged about this issue:
You may be one of those companies that makes much of hiring only the top 1% of the top 1% of applicants. This can be good for morale (“We are the best of the best”) but a too-selective hiring process is quite hazardous. If you find a way to effectively measure your false-negative rate, (the no-hires that you should have hired) you may find out that half your current team wouldn’t make it through your current hiring process. If people commonly apply to your company three or four times before getting in, your highly-selective hiring process is probably filtering on random noise more than skill or talent. In other words, your precision and recall[1] are both bad, the bad results are coming from a costly process, and adding more filters just makes things worse.
At least where I work, we are painfully aware of this tradeoff, but are willing to make the sacrifice because of the severe costs of making a bad hire. We are small, and cannot afford to lose 90 days.
I realize this article refutes this point of view, but finance and HR do not see it this way.
I am more happy to devote several hours of each week interviewing than entire days/weeks mentoring and reviewing code for a bad hire, or worse, dealing with the widespread consequences of a culture misfit in a small team.
At least where I work, we are painfully aware of this tradeoff, but are willing to make the sacrifice because of the severe costs of making a bad hire. We are small, and cannot afford to lose 90 days.
Small or large, I think this is fair. But also fair is to take this into account when making claims about any alleged shortage of qualified workers.
Not that you or your company have necessarily made any such claims but plenty of companies that similarly follow a low false positive at the expense of false negatives approach have done so, including in sworn testimony before legislative bodies.
I'd distinguish between screening false positives (which my article focuses on) and hiring false positives. Screening rejections (before the final interview) are usually done with very limited information, so there's more room for bias and noise.
I've been the tail end of the interview funnel when experimenting with our screening strictness. The emotional toll that less stringent screening had on my day-to-day work (faster, rapid, and high frequency on sites) was extremely high. It led to a month or two of burnout as the lead engineer on the team, despite probably leading to finding a hire sooner. The cost is, in my opinion, immeasurable.
What I'm worried about is the b2b crowdsourcing of these rankings between companies. Corporations have shown an affinity for sharing data, like Facebook and Google buying ad targeting data. I feel like it is only a matter of time before your tinder data, resume data, etc., is all fed into your credit score, etc., to the point where a minority of the population is completely screwed over by false negative rates made by bad data scientists.
Any type of score is regulated by the Fair Credit and reporting act. These scores may fall under that regulation, especially if they are operating as a clearinghouse.
That's true for your credit score, but these screenings could effectively blacklist someone from the industry if a bad encounter with TripleByte or someone like them is shared between all employers.
I wouldn't be surprised if TripleByte has already triangulated data with HN or other YC companies to get additional profile information on applicants.
The unpredictability of what will show up on the tests, and the unknown future of my 'grade', is a major turn-off. I'd prefer to see a certification process, where you can study for the test, then take the test, and pay to retake it if you want to improve your score. If the technical knowledge in the tests is that important for their clients, then sharing that openly will increase the pool of successful candidates over the long term.
Comments
Companies seem to be adding more screening steps to try to reduce their false positive rate -- the rate at which they interview people who aren't hireable.
But most don't seem to understand that mathematically, there's a tradeoff for a higher false negative rate.
Screening false negatives are people who would have done well in an interview, but don't make it to that stage. These are more hidden to the company, but quite expensive for the hiring process, and painful for people who are wrongly rejected. If we put this in probabilistic terms, I hope we can have a deeper conversation about what's happening and how this issue impacts engineers.
I couldn't agree more. I think the illusion that hiring is a deterministic and rational process, combined with the fear of firing, leads to adding more and more filters, which just makes things worse.
It's a really expensive and inefficient system, and tends to worsen as companies grow. It would be much better to hire fast and fire fast.
As I recently blogged about this issue:
You may be one of those companies that makes much of hiring only the top 1% of the top 1% of applicants. This can be good for morale (“We are the best of the best”) but a too-selective hiring process is quite hazardous. If you find a way to effectively measure your false-negative rate, (the no-hires that you should have hired) you may find out that half your current team wouldn’t make it through your current hiring process. If people commonly apply to your company three or four times before getting in, your highly-selective hiring process is probably filtering on random noise more than skill or talent. In other words, your precision and recall[1] are both bad, the bad results are coming from a costly process, and adding more filters just makes things worse.
[1]: https://en.wikipedia.org/wiki/Precision_and_recall
At least where I work, we are painfully aware of this tradeoff, but are willing to make the sacrifice because of the severe costs of making a bad hire. We are small, and cannot afford to lose 90 days.
I realize this article refutes this point of view, but finance and HR do not see it this way.
I am more happy to devote several hours of each week interviewing than entire days/weeks mentoring and reviewing code for a bad hire, or worse, dealing with the widespread consequences of a culture misfit in a small team.
Hiring is hard.
At least where I work, we are painfully aware of this tradeoff, but are willing to make the sacrifice because of the severe costs of making a bad hire. We are small, and cannot afford to lose 90 days.
Small or large, I think this is fair. But also fair is to take this into account when making claims about any alleged shortage of qualified workers.
Not that you or your company have necessarily made any such claims but plenty of companies that similarly follow a low false positive at the expense of false negatives approach have done so, including in sworn testimony before legislative bodies.
(Original article author here.)
I'd distinguish between screening false positives (which my article focuses on) and hiring false positives. Screening rejections (before the final interview) are usually done with very limited information, so there's more room for bias and noise.
Hiring is hard! :)
You're right, the difference is important.
I've been the tail end of the interview funnel when experimenting with our screening strictness. The emotional toll that less stringent screening had on my day-to-day work (faster, rapid, and high frequency on sites) was extremely high. It led to a month or two of burnout as the lead engineer on the team, despite probably leading to finding a hire sooner. The cost is, in my opinion, immeasurable.
What I'm worried about is the b2b crowdsourcing of these rankings between companies. Corporations have shown an affinity for sharing data, like Facebook and Google buying ad targeting data. I feel like it is only a matter of time before your tinder data, resume data, etc., is all fed into your credit score, etc., to the point where a minority of the population is completely screwed over by false negative rates made by bad data scientists.
Any type of score is regulated by the Fair Credit and reporting act. These scores may fall under that regulation, especially if they are operating as a clearinghouse.
That's true for your credit score, but these screenings could effectively blacklist someone from the industry if a bad encounter with TripleByte or someone like them is shared between all employers.
Still relevant and still covered.
I wouldn't be surprised if TripleByte has already triangulated data with HN or other YC companies to get additional profile information on applicants.
The unpredictability of what will show up on the tests, and the unknown future of my 'grade', is a major turn-off. I'd prefer to see a certification process, where you can study for the test, then take the test, and pay to retake it if you want to improve your score. If the technical knowledge in the tests is that important for their clients, then sharing that openly will increase the pool of successful candidates over the long term.