It's been known for a long time that standard interviews are a poor indicator of future job performance, so probably any analyses of data regarding behavioural modes, a person's intrinsic motivators, cultural fit etc, will do a slightly better job. Presumably all of this will still be subordinate to an on-the-job trial anyway, though.
It'd worry me that, although this guy says you need to increase behavioural diversity but minimise value diversity, that you're effectively just minimising the pool of potential employees, rather than figuring out ways that a larger pool of people could fit. Or in other words, whether it's Myers-Briggs, Belbin's roles, IQ tests etc, it seems that evaluation tools are trying to quantify the diversity of people, and pick off, with increasing accuracy, the exact archetype that aligns with organisational goals. But if you assume that people are diverse, and that employment roles are diverse, and that there's a large pool of both, would it not be a better idea to focus on quantifying the differences in attributes required for your employment roles, so as to maximise your pool of potential applicants?
I'm willing to bet that someone who's slobbish and lazy and unpleasant could play a valuable role if you could quantify the requirements and goals of positions in your company, for example. And does value alignment matter for all roles in all companies, or is it just a phenomenon arising in the last decade, being an intuitive way to maximise employee investment and increase profit? Why would financial staff need to want to 'change the world' or some bullshit to work at crappy Startup X?
if you could quantify the requirements and goals of positions in your company
That hits on one big problem I see with all this: How do you know what the company actually wants? All/Loads of companies claim to want the same thing, and have the same vision of "changing the world". But some companies are run by abusive wankers who want cheap developers who'll be happy working 24/7 for low money. You can tell by the company's actions, not their words. If the only pay peanuts, then they only want cheap labour.
Will this big data be turned around to the company, so employees can see what sort of place the company is?
I was also hoping to see the analysis apply both ways to some degree. Rather than simply having a paying company ask you to find the best fit, is it not better to have something akin to a dating service, where you're not just finding the best fit for one party; you're trying to find the best fit for all parties. An employee who's not a great fit for Company A may be a perfect fit for Company B -- this, ideally, is the sort of value I see from these ideas, rather than just ending with 'He's not a great fit for company A'.
Well, that might be nice, but I wouldn't expect to see it happen. The company formed to collect this data, Q, is a B2B play that will want both Companies A and B to pay for each individual assessment. Persons P and Q will not want to pay for the reverse service, and even if they were so inclined, Q's client base won't really be large enough to make it worthwhile.
In other words, hand-wavy big data aside, this is yet another HR tool.
I wish there were an easy way for people to be exposed to more companies and companies to be able to survey a broader range of people without the commitments of a formal revue process and the recruitment scam.
I really think this informal "who you know" is how many jobs actually happen. Something like speed dating but for employment.
An individualised test that advises you to steer clear of a company because you're way less deferential to management than their typical successful hire, or provides helpful hints that they're much more laissez faire than your previous employers is potentially a lot more useful than a review site (which like any review site can easily be abused by astroturfers or ex-employees fired for good reason).
I'm not a regular interviewer but I've been on a panel for a few recently. For one job in particular the standout candidate on paper gave the worst interview I have ever seen.
He was arrogant, confrontational, couldn't answer simple questions and was half an hour late - and not apologetic at all. He was just weirdly and inexplicably unpleasant to everyone.
Perhaps he would have been great in some role or would have aced a psychological test .. but I'm glad we did an interview so we could reject him out of hand.
I'm sure there is a place for widespread empirical evaluations .. but ultimately if you are employing someone to work with you there has to be space to say: "Err, No! - I don't care what the computer says".
Disclaimer: I work at a company that provides this type of service.
We are huge believers that this sort of testing is purely a pre-interview step. You get everyone to take it and you save hours and hours of digging through resumes. That's the big money saver.
Why do you think an empirical evaluation wouldn't have picked up on that?
It strikes me you could easily record the interview and run it past a machine looking for the tempo of the conversation, the tone of voice and so on - and see whether that correlated with employment a year later.
This may be a bit off topic but there really needs to be a hard line drawn on what information can be collected. The idea of recording an interview for the sake of empirical analysis is creepy as hell and I'm a bit worried anyone would think that is an appropriate thing to do.
Could you maybe explain your reasoning for that one?
I'd be for much stronger data controls, and that might rule something like that out just as collateral damage. Fair enough, it's not a very strong principle if you violate it the minute something you approve of comes along.
That said, I'm vastly more creeped out by the idea of someone collecting email addresses and looking up linkedin profiles for recruiting, or looking you up on facebook when you apply for a job, than I am with the idea of someone recording an interview. The latter is in a limited context, where you expect to be examined anyway. The former is an intrusion on your private life to one degree or another.
I'm not saying, by the way, that it's an acceptable thing to do behind someone's back - that would be extremely scummy.
I just don't really see the problem with running something past a computer. Would it make a difference to you if there was a typist there and you ran the transcript past the computer? Is it just the emotional ick of it being your voice they have on record?
Edit: Is it the possibility of mass data collection? I can see that being creepy with things like car number-plates because there the difference in speed and ability enables things like continual tracking of individuals day to day lives. But I'm not sure how that would be a concern here.
Sure, there are situations in which I believe there should be a certain expectation of privacy. Personal conversations like interviews are one of those situations. If I put something on facebook or linkedin I understand that there is a certain level of privacy I am giving up, it's a tradeoff many of us make for a service. Also, I have control over what I expose about myself on the internet, a recorded conversation that stays with an interviewer is not data I can control. If I have a one on one conversation with another individual or individual(s), I don't feel I should have to worry that what I'm saying is being recorded for analysis later on.
I'm willing to bet that someone who's slobbish and lazy and unpleasant could play a valuable role if you could quantify the requirements and goals of positions in your company, for example.
Yeah, but the positions of CEO and HR are usually filled at an early stage...
The best people are rarely going to sit though an on-the-job trial or the classic tech 6+ hours of interviews over 3+ days. More importantly the single most important part of the hiring process takes place after the fact. If you don't take care of your people and deal with people that are a poor fit minor changes in how you higher quickly becomes meaningless. On the other hand if you can manage people appropriately then you can almost randomly higher people and get by simply because your filtering based on a long term on-the-job trial after the fact.
Comments
It's been known for a long time that standard interviews are a poor indicator of future job performance, so probably any analyses of data regarding behavioural modes, a person's intrinsic motivators, cultural fit etc, will do a slightly better job. Presumably all of this will still be subordinate to an on-the-job trial anyway, though.
It'd worry me that, although this guy says you need to increase behavioural diversity but minimise value diversity, that you're effectively just minimising the pool of potential employees, rather than figuring out ways that a larger pool of people could fit. Or in other words, whether it's Myers-Briggs, Belbin's roles, IQ tests etc, it seems that evaluation tools are trying to quantify the diversity of people, and pick off, with increasing accuracy, the exact archetype that aligns with organisational goals. But if you assume that people are diverse, and that employment roles are diverse, and that there's a large pool of both, would it not be a better idea to focus on quantifying the differences in attributes required for your employment roles, so as to maximise your pool of potential applicants?
I'm willing to bet that someone who's slobbish and lazy and unpleasant could play a valuable role if you could quantify the requirements and goals of positions in your company, for example. And does value alignment matter for all roles in all companies, or is it just a phenomenon arising in the last decade, being an intuitive way to maximise employee investment and increase profit? Why would financial staff need to want to 'change the world' or some bullshit to work at crappy Startup X?
if you could quantify the requirements and goals of positions in your company
That hits on one big problem I see with all this: How do you know what the company actually wants? All/Loads of companies claim to want the same thing, and have the same vision of "changing the world". But some companies are run by abusive wankers who want cheap developers who'll be happy working 24/7 for low money. You can tell by the company's actions, not their words. If the only pay peanuts, then they only want cheap labour.
Will this big data be turned around to the company, so employees can see what sort of place the company is?
I was also hoping to see the analysis apply both ways to some degree. Rather than simply having a paying company ask you to find the best fit, is it not better to have something akin to a dating service, where you're not just finding the best fit for one party; you're trying to find the best fit for all parties. An employee who's not a great fit for Company A may be a perfect fit for Company B -- this, ideally, is the sort of value I see from these ideas, rather than just ending with 'He's not a great fit for company A'.
Well, that might be nice, but I wouldn't expect to see it happen. The company formed to collect this data, Q, is a B2B play that will want both Companies A and B to pay for each individual assessment. Persons P and Q will not want to pay for the reverse service, and even if they were so inclined, Q's client base won't really be large enough to make it worthwhile.
In other words, hand-wavy big data aside, this is yet another HR tool.
I wish there were an easy way for people to be exposed to more companies and companies to be able to survey a broader range of people without the commitments of a formal revue process and the recruitment scam.
I really think this informal "who you know" is how many jobs actually happen. Something like speed dating but for employment.
This sounds like what Glassdoor does. Even small local companies get reviews on it.
An individualised test that advises you to steer clear of a company because you're way less deferential to management than their typical successful hire, or provides helpful hints that they're much more laissez faire than your previous employers is potentially a lot more useful than a review site (which like any review site can easily be abused by astroturfers or ex-employees fired for good reason).
I'm not a regular interviewer but I've been on a panel for a few recently. For one job in particular the standout candidate on paper gave the worst interview I have ever seen.
He was arrogant, confrontational, couldn't answer simple questions and was half an hour late - and not apologetic at all. He was just weirdly and inexplicably unpleasant to everyone.
Perhaps he would have been great in some role or would have aced a psychological test .. but I'm glad we did an interview so we could reject him out of hand.
I'm sure there is a place for widespread empirical evaluations .. but ultimately if you are employing someone to work with you there has to be space to say: "Err, No! - I don't care what the computer says".
Disclaimer: I work at a company that provides this type of service.
We are huge believers that this sort of testing is purely a pre-interview step. You get everyone to take it and you save hours and hours of digging through resumes. That's the big money saver.
Or he took one look at the place, decided he (or she) did not want the job and tanked the interview.
I confess, I don't see how that would make it reasonable to act like an incompetent jerk. You can "pass" the interview and decline the job offer.
Or just politely say, "Now that I'm here, I can really see this is not the opportunity for me. Let's not waste any more time. Good day."
We're all lovely people and we served nice coffee and delicious biscuits.
eww, biscuits!
I think that sometimes happens subconsciously. People internally don't want to the job and tank the interview.
Why do you think an empirical evaluation wouldn't have picked up on that?
It strikes me you could easily record the interview and run it past a machine looking for the tempo of the conversation, the tone of voice and so on - and see whether that correlated with employment a year later.
This may be a bit off topic but there really needs to be a hard line drawn on what information can be collected. The idea of recording an interview for the sake of empirical analysis is creepy as hell and I'm a bit worried anyone would think that is an appropriate thing to do.
Could you maybe explain your reasoning for that one?
I'd be for much stronger data controls, and that might rule something like that out just as collateral damage. Fair enough, it's not a very strong principle if you violate it the minute something you approve of comes along.
That said, I'm vastly more creeped out by the idea of someone collecting email addresses and looking up linkedin profiles for recruiting, or looking you up on facebook when you apply for a job, than I am with the idea of someone recording an interview. The latter is in a limited context, where you expect to be examined anyway. The former is an intrusion on your private life to one degree or another.
I'm not saying, by the way, that it's an acceptable thing to do behind someone's back - that would be extremely scummy.
I just don't really see the problem with running something past a computer. Would it make a difference to you if there was a typist there and you ran the transcript past the computer? Is it just the emotional ick of it being your voice they have on record?
Edit: Is it the possibility of mass data collection? I can see that being creepy with things like car number-plates because there the difference in speed and ability enables things like continual tracking of individuals day to day lives. But I'm not sure how that would be a concern here.
Sure, there are situations in which I believe there should be a certain expectation of privacy. Personal conversations like interviews are one of those situations. If I put something on facebook or linkedin I understand that there is a certain level of privacy I am giving up, it's a tradeoff many of us make for a service. Also, I have control over what I expose about myself on the internet, a recorded conversation that stays with an interviewer is not data I can control. If I have a one on one conversation with another individual or individual(s), I don't feel I should have to worry that what I'm saying is being recorded for analysis later on.
Ah, I see how that would make some people uncomfortable. Thank you! ^_^
Yeah, but the positions of CEO and HR are usually filled at an early stage...
The best people are rarely going to sit though an on-the-job trial or the classic tech 6+ hours of interviews over 3+ days. More importantly the single most important part of the hiring process takes place after the fact. If you don't take care of your people and deal with people that are a poor fit minor changes in how you higher quickly becomes meaningless. On the other hand if you can manage people appropriately then you can almost randomly higher people and get by simply because your filtering based on a long term on-the-job trial after the fact.