I don’t work in the “high-tech HR” space, but do have experience with embedding vector methods, joint-embeddings, and trying to solve generic information retrieval tasks, where the underlying data is not text or images, by using embedding methods.
Even in a domain like reverse image search, where this approach has been studied to death, there are big concerns about how much of the problem is solved because a vector similarity approach can approximate the real manifold distance of some underlying true structure vs. just having huge models overfit to a particular class of natural image statistics.
This problem is discussed in [0], which creates a lot of problems for people who want to believe that some deep, internal layer of a neural net can capture semantically relevant features.
But a much bigger problem looms for trying to extend this idea to matching people to jobs. In that problem, you don’t even have the option of overfitting to population statistics because the population is constantly changing and the individuals in the population have an insanely high-dimensional set of internal unobserved variables, like their emotions, goals, current life or family priorities, interests, relative free time, sensitivity to stress, etc.
By comparison, the space of latent variables giving rise to observed natural photos is tiny.
Essentially, if you really wanted to take a scary, Orwellian, big data approach to quantifying a candidate’s degree of match to a job, you would need much more data on the conditional distribution of the observables (resume items, college degree, skills assessment, etc) when given information on the internal state (work ethic, introvert vs extravert, motivation for looking for a job, intelligence metrics, disposition, response to stress, etc) at a given moment of time.
This would let you model the posterior distribution of those hidden, internal characteristics of the applicant, and those characteristics could maybe be used to understand a holistic match to a certain employer-team-role situation.
But that’s a ludicrously high-dimensiomal problem that observable data like resumes or skills assessments does little to solve.
So overall, I probably share your opinion that this is an extremely shallow model.
It’s like learning a vector space model of spaghetti and a vector space model of walls and then claiming your model can predict what will stick when you throw spaghetti at the wall.
I do like that last bit. For full disclosure: I'm sure that for many places their recruiting process is so bad that just enumerating the required skills with some external reference will improve outcomes, but I'm not sure that this is what people think is actually going on.
Comments
I'm sure they'd like their investors to believe it is plausible and obvious. Working in a related space, I strongly disagree.
I don’t work in the “high-tech HR” space, but do have experience with embedding vector methods, joint-embeddings, and trying to solve generic information retrieval tasks, where the underlying data is not text or images, by using embedding methods.
Even in a domain like reverse image search, where this approach has been studied to death, there are big concerns about how much of the problem is solved because a vector similarity approach can approximate the real manifold distance of some underlying true structure vs. just having huge models overfit to a particular class of natural image statistics.
This problem is discussed in [0], which creates a lot of problems for people who want to believe that some deep, internal layer of a neural net can capture semantically relevant features.
But a much bigger problem looms for trying to extend this idea to matching people to jobs. In that problem, you don’t even have the option of overfitting to population statistics because the population is constantly changing and the individuals in the population have an insanely high-dimensional set of internal unobserved variables, like their emotions, goals, current life or family priorities, interests, relative free time, sensitivity to stress, etc.
By comparison, the space of latent variables giving rise to observed natural photos is tiny.
Essentially, if you really wanted to take a scary, Orwellian, big data approach to quantifying a candidate’s degree of match to a job, you would need much more data on the conditional distribution of the observables (resume items, college degree, skills assessment, etc) when given information on the internal state (work ethic, introvert vs extravert, motivation for looking for a job, intelligence metrics, disposition, response to stress, etc) at a given moment of time.
This would let you model the posterior distribution of those hidden, internal characteristics of the applicant, and those characteristics could maybe be used to understand a holistic match to a certain employer-team-role situation.
But that’s a ludicrously high-dimensiomal problem that observable data like resumes or skills assessments does little to solve.
So overall, I probably share your opinion that this is an extremely shallow model.
It’s like learning a vector space model of spaghetti and a vector space model of walls and then claiming your model can predict what will stick when you throw spaghetti at the wall.
[0]: < https://arxiv.org/abs/1711.11561 >
I do like that last bit. For full disclosure: I'm sure that for many places their recruiting process is so bad that just enumerating the required skills with some external reference will improve outcomes, but I'm not sure that this is what people think is actually going on.