The vector of skills vs. vector of job properties thing could be a coherent idea. I'd argue it's not a good idea (and I have created joint-embedding models for multi-modal text and image nearest neighbor search systems). But it could be a coherent idea.
In deep learning anyway, it's a very similar idea in spirit to something like word2vec. In word2vec you learn how to create some arbitrary vector of real numbers, let's say a 256-component vector, that represents words. The components of the vector for each word are learned by forcing the model to solve a prediction task, like predicting whether two words were truly seen in the same context in a given document or not. The model updates the 256-vector of each word such that this prediction task (which can be based on the inner product of those 256-vectors for sets of words) gets better and better. By the end of this, you can in some sense claim a word's mapping into its 256-vector "represents" some "context" intrinsic to that word. And that 256-dimensional "space of context" where the vectors reside is known as an embedding space. So the 256-vector of a word is often called that word's embedding.
For more generic problem, you aren't learning just one type of embedding, but possibly many. In TripleByte's case, I'm guessing they are inputting text characteristics of candidate resumes, self-descriptions, GitHub repos, Stack Overflow profiles, LinkedIn pages, etc. etc., and treating a candidate like a bag-of-words from all those combined sources of data. And maybe they also have other data too, like performance on skills assessments, favorite craft beer, and whether you think dog-friendly offices are unprofessional or you think they are super cool.
On the other side they input a bunch of data about jobs: bag-of-words text from job ads, tech blogs from that company, the company's GitHub repos if any are public facing, profiles of their existing team members, what gourmet coffee they keep in the office, and whether or not they believe in "unlimited" vacation.
If they have labeled training data, like say a bunch of people who work in their client companies that offered to give them resumes and personal data, and then use the existing job those people occupy as a ground truth label for a "positive" matching between a candidate and a job, then they could train a model that learns how to map all that candidate data and also all that job-specific data both into 256-component embedding vectors that are constrained to reside in the same space.
And if they really want for the dot product to be the key way to describe applicant-to-job similarity, they could make the loss function work based on cosine distance between the learned embedding vectors of candidates and the learned embedding vectors of their ground truth actual jobs.
I have no idea if this approach, or something like it, is what TripleByte actually does. But it sounds like they want to market themselves as really fancy machine learning engineers by at least talking about this.
Frankly, in deep learning this is a fairly cookie cutter approach and is often the first thing someone tries. In fact, I just Googled for "machine learning visual searches" and one of the first links was this post from the Squarespace engineering team (I do not work at or have any affiliation with Squarespace):
From the blog post it looks like this was just a sort of fun side project, with a goal of developing it into an actual search service for Squarespace's search engine at some point (maybe they already did?).
The reason why TripleByte's attempts to use this in advertisements fall flat is because it's a commonplace idea -- so common that here we even have a website hosting and content platform demoing how the basic idea works for image searching as some random blog post.
My guess is that other experienced ML engineers would see TripleByte's choice with this ad as at best a little self-aggrandizing and silly, or at worst maybe even intellectually dishonest about the real importance or effectiveness of this type of embedding-based similarity approach.
I don’t work in the “high-tech HR” space, but do have experience with embedding vector methods, joint-embeddings, and trying to solve generic information retrieval tasks, where the underlying data is not text or images, by using embedding methods.
Even in a domain like reverse image search, where this approach has been studied to death, there are big concerns about how much of the problem is solved because a vector similarity approach can approximate the real manifold distance of some underlying true structure vs. just having huge models overfit to a particular class of natural image statistics.
This problem is discussed in [0], which creates a lot of problems for people who want to believe that some deep, internal layer of a neural net can capture semantically relevant features.
But a much bigger problem looms for trying to extend this idea to matching people to jobs. In that problem, you don’t even have the option of overfitting to population statistics because the population is constantly changing and the individuals in the population have an insanely high-dimensional set of internal unobserved variables, like their emotions, goals, current life or family priorities, interests, relative free time, sensitivity to stress, etc.
By comparison, the space of latent variables giving rise to observed natural photos is tiny.
Essentially, if you really wanted to take a scary, Orwellian, big data approach to quantifying a candidate’s degree of match to a job, you would need much more data on the conditional distribution of the observables (resume items, college degree, skills assessment, etc) when given information on the internal state (work ethic, introvert vs extravert, motivation for looking for a job, intelligence metrics, disposition, response to stress, etc) at a given moment of time.
This would let you model the posterior distribution of those hidden, internal characteristics of the applicant, and those characteristics could maybe be used to understand a holistic match to a certain employer-team-role situation.
But that’s a ludicrously high-dimensiomal problem that observable data like resumes or skills assessments does little to solve.
So overall, I probably share your opinion that this is an extremely shallow model.
It’s like learning a vector space model of spaghetti and a vector space model of walls and then claiming your model can predict what will stick when you throw spaghetti at the wall.
I do like that last bit. For full disclosure: I'm sure that for many places their recruiting process is so bad that just enumerating the required skills with some external reference will improve outcomes, but I'm not sure that this is what people think is actually going on.
Comments
The vector of skills vs. vector of job properties thing could be a coherent idea. I'd argue it's not a good idea (and I have created joint-embedding models for multi-modal text and image nearest neighbor search systems). But it could be a coherent idea.
In deep learning anyway, it's a very similar idea in spirit to something like word2vec. In word2vec you learn how to create some arbitrary vector of real numbers, let's say a 256-component vector, that represents words. The components of the vector for each word are learned by forcing the model to solve a prediction task, like predicting whether two words were truly seen in the same context in a given document or not. The model updates the 256-vector of each word such that this prediction task (which can be based on the inner product of those 256-vectors for sets of words) gets better and better. By the end of this, you can in some sense claim a word's mapping into its 256-vector "represents" some "context" intrinsic to that word. And that 256-dimensional "space of context" where the vectors reside is known as an embedding space. So the 256-vector of a word is often called that word's embedding.
For more generic problem, you aren't learning just one type of embedding, but possibly many. In TripleByte's case, I'm guessing they are inputting text characteristics of candidate resumes, self-descriptions, GitHub repos, Stack Overflow profiles, LinkedIn pages, etc. etc., and treating a candidate like a bag-of-words from all those combined sources of data. And maybe they also have other data too, like performance on skills assessments, favorite craft beer, and whether you think dog-friendly offices are unprofessional or you think they are super cool.
On the other side they input a bunch of data about jobs: bag-of-words text from job ads, tech blogs from that company, the company's GitHub repos if any are public facing, profiles of their existing team members, what gourmet coffee they keep in the office, and whether or not they believe in "unlimited" vacation.
If they have labeled training data, like say a bunch of people who work in their client companies that offered to give them resumes and personal data, and then use the existing job those people occupy as a ground truth label for a "positive" matching between a candidate and a job, then they could train a model that learns how to map all that candidate data and also all that job-specific data both into 256-component embedding vectors that are constrained to reside in the same space.
And if they really want for the dot product to be the key way to describe applicant-to-job similarity, they could make the loss function work based on cosine distance between the learned embedding vectors of candidates and the learned embedding vectors of their ground truth actual jobs.
I have no idea if this approach, or something like it, is what TripleByte actually does. But it sounds like they want to market themselves as really fancy machine learning engineers by at least talking about this.
Frankly, in deep learning this is a fairly cookie cutter approach and is often the first thing someone tries. In fact, I just Googled for "machine learning visual searches" and one of the first links was this post from the Squarespace engineering team (I do not work at or have any affiliation with Squarespace):
< https://engineering.squarespace.com/blog/2017/using-machine-... >
From the blog post it looks like this was just a sort of fun side project, with a goal of developing it into an actual search service for Squarespace's search engine at some point (maybe they already did?).
The reason why TripleByte's attempts to use this in advertisements fall flat is because it's a commonplace idea -- so common that here we even have a website hosting and content platform demoing how the basic idea works for image searching as some random blog post.
My guess is that other experienced ML engineers would see TripleByte's choice with this ad as at best a little self-aggrandizing and silly, or at worst maybe even intellectually dishonest about the real importance or effectiveness of this type of embedding-based similarity approach.
I'm sure they'd like their investors to believe it is plausible and obvious. Working in a related space, I strongly disagree.
I don’t work in the “high-tech HR” space, but do have experience with embedding vector methods, joint-embeddings, and trying to solve generic information retrieval tasks, where the underlying data is not text or images, by using embedding methods.
Even in a domain like reverse image search, where this approach has been studied to death, there are big concerns about how much of the problem is solved because a vector similarity approach can approximate the real manifold distance of some underlying true structure vs. just having huge models overfit to a particular class of natural image statistics.
This problem is discussed in [0], which creates a lot of problems for people who want to believe that some deep, internal layer of a neural net can capture semantically relevant features.
But a much bigger problem looms for trying to extend this idea to matching people to jobs. In that problem, you don’t even have the option of overfitting to population statistics because the population is constantly changing and the individuals in the population have an insanely high-dimensional set of internal unobserved variables, like their emotions, goals, current life or family priorities, interests, relative free time, sensitivity to stress, etc.
By comparison, the space of latent variables giving rise to observed natural photos is tiny.
Essentially, if you really wanted to take a scary, Orwellian, big data approach to quantifying a candidate’s degree of match to a job, you would need much more data on the conditional distribution of the observables (resume items, college degree, skills assessment, etc) when given information on the internal state (work ethic, introvert vs extravert, motivation for looking for a job, intelligence metrics, disposition, response to stress, etc) at a given moment of time.
This would let you model the posterior distribution of those hidden, internal characteristics of the applicant, and those characteristics could maybe be used to understand a holistic match to a certain employer-team-role situation.
But that’s a ludicrously high-dimensiomal problem that observable data like resumes or skills assessments does little to solve.
So overall, I probably share your opinion that this is an extremely shallow model.
It’s like learning a vector space model of spaghetti and a vector space model of walls and then claiming your model can predict what will stick when you throw spaghetti at the wall.
[0]: < https://arxiv.org/abs/1711.11561 >
I do like that last bit. For full disclosure: I'm sure that for many places their recruiting process is so bad that just enumerating the required skills with some external reference will improve outcomes, but I'm not sure that this is what people think is actually going on.