This seems like a tough build vs buy sell. For a lot (most?) companies, the search/recommendation system isn't necessarily optimized for the customer's search. Instead, it's a way to maximize revenue via preferred placement or inject ads. This almost always leads to a gigantic if/else chain of bespoke business analyst driven decisions for the marketplace.
How are you going to allow folks to influence the system? Or do you see your system integrated behind their pseudo-recommendation engine?
The build vs buy decision does come up, but like you mentioned, the product direction of Shaped is to be primitives for search and recommendation, allowing users that want to build use Shaped to empower them to build quicker (e.g. integrated behind their psudo-recommendation engine). In truth we have multiple abstractions to Shaped allowing more technical teams to integrate like this, or less technical ones to have more of an end-to-end integration experience.
The other related market trend we think about here: recommendation is going through a similar journey to what search did 10 years ago. Search at some point was more build leaning, but over time the technology became democratized and then companies like Elastic and Algolia had offerings that pushed search to lean towards buy. We're seeing recommendations going through the same revolution now that the technologies and system design (e.g. 4 stage recommenders) are more solidified. It's the data that makes these systems unique between companies not the infrastructure or algorithms.
Dan here. On bespoke business decisions—these are handled through SQL and model configs. This involves deciding what items to filter, how to set the objective function (what to optimize the model for), and controlling exploration and diversity in the results.
Setting the objective function is often the most challenging. Different teams may prioritize different objectives and often it requires balancing multiple at once! For instance, how does a company think about the types of user engagement and long-term metrics like retention? A model optimized for clicks might be worse for retention in some cases, but not in others. Ultimately, we A/B test to find out. Surprises and counter-intuitive results are common!
Can you tell me what industry your viewpoint is from? My viewpoint from another industry is also about maximizing revenue - but if/else statements have no part, it's data-derived.
Comments
This seems like a tough build vs buy sell. For a lot (most?) companies, the search/recommendation system isn't necessarily optimized for the customer's search. Instead, it's a way to maximize revenue via preferred placement or inject ads. This almost always leads to a gigantic if/else chain of bespoke business analyst driven decisions for the marketplace.
How are you going to allow folks to influence the system? Or do you see your system integrated behind their pseudo-recommendation engine?
The build vs buy decision does come up, but like you mentioned, the product direction of Shaped is to be primitives for search and recommendation, allowing users that want to build use Shaped to empower them to build quicker (e.g. integrated behind their psudo-recommendation engine). In truth we have multiple abstractions to Shaped allowing more technical teams to integrate like this, or less technical ones to have more of an end-to-end integration experience.
The other related market trend we think about here: recommendation is going through a similar journey to what search did 10 years ago. Search at some point was more build leaning, but over time the technology became democratized and then companies like Elastic and Algolia had offerings that pushed search to lean towards buy. We're seeing recommendations going through the same revolution now that the technologies and system design (e.g. 4 stage recommenders) are more solidified. It's the data that makes these systems unique between companies not the infrastructure or algorithms.
Dan here. On bespoke business decisions—these are handled through SQL and model configs. This involves deciding what items to filter, how to set the objective function (what to optimize the model for), and controlling exploration and diversity in the results.
Setting the objective function is often the most challenging. Different teams may prioritize different objectives and often it requires balancing multiple at once! For instance, how does a company think about the types of user engagement and long-term metrics like retention? A model optimized for clicks might be worse for retention in some cases, but not in others. Ultimately, we A/B test to find out. Surprises and counter-intuitive results are common!
Can you tell me what industry your viewpoint is from? My viewpoint from another industry is also about maximizing revenue - but if/else statements have no part, it's data-derived.
A company doing that doesn't understand LTV.