"There is no persistence layer involved for experiment group allocation, so that we could minimize the latency/load on our production services. We left all of the complicated metric computations to offline data processing." Does that mean that the experiment to which the user is allocated is stored through a log in memory instead of being stored on disk/on disk and in memory (through logs)? If so, are there frequent trade-offs between memory/cpu usage in running experiments?
Putting ("triggering") someone into an experiment group is most often some function calls: do they meet the criteria (e.g., country, ..), do they fall in the right bucket. The fact that they've been triggered is logged to Kafka, an open-source messaging system whose logs we push (eventually) into Hive.
Comments
"There is no persistence layer involved for experiment group allocation, so that we could minimize the latency/load on our production services. We left all of the complicated metric computations to offline data processing." Does that mean that the experiment to which the user is allocated is stored through a log in memory instead of being stored on disk/on disk and in memory (through logs)? If so, are there frequent trade-offs between memory/cpu usage in running experiments?
Putting ("triggering") someone into an experiment group is most often some function calls: do they meet the criteria (e.g., country, ..), do they fall in the right bucket. The fact that they've been triggered is logged to Kafka, an open-source messaging system whose logs we push (eventually) into Hive.