1.) Over time, the number of new players playing a game decreases, to the point where most writes will be updates. That being said, in some cases we have access to totals via the APIs, so we just do a replace/upsert.
2.) We tend to do a bit of both. One of the main uses we have for aggregation right now is for global-level analytics (ie. how are certain weapons in the game being utilized over time)
I think the second approach sounds more closely aligned to what we're interested in doing.
The trouble with any of these solutions (and this is where I sort of neglected to make my description of the problem more complex) is that their fairly one dimensional. The reality is that one player may have multiple characters each with their own stats, and we may want to track the statistics of each weapon of each character for each player as well. We may want to track all of this per playlist as well. The relationships start to get very complex, as does the data.
Well, one of the reasons they call it Big Data is that it can get big. You can store the same data point multiple times - once in the "player" record, once in the "game" record, once in the "weapon" record, etc. Whatever makes it faster/easier to process.
You have to forget everything you learned about "normalization" if you've ever studied relational databases and SQL. If redundant copies or even multiple platforms and data models allow you to make faster queries, use them!
Comments
1.) Over time, the number of new players playing a game decreases, to the point where most writes will be updates. That being said, in some cases we have access to totals via the APIs, so we just do a replace/upsert.
2.) We tend to do a bit of both. One of the main uses we have for aggregation right now is for global-level analytics (ie. how are certain weapons in the game being utilized over time)
I think the second approach sounds more closely aligned to what we're interested in doing.
The trouble with any of these solutions (and this is where I sort of neglected to make my description of the problem more complex) is that their fairly one dimensional. The reality is that one player may have multiple characters each with their own stats, and we may want to track the statistics of each weapon of each character for each player as well. We may want to track all of this per playlist as well. The relationships start to get very complex, as does the data.
Well, one of the reasons they call it Big Data is that it can get big. You can store the same data point multiple times - once in the "player" record, once in the "game" record, once in the "weapon" record, etc. Whatever makes it faster/easier to process.
You have to forget everything you learned about "normalization" if you've ever studied relational databases and SQL. If redundant copies or even multiple platforms and data models allow you to make faster queries, use them!