Tuesday, February 15, 2011

Understanding The Matchmaking System in Starcraft II: Addendum (In Progress)

(First, if you haven't had the chance to read the preceding post, you may consider doing that.  The following post assumes knowledge of the preceding post.)


Introduction

First, I just want to say thanks again for everyone who has read and commented about the previous post.  It represents weeks of personal data analysis, and for all the people who have sent me messages offering to help, I'd just like to extend a hand of gratitude (or arms, if you're a hug guy like me).

Truthbetold, my only reason for doing this in the first place was because I couldn't get myself to consciously ladder without satisfying the curiosity of knowing where I stood relative to everyone else.  I am a very competitive person and I tend to get compulsively addicted to anything I do.  As such, being unable to rate myself with other players, I then became set on and consumed by figuring out this system enough to satisfy myself.

I feel that I've accomplished that, and as such am losing interest in pursuit of the final pieces of the puzzle--partly because I'm content, but mostly because the promotion aspect hopefully won't apply to me once I can sit down, improve, and just play more games.

The goal of this addendum will be to address some of the prevalent questions, criticisms and comments about the post.  I'll try to be clearer about things I wasn't as clear about before, in hopes that it will better enable other data-curious individuals to continue this work, should they choose to (and I hope you do, because I'm still very curious!).



A Modeling Primer and the Promotion Prediction Equation

First and foremost, I feel the extended focus on the Promotion Prediction Equation to be a bit unfortunate.  This was one of the very last things I created before posting that, and I didn't properly address the limitations of that equation.


W = Win %
Pp = Rank Point Differential
M = Magic Number
A = Average Win (bonus pool adjusted)*
B = Average Loss
G = Games required for promotion. (This is what we're calculating.)
*Bonus Pool matches your earned points from a win.  Subtract out these gained points from each win.

G = 5 + [M - Pp] / [W*A + (W-1)*B]


As with any model, it is only as accurate as the most current information available.  Even with every unknown solved, access to the real function(s) used in calculating MMR and promotion, and working at Blizzard, it is not possible to determine exactly when you will be promoted beyond one game.

You must actually play the games, and with every new piece of data you create, the model may become less and less accurate, as it forecasts future data, which you yourself must determine.

However, for players who are far away from promotion, this equation creates a simple static-slope line that approximates how many games you need to play for your Rank Point Differential to equal or eclipse your corresponding Magic Number.  5 games are then added to get your Simple Moving Average over 10 games to also equal or eclipse that Magic Number.

In other words, because the slope is a constant, it assumes a static linear increase in your MMR (usually between 1-2 points).  For an average win of 13 and loss of 11 at 50% win rate, this equation assumes that you alternate wins and losses every game such that your MMR jumps up 1 point per game until you reach  your M-value.

This equation makes assumptions based on an already-simplified equation, such that you can very quickly get a rough estimate if you are far away from promotion.

If you have a win rate less than 50%, you may get a negative number, because you may never get promoted with such a win rate.  Also, if M - Pp < 0, you're going to get a negative number as well, because your Rank Point Differential has already eclipsed the Magic Number.

At this point, the equation has limited if any predictive value because you're very close to promotion, and this equation is optimized for players who are not close.

Therefore, if you decide to make use of this equation, you should keep in mind that it's only useful as a generalized estimator of the number of games you would need to play, at your current skill level, and current win rate, to be promoted.  Those values change with every game you play (albeit slightly in most cases), and so after every game you play, the equation's original prediction may become less and less accurate unless you can continue to update it.



The MMR Equation and Working Backwards

For completeness, consider again the Matchmaking Rating Equation in its unsimplified form:


P = Rank Points
U = Unspent Bonus Pool
B = Total Bonus Pool
W = Win Percentage
M = Magic Number
Pp = Rank Point Differential
C = League Conversion Constant
K = Division Modifier
MMR = P + U*W*2 - B + K + C

It should be pretty obvious to most individuals with an inkling into statistics and rating systems that absolutely none of these variables is a factor in Blizzard's hidden MMR formula--not a single one.

Blizzard's method involves what is likely some pretty hairy mathematics to determine your MMR.  How can it possibly follow that this equation has any relevance whatsoever?

Consider the following poorly drawn picture analogy:


I spent too much time in ms paint.
Basically, Blizzard's system does a lot of work to output a number, your MMR.  Then, in order to make the game fun for everyone who doesn't care about the MMR, rather than leave it naked, they give it sunglasses (division constant), a hat (league constant), and a trenchcoat (bonus pool).

We can arrive at MMR in two ways: either derive Blizzard's function, or try to guess at the simple math.  Since Blizzard gives us 20,000,000 outputs grouped by six leagues and subgrouped by divisions of up to 100; finding these constants is not a complex endeavor.

It isn't coincidence that this works very well and accurately for most players.  I promise.



The Bonus Pool

Of minor note, someone was kind enough to show me that sc2ranks.com has its own calculator that determines bonus pool at any given time.  If you are having trouble looking this up, you can use this link (and hopefully Shadow doesn't mind, but as his site is overloaded as it is, I recommend either paying him to use this link, or just looking the number up by finding a team that has only played 5 placement matches on your server):

view-source:http://sc2ranks.com/api/bonus/pool?appKey=asdf 
(this works in the latest version of Google Chrome; no idea if it works in other browsers or those few of you who were brave enough to actually try and read this blog on a mobile device.)

The Bonus Pool accrues at a rate of 1 point per 112 minutes, or about 90 points a week.  It is about the same for all servers.  I thought it was the same for all servers, but it turns out it's only the same (half the time) for the three I play on.



Win Percentage as an Estimator

First, you must factor out the Bonus Pool in order to work backwards because it contaminates your displayed points.  But simply subtracting it out will not reveal your MMR for players with fewer games played because your displayed rating simply hasn't had the opportunity to catch up.

To work around this, the Matchmaking Equation, among others use the Bonus Pool to estimate your displayed points, should you play a lot more games in case you haven't yet.

This means the Bonus Pool serves two functions for us:  first, it skews your displayed points, and second, since it is already in the equation for every player, we can use it to estimate a higher number of games played for all less active players.

Using your current Win Percentage is the the best, most easily accessible means to modify the Bonus Pool to assume a win percentage that isn't 50%.  If you're sure that your current win percentage isn't an accurate reflection of your current skill level, you're free to put whatever number you like in there.

You can estimate a simple moving average of your win percentage by using your last several games (however many you want) played if you want.  I chose to use current win percentage because it is necessarily more accurate than assuming 50% for everyone, and it is the most easily attainable number without any tedious grunt work.



Calculating Your Displayed Matchmaking Rating

When you load a quick match, the matchmaking system will compare your displayed MMR to your opponent's hidden MMR.  If your displayed MMR equals his hidden MMR (within some small range yet to be determined), you will see "even match."

If you are curious about how to calculate your displayed MMR, you can use the following very simple equation:

P = Displayed Rank Points
U = Unspent Bonus Pool
B = Total Bonus Pool
C = League Conversion Constant
K = Division Tier Constant
dMMR = Displayed Matchmaking Rating (this is what we are calculating)

Current League Conversion Constant:
Bronze - 0 or 500
Silver - 1000
Gold - 1250
Platinum - 1500
Diamond - 1750
Master - 2225

dMMR = P + U - B + C + K

For players in the most common division tier (Rank 1), K=0, simplifying this further.  Note that while the Matchmaking Rating Equation estimates your MMR by spending all your Bonus Pool, this merely factors it out and tells you your current displayed rating.

If your dMMR is much lower than your hidden MMR, then you will see higher gains for a win (and lower points for a loss).



Other Questions and Concerns

I'll update this post as I can when I find more information worth posting here.



Closing Comments

Hopefully this is helpful and cleared some stuff up.  As always, I'm more than happy to discuss anything and everything I know with you, should you so desire.

Lastly, there are a few people currently working on methods (web calculator, spreadsheet, data-mining tool) to compile more data in aggregate and help you to use your more easily use your own data to understand where you are relative to other players, in case you are having difficulty.

I'll try to post links or other relevant information when they are finished.

No comments:

Post a Comment