Hidden MMR in Solo Shuffle?

Solo Shuffle is currently released as a brawl, meaning that there is no explicit viewable rating attached to the mode. Most brawls, however, have no hidden MMR either: if you queue up for Gravity Lapse (the Eye the Storm BG where players regularly get shot up into the air), it functions just like a random BG with no consideration of your ability or game history. Skirmishes, at least low-level ones, allegedly have some hidden MMR, but what about Solo Shuffle?

There are at least two ways that a hidden MMR would manifest itself in a manner that is observable to us as players.

1. Differing queue times

If a hidden rating operates in the matchmaking process, we would expect to see different queue times for characters that have played loads of games (and thus have an established rating) and those who have played no games (and who are still playing placement games). Specifically, characters that have done very well in solo queue (and should have a high MMR) should have a very high queue time compared to characters that have not.

In addition to the supposed presence of rated (low-level) skirmish, the thing that spurred my interest in this question was that I was seeing different queue times for Solo Shuffle on different characters, with my main notably having longer Solo Shuffle queue than my alt. This is exactly what we would expect to observe if there was a hidden MMR operating (since the higher MMR pool would be smaller leading to longer queues).

For the following screenshot, I first queued my alt with 2 Shuffles played (5 minute average queue), then my main with ~20 Shuffles played (7 minute average queue), and lastly another alt with 0 Shuffles played (5 minute average queue). The fact that both alts have the same average queue when they are both in principle in the same pool (i.e. in the "placement game" pool) is very interesting. Note that all these characters were queued within minutes of each other, had the same roles (DPS), and were all queued on EU Alliance.

2. Different player pools

If there is no MMR, the 6 players in any game of Solo Shuffle should effectively be drawn from the same general population. If, on the other hand, there is hidden MMR, we would expect that players at different MMRs see different player pools. A simple way to test this is to compare the average equipped ilvl of players for one character who has played many games of Solo Shuffle (and given a modicum of success should have high MMR) to that of a character who hasn't played many games of Solo Shuffle (and thus should be playing placement games).

In order to test this, I played 10 Solo Shuffle matches spread across three of my alts. I calculated the average ilvl for each of the characters (excluding my own), which yielded a sample of 50 characters (5 characters per match). Of these, 48 were unique (i.e. two players were encountered in two separate Shuffles). I also played 7 matches on my main, which yielded a sample of 35 characters, all of which were unique. None of the alts had played more than 10 Solo Shuffle matches, and the main had played at least 15 at the time of the first game with a 75% win rate. In principle, if there is hidden MMR, we would expect that the ilvl distribution should be significantly higher for the non-placement games where I queued my main, compared to the placement games where I queue d my alts. A consistent 75% win rate over 15 games would have increased my MMR by over 300 rating above the starting point.

Below, I am showing the average equipped ilvl distribution of the characters that I encountered on my alts (placement) and main (non-placement). These distributions are incredibly similar, with means of 251.4 and 252.9 for the placement and non-placement games, respectively. Statistically, there is nothing that suggests that these two independent samples are different (P=0.39, Mann-Whitney U).

An important observation is that one of the players that I was playing with was in the same match on both my main and my alt. On its own this isn't necessarily be problematic since the player could simply have graduated from placement to non-placement games. However, I encountered this player first on my main (during supposed non-placement games) and subsequently on my alt (during supposed placement games). This would never happen in a normal rated arena match (i.e. you would not "regress" back to placement games). A final piece of evidence against an MMR is that both my win rate and that of a number of other people I have spoken to has not changed as we've played over 20 Solo Shuffle games.

I have to say that this was a bit of a rollercoaster, since I started off this journey believing that there must be a hidden MMR owing to the queue discrepancy I showed at the beginning of the article. However, the data does clearly show that there is no ilvl disparity between "placement" and "non-placement" games, and anecdotally, I also perceived that many of the players that was in the Shuffle when playing my main would not have been able to sustain a 75% win rate (or anything close to it) for ~15 games. The elephant in the room is then what's causing the queue disparity, and to that I really don't have a good answer. But having now played a whole bunch of Solo Shuffles on alts and on my main, I definitely get the sense that I am playing against the same population of people on both my main and my alts and the ilvl data does seem to back that up. It's also worth noting that even after all these games (which should have pushed my MMR high up), I still maintain the same or higher win rate as before I ran this experiment.

If you have any questions or comments about this article or anything WoW PvP and data, the best way to get in touch with me is to pop by my Discord.