The question of rating distribution in Shadowlands compared to previous expansions (in particular the final season of BfA) has created a lot of debate. Indeed, one of the reasons why our State of the Ladder posts and LadderDex tools have received so much coverage is arguably that the rating distribution is quite different to what it previously has been.
Nevertheless, getting an estimate of the rating distribution of previous seasons is difficult since we did not run our ladder sampling procedure in BfA. Thus, quantifying how different rating is this season compared to last is not straightforward. An alternative way to approach this is to use matchup data from our REFlex dataset. This dataset contains rating from players and their opponents for a large number of user-submitted REFlex files (REFlex is an addon that tracks and records the outcomes of arena games). This has a few drawbacks compared to a proper ladder-based analysis, but it should nevertheless provide a rough estimate of rating in SL compared to in BfA.
To ensure that this comparison is as fair as possible, we restrict ourselves to individuals who show up in both the BfA S4 and SL S1 datasets. When individuals had more than one game in either season, we calculate the median rating they had across all observations in that season. Most characters only show up once or twice in either season, and in total we are able to identify 12,615 characters that showed up in datasets from both seasons.
Note that the rating distribution is higher than on the overall ladder: this is to be expected since people submitting their REFlex files to our website are very likely to be far more serious about PvP than the average player. Nevertheless, what is interesting here is the difference in rating between the two seasons. These distributions contain the same exact set of characters; the only difference is which season the rating was sampled from. For BfA S4, the mean rating is 2000. For SL S1, the mean rating is 1748. Thus, on average, we see that rating is on average around 250 rating lower in SL S1 compared to BfA S4. Note that this is likely a very conservative estimate since people who played in both seasons are likely to have at least improved a little bit compared to the average player. Thus, it may be that the actual difference between the two seasons is closer to 300 or more. In both seasons, the observed standard deviation of the rating distribution in our dataset is about the same (around 275).
Below, we're showing the character-wise rating difference between Shadowlands S1 and BfA S4. A value of 200 on the x-axis means that a character had 200 rating higher in BfA S4 than they did in SL S1. 0 would mean no change, and -100 means that the character has higher rating in SL than they did in BfA.
Looking at these data, we find that most characters (84%) had higher rating in BfA S4 than they do in SL S1, and we can see that the average rating difference between the seasons is approximately 250.
Recently, Blizzard reduced the rating requirement for the Vicious War Spider from 1400 to 1000. This mount is intended as a reward for participating in rated PvP. This massive 400 rating change difference is very likely informed by the dramatic change in the rating distribution that we have seen going from BfA to Shadowlands. This rating reduction is higher than our decidedly conservative estimate of 250 rating, but could well reflect the real rating difference between the seasons.