In PvP, gear matters. There is no denying that. If you are facing off against a team whose average ilvl is 50 above your own, you are definitely going to have a hard time, even if you massively outskill them. But "how much" gear matters is a pretty controversial issue and hard to answer without looking at data. A complicating factor is that the importance of gear (and in particular ilvl) has changed with time. In this article, I'll try to show exactly how much gear matters in PvP in Season 1 of the Shadowlands expansion.
We are going to be looking at two populations of interest: the general population of WoW PvPers and PvPers at above 2000 rating.
Let's first take a look at a cohort of randomly sampled characters from the general WoW population. I will be constraining the analysis to characters that have played at least 50 games in 2v2. In my sample of just over 40,000 characters, this yielded 5943 characters. Below, I am showing the average ilvl against 2v2 rating. Each dot in the plot corresponds to an individual character.
What we observe is a moderate correlation between average ilvl and 2v2 rating. If gear was the only thing that mattered, all the points would fall snugly on a diagonal line: the better gear you had, the higher rating you would have. On average, for each increase in ilvl, you gain around 11 rating, but we also see a very substantial scatter, meaning that a high ilvl does not on its own guarantee high rating. Nevertheless, there is a clear and statistically significant correlation between the two. Two things are worth noting about this plot. First, there are very few people at high rating with bad gear (e.g. lower than 180 ilvl). The implication is that it is very hard to get high rating with bad gear, i.e. gear is probably a necessary requirement for high rating. The second observation is that just because you have good gear does not mean that you have high rating. In other words, gear is necessary, but not sufficient for high rating. Overall, a character's average ilvl can explain 20% of the character-to-character variability in rating in the general 2v2 population.
This is informative, but hardly surprising stuff, so let's move on to something that is surprising. If we look at the top of the 2v2 ladder (players above 2000 rating) and repeat the same analysis, we observe a very different relationship.
The effect of ilvl has all but disappeared. Surprisingly, whether you have 190 ilvl or 210 ilvl is essentially not predictive of your rating above 2000 rating. At above 2000 rating, each ilvl increase only affords, on average, 1.4 rating, and average ilvl can only account for around 1% of the character-to-character variability in rating. While this does come out as statistically significant, the effect of gear is clearly incredibly diminished at these ratings. The data I have shown has been for 2v2, but everything is pretty much exactly the same in 3v3.
Before we get carried away, let's be clear about something: if you were to put these players in terrible gear (e.g. 160 ilvl), they would undoubtedly get smashed at high rating. However, the data suggests that once you get to around 190 ilvl, the effect of gear is actually very modest. It's not immediately obvious why the effect of gear should be smaller at higher ratings since there isn't any diminishing returns associated with ilvl (there is diminishing returns on secondary stats above 30%, but very few characters currently experience this). One possibility is that it's not the effect of gear per se that is weaker, but instead the effect of other variables, such as team composition, that is stronger at higher rating compared to at lower rating. This is supported by the fact that we generally see a more constrained representation of classes at the top of the ladder compared to the middle. It also agrees with a commonly held idea that comps matter more at high rating: you can get 1800 rating as sub rogue/unholy DK in 2v2, but you'll probably struggle to get 2200. If comps matter more at high rating, something else has to matter less. Another possibility is that there is a negative correlation between player skill and ilvl at high rating (high skill means, on average, worse gear). We can't really rule this out based on the data alone, but it is a less parsimonious explanation than the first and arguably less plausible.
If we wanted to be proper scientists and really to say something causal about the impact of gear on rating, we would really have to run experiments (e.g. we might want to change one team's ilvl and observe the effect). Nevertheless, the absence of a strong correlation strongly suggests that it has a very minimal impact on game outcomes at higher ratings.
So there you have it. The aforementioned caveats aside, gear appears to matter a fair bit at lower rating, but once you get decent gear at high rating, other factors such as comp and player skill account for essentially all of the rating differences among characters.