Predicting the Eleague Major: Boston 2018, a 1 million iteration Monte Carlo simulation

 in Categories CSGO, Majors, Predictions, Tournaments

Now TyLoo have dropped out of the Tournament the tournament probabilities have been recalculated here Not much has changed so the writeup below is still relevant. Consider exchanging any mentions of TyLoo with Flash Gaming as our algorithm has them very evenly matched

For those of you unfamiliar with the process for these predictions, I use insight.gg’s unique Glicko team rankings to seed a Monte Carlo simulation of the upcoming tournament, then run a million simulations of the tournament and collect the finishing places of each team and how often they finished in them. From this we can then build a picture of the most likely performances of teams in the tournament.

When I did this exercise for the ECS season 4 finals this worked out pretty well. We picked the winner, expected mousesports to do well, picked all the qualifiers from the group stage (bad news for NA), expected Astralis to perform more strongly, and predicted 10 out of 13 matches correctly.

This time around the tournament is far larger and takes place in multiple steps so I’m going to look at things stage by stage.

Challenger Stage

This stage is a swiss tournament for teams that don’t have legend status, and the goal is to qualify for a second swiss tournament that does include teams with legend status.

A lot of the lesser teams will be weeded out here, but as it’s a series of Bo1 games there’s a good chance of some upsets happening. Let’s take a look at the chances of qualifying.

No real surprises at the top as FaZe are obviously one of the favourites for the whole thing, and mousesports have recently performed well at the ECS finals. EnVyUs are the biggest name we think will struggle to make it only having a coin flip. Turkish outfit Space Soldiers are the best bet for an  unexpected breakthrough.

At least one North American team should make it through, Cloud9 stand an excellent chance and Team Liquid are also strongly favoured, while the hybrid teams Renegades and Misfits also have a good chance to advance. There should be one, probably two and an outside chance of 3.

The non-western teams are clearly expected to end their participation here, but with several of them on a 10% or better chance one surprise package isn’t that unlikely overall.

Legends Stage

When we run the simulation different teams make it through to this stage each time, so regardless of their predicted chance to advance above they will definitely have a potential presence in this stage as well.

Let’s look at the each team’s chance of qualifying from the Legend’s stage to the quarter finals part of the tournament

Obviously this stage is where the big boys join the fray and things get a lot tougher. This is where the advantage of starting in this stage by default becomes concrete. For example since their major win and Zeus’s departure Gambit have been very quiet, rated at 2411 nearly 30 ranking points behind Cloud9 on 2440. But because Cloud9 face the risk of being upset in the first stage their overall chance of getting through this stage is only around 52%, while Gambit’s is 62%.

Of those teams qualified for the Legends stage by right BIG are the most vulnerable only advancing further 19% of the time. 100 Thieves are only on 40%, but their players have been inactive for some time and their current ranking was taken at the end of a series of bad results so their true strength may be difficult to predict.

This is also the first time the Virtus Pro problem presents itself. VP are the team that will spend 90% of the season doing OK in mid level tournaments and then make the final of every Valve major on the calendar* automatically qualifying themselves for the next time. They are the ranking algorithm’s nightmare. We’re giving them a 50% chance of making it through and if they do we didn’t really get it that right or that wrong.

Fnatic also look a bit shaky here. They performed less well than mousesports overall in ECS and although we expect them to make it through they’re vulnerable to being picked off by an unexpected surge by a dangerous team like mousesports or 100 Thieves, or maybe a resurgent NaVi.

Speaking of whom we haven’t seen much of them since winning Dreamhack winter. Will they confound the rating system by continuing the improvement they showed there as well as a strategy bag stuffed full of secrets cooked up in the NaVi lab? Time will tell, we suspect they will flop but they are potentially dangerous to the favoured teams.

From the perspective of the NA teams things start to look a bit sketchy, should Cloud9 fail to qualify for the champions stage there are no other front runners. Liquid only really stand a chance if they choose this moment to unveil a brand new and more successful approach to their T sides, and the other hybrid NA outfits are really looking to roll very high to make it through.

*Joke, not to be taken literally

Champions Stage

The final knockout stage becomes more polarised, the Bo3 format means that results will be more consistent with a team’s true strength and that the chance for an upset maps to unexpectedly catapult a team deep into this stage just aren’t there any more. Weak will be seeded against the strong, it’s time to sort the men from the boys.

One of the things about a Monte Carlo simulation is that if you just run them enough times it makes it seem that anything is possible, but a million rolls of the dice aren’t enough for the 3 weakest teams in the tournament – there just isn’t a universe in which Quantum Bellator Fire can win the big one.

Also examine the bottom axis, no team has more than 20% chance of taking the whole tournament if they even get this far and we’re looking at some pretty restricted ranges of probability. Even so everybody’s pre-tournament favourites are the standout pairing. But now FaZe are favoured out of the two, when SK was favoured in the Legends Stage. The reason is that the algorithm currently rates FaZe higher and once we get into the Bo3 stage that rating asserts itself more strongly. Also the chance of SK being surprisingly dumped out of the group stage is now watering down their chances as it did for FaZe in the first round.

SK may be the stronger of the two teams, but the algorithm likes FaZe’s activity level and SK have a habit of loafing a bit in group stages and dropping the odd map to a lesser team. That prevents their big wins being embedded into their rating as fully as they might be.

Astralis are the only other team given more than  10% chance of taking home the big prize, a lot of which will depend on the team continuing to get good production from their stand in for dev1ce. G2 also lurk either as a danger to the top teams or as a potential embarrassment to the ratings or even both. They have been inactive for some time and it remains to be seen how relevant their rating still is. If you want to look like an expert prognosticator and don’t mind taking a risk you could choose a worse dark horse.

Cloud9 are the only NA representation that has a realistic chance to win, but even then it’s very marginal.

Finishing Placement Chances

TeamQuarter FinalistSemi FinalistRunner UpWinner
FaZe Clan24.1%16.6%11.1%17.6%
SK Gaming26.2%18.3%11.5%16.4%
Astralis28.7%17.8%10%10.9%
G225.4%15.4%8.8%9.7%
North28.4%16.6%8.7%8.4%
Cloud924.4%14.1%7.6%7.4%
Gambit Esports29.8%16.3%8.2%7.2%
mousesports23.5%13.1%7%6.6%
Fnatic27.6%14.3%6.7%5.3%
Virtus Pro29.7%12.8%5.1%3%
NaVi21.1%9.2%3.8%2.3%
Liquid18.4%7.6%2.9%1.6%
100 Thieves25.2%9.2%3.1%1.4%
Renegades13%4.2%1.3%0.4%
EnVyUs11.1%3.6%1.1%0.4%
Misfits12.4%3.8%1.1%0.4%
Space Soldiers8.2%2.2%0.5%0.1%
BIG15%3.2%0.6%0.1%
FlipSi3 Tactics2.4%0.3%0.03%0%
Vega Squadron1.8%0.2%0.02%0%
TyLoo1.1%0.1%0%0%
AVANGAR0.4%0.02%0%0%
Sprout0.5%0.03%0%0%
Quantum Bellator Fire0.4%0.02%0%0%

No team is rated as more likely to win than get knocked out in the playoffs, and no team is more favoured to win than be knocked out in the quarterfinal (although SK and FaZe are more likely to get to the final than be knocked out in the QF). The format is too long and tough and the competition refined so effectively that the competition is much tougher at this stage.

Looking at the Winner vs Runner Up chances, the top 4 of FaZe, SK, Astralis and G2 have on average faced opponents they were favoured against, although obviously in the case of Astralis and G2 it was far more marginal, but no other team is likely to be favoured in the final should they get there.

As well as those 4 teams, mousesports, Gambit, Cloud9 and North are the only other teams more likely than not to advance to the semifinals or better if they get this far.

G2 would probably be the biggest surprise winner for most people given their patchy record in tournament play. Their rating is really based off not playing for quite a while, a lot of their rating points were earned at a time when the top teams had a big advantage over the field and that has narrowed with time. It’s fair to expect they might not be able to live up to this standard and I have sympathy with the view they are overrated here, but it can also be dangerous to go against the wisdom of the algorithm and the raw fragging capabilities of KennyS.

100 Thieves are a similar unknown quantity, their rating is also old and comes from a period of greater rating disparity in the scene, but at the same time they can play to a high level and the chaos that surrounded them when they left Immortals contributed to a sharp rating decline before the split. Perhaps they will in fact be stronger than expected.

Prediction confidence

Are the predictions as likely to be as successful as at the ECS season 4 finals? The two tournaments are very different. At ECS there was a small field, clear splits in quality and strong favourites, as well as the guaranteed early elimination of a potential winner that conversely made later predictions easier.

At this tournament the field is far wider, the system is designed to be more competitive and keep stronger teams in and there are multiple elimination stages. All of these things make it much harder to predict and this is reflected in less decisive percentages. Even the favourite wins the tournament less than 1 time in every 5 simulations.

Added to this there is additional uncertainty. Teams will have been hiding their true strength in the run up to the tournament, saving key strategies, developing new map approaches, possibly updating their map pools. Surprise is potentially worth a lot and can temporarily inflate a team’s effective ranking a great deal, but it will also throw off our algorithmically generated predictions.

Also considering the lack of activity of some strongly rated teams and there is certainly plenty of potential for surprises. It will be an interesting tournament to compare to our predictions, and probably the toughest test on the calendar.

Ultimately this exercise deals in probabilities and sometimes there will be upsets and shocks. Assessing the accuracy of our predictions doesn’t depend on avoiding these entirely, but on just how many of these outliers there are and whether that represents more than we would reasonably expect.

On a technical point it’s not entirely clear exactly how the seeding will work in some respects so we’ve had to make some judgement calls on that. These factors can be important when it comes to the final result so we may have biases in our model that will work out differently in the tournament itself.