Draftsim’s recent study on MTG draft bots is one of the most fascinating things I’ve ever read. Drafting is something that pretty much all card game players love, regardless of the game we’re playing. Drafting in MTG Arena is different than say, Hearthstone though. Why is that? Hearthstone is a single-player draft. There’s no “hidden information”. You don’t have to worry about someone hate-drafting a card or another player picking up a card you’re after, but it isn’t a top priority. In that, MTG Arena’s drafting is very similar to real-life drafting. Draftsim itself has a Draft Simulator, which is where I first found the site. They have some interesting resources as far as card game growth goes.

This may be the first peer-reviewed academic paper about MTG draft bots – at least that we’re familiar with. Quite frankly, I’m pretty bad at drafting, but I love doing it casually. I feel like I lean towards strategies that don’t work quite as well in draft/sealed environments. That’s okay though! So what was the goal of this academic paper?

This MTG draft bots project wasn’t about making the best deck possible. Instead, it’s all about mimicking human behavior and I think they captured this pretty expertly. There are all kinds of draft players, that’s for sure. Some people only hunt rares (looking to flip them for money), some people kind of pick at random, and go from there. The goal is to create these more human-like draft experiences, and to hopefully reduce “bad experiences, like playing against mill in every single match.” Amen to that.

The data for this study came from 107,949 user Core Set 2019 drafts, through Draftsim’s draft simulator. Some parts of this were used to train a pair of the bots: BayesBot and the NNBot. The rest was used to test the bots against each other, which is pretty interesting. Who were the competitors though? There are five total bots:

Baseline:

• RandomBot: Picks random cards.
• RaredraftBot: Slightly better, it pricks the rarest card in the pack. Ties pick the card you have the most of, colorwise.

Complex:

• DraftsimBot: The bot used for Draftsim draft simulator. An expert-tuned agent, it estimates card strength and color matching.
• BayesBot: Uses probability and something called “log likelihood” of human user picking given cards from a pack. Definitely beyond my comprehension.
• NNetBot: A “deep neural network agent, trained to predict human choices using a machine learning model.

The 107,949 human drafts across simulated packs of 265 unique cards split 82/20 were used for the testing/training process. The bots themselves are pretty complex, so you can read further details on them here. We’ve discussed something potentially similar to “deep neural networking” and machine learning before, thanks to IBM. The NNetBot and Bayes Bot are both pretty complex and are perhaps a bit more complex than Draftsim. The goal is to ultimately make this draft simulating and AI drafting better. If it can correctly imitate human behavior, we can have intense, challenging drafts with bots instead of always needing people.

We can guarantee without even reading the whole article that Random and Raredraft bots were both completely outclassed. All they do is pick on a very simple basis. The winner, according to Draftsim was NNetBot. Neural Networks are incredibly good at this kind of behavior, and it’s been used across a variety of games. As DraftSim points out, Chess, Shogi, and Go have all used a similar work – DeepMind, by AlphaZero.

NNetBot defeated the other bots handily but only had about 40% accuracy. This is because we as people don’t even really pick in drafts the same way as other people. Draftsim points out that people have a hard time picking the medium-strong cards when it gets to the middle of the pack. So that means pack position matters. If you’re the first pick of a pack, you have the best chance to pick the most powerful card.

Once you get towards the middle of a pack, after a few turns have passed, you’ve likely just got a few commons/uncommons left. It becomes very hard to determine what you should pick. The final cards tend to be the least playable. At that point, as Draftsim said, it’s easier to predict what’s picked at that point. But what do we learn from all of this? To me, it’s far more than simply having “better MTG draft bots”, but that is awesome. Some people want to only play against the AI but want a challenge. There’s no shame or problem with that.

What I like about this, is it will help players get better at drafting. You can keep an eye on what’s being picked across specific expansions and deck archetypes. Even if you aren’t making the best decisions, you can see what these incredibly smart AI are doing, and it can help you as the player make more informed decisions. It’s something I’m excited for, as someone who is mediocre at best at drafting. You can read the full, technical piece right here as well if you’re interested.