State of competitive AI

[excessum]

This post is a brief summary of the current state of the AI playing competitive decks for Standard mainly. I am writing this mainly to help organise my thoughts and hopefully, get some feedback or help from the rest of the community.

Firstly, I am only addressing the AI playing Standard and Limited decks since any meta-game with significant numbers of combo decks is one where the AI is hopeless. For starters, the AI should be given post-sideboarded decks to make up for the fact that the human gets to sideboard from game 2 onwards while the AI is stuck with the same deck.

Things the AI is bad at or cannot do:
1. make sequences of decisions
2. optimise land drops (largely relevant for the current Standard with fetches and battle-lands)
3. mulligan (spells + lands, no cheap threats, no interaction, etc.)
4. play control (what to counter, when to sweep, play defence)

The AI is at its best playing classic mid-range decks where the curve is balanced and the best course is usually to play the most expensive card available. Examples in recent years are Mono-Black Control from Theros, Abzan/Jeskai from KTK and BFZ. In most of these cases, it is usually good enough to grab winning deck-lists and replace AI-unsuitable cards to have a decent match.

The AI is largely competent with more aggressive decks like Mono-Blue, Abzan Aggro and BR Aggro since the plan of playing threats and going for the win is still valid. Points 2 and 3 above are more problematic here since aggressive decks are more reliant on closing the game quickly.

The AI is usually bad with control decks since the nuances of card advantage are lost on it. It might win if it is lucky with having the right spells at the right time or if control cards are exceptionally powerful (eg. Supreme Verdict + Sphinx's Revelation). However, most games end with the AI defeating threat after threat but finally failing to stop that last threat.

Future plans:
Point 3 might be solved by implementing a generic AI strategy controller that considers the deck being played. My current idea is to consider creatures and planeswalkers as "threats" and force a mulligan if there are insufficient threats below CMC 4. This controller can then be expanded later to incorporate better decisions with scry/filter/discard effects and maybe tackle point 4 eventually.

[reporter]

You mentioned elsewhere that:

Many static effects (I think Winter Orb is one of them) are not considered in the AI code at all since it would mean inserting "if-else" blocks all over the place to support a single card or effect.

I think having these sort of if-elses is actually the optimal solution, as it has minimal performance hit and it models the problem exactly (if there are multiple cards with similar effects, they could map to certain restrictions that are then evaluated instead). Having the problem modelled accurately doesn't equal simple code, but how messy it is depends a bit on architecture I assume. The only other approach I can see is brute force, but it has to be limited and preferably optimized or used with heuristics..

On AI development: Who develops it? Who designed the current one? Is the design sketched anywhere other than in code? Are people trying to improve the current one or rewrite it? Do people discuss (or agree on) what the design should be anywhere?

I think the best way to organize your thoughts regarding AI design is to play the game and write up each human decision moment and then classify them later. From your list mulligan is a relatively clear decision example, but play control is too ambiguous for design.

[excessum]

At the rate at which new effects are appearing, the performance hit is not negligible. The bigger concerns are that they clutter the code unnecessarily for developers and that interactions between multiple effects are not handled. If there are multiple cards with similar effects (and someone bothers to code the AI for it), they might be coded (eg. cannot gain life).

For the AI development questions, it was decided long ago and subsequent developers just added on bits and pieces as with all open-source projects. Personally, I have zero interest in fixing the long list of issues in the AI threads since they concern ancient cards in "fun" decks where it is impossible to predict how the player can break the AI decisions. Besides, any competitive deck in the eternal formats must involve knowledge of the meta-game or be meaningless (ie. what must be countered, named for Pithing Needle, silver bullets to sideboard in like Rest in Peace) and there is absolutely no way for a coherent AI to be designed for such things.

I do know how the AI will behave in most decisions involving Standard/Limited decks or can check it up in the source code quickly enough. My problem is deciding how to improve it without overhauling the entire system and introducing new bugs. There has been several developers that attempted to do such an overhaul over the years but they are still incomplete.