Make AI better

[tc85123]

Many times the AI plays a bad card when it shouldn't or attacks and defends when it shouldn't. There are just a lot of general things the AI does that is bad... Is there anyway we can create a button to hit or something to make the AI take back the dumb move it just did and for the AI to not be able to replicate that move again on that turn... The AI will be forced to do something else other than make the mistake it just made.. This would help a lot as it really ruins games.

I noticed we have the back button but if the AI does something dumb and I push the back button the AI might do the same move 5 times before it chooses to do something else...

[PalladiaMors]

What AI are you using? Notice that there can be a significant difference in skill level among the various available AIs. According to the wiki, Monte Carlo Tree Search at level 4 is the currently strongest available setting for the AI, so try setting yours that way and it should improve a bit.

Also, notice that the AI doesn't play every type of deck equally well. It has difficulty playing combo decks, since those require understanding the deck strategy as a whole, which right now I believe it can't do. Looking at the results from Firemind, you can see that Magarena's AI is better at playing aggressive decks. Try to give it one of the top decks from Firemind and the challenge should increase a lot, since it will be playing a deck that it "understands" well.

For example, try to give it one of these decks (you can download the .dec from the site):

https://www.firemind.ch/deck_lists/537 (Modern format - play a Modern deck as well in order to keep things balanced)

https://www.firemind.ch/deck_lists/133 (Legacy format)

These should increase the level of challenge quite significantly, as long as you play a deck of similar strength against them - obviously taking a Vintage deck against these won't be fair. In the end, you have to realize that the AI is not near human-level strength yet. Magarena is a few years old, it took several decades to get computers to play chess at master strength and there was a *lot* of people working on that, people have been trying to do strong Poker bots for a while but nothing decent has come up yet... We've got to be a bit patient, Magic has very complex rules in comparison to those two examples and there's not nearly as many people working on a MtG AI, this isn't something that's gonna happen overnight.

[Lodici]

For example, try to give it one of these decks (you can download the .dec from the site):

https://www.firemind.ch/deck_lists/537 (Modern format - play a Modern deck as well in order to keep things balanced)

https://www.firemind.ch/deck_lists/133 (Legacy format)

No need to download - these are available by selecting the "Firemind Top Decks" option in the deck chooser (assuming you are running the latest version of Magarena).

[Lodici]

Many times the AI plays a bad card when it shouldn't or attacks and defends when it shouldn't. There are just a lot of general things the AI does that is bad... Is there anyway we can create a button to hit or something to make the AI take back the dumb move it just did and for the AI to not be able to replicate that move again on that turn... The AI will be forced to do something else other than make the mistake it just made.. This would help a lot as it really ruins games.

I noticed we have the back button but if the AI does something dumb and I push the back button the AI might do the same move 5 times before it chooses to do something else...

You should try all three types of AI - for example, up until recently I have been using Monty Carlo because everyone says this is the best but I found it quite often prone to making really stupid moves. I have since switched to Les Vegas and it seems more aggressive and I seem to be losing more often (is that good or bad?). Note, the AI level is actually the maximum thinking time in seconds that the AI has to make its move.

It would be really interesting (if it is even possible) if you could indicate to the AI that "no, that is not a good move, why don't you try again" like it was a child or beginner and it would somehow remember not to do that next time. But that is getting into the realms of self-learning - perhaps melvin can knock something up before the next release!

[tc85123]

Thanks for the advice... In a perfect world we push a button tell the AI no that's a bad move and the AI learns from it... However that might be impossible... I'm no programmer but I would assume it easier to just tell the AI no independently on this turn for this move and cut off the option for the AI to make that move again until next turn ... it doesn't learn from it, it might make the same dumb move later in the match or another match completely but at least pushing the button would just take away that dumb move and force the AI to make a better move.

I think this is the simplest way to fix little problems like this for now... So I hope something can be done.

[mike]

I have been playing around with the idea of providing the AI with some heuristics based on game state data.
As I understand it the AI curretly scores only individual cards and known game outcomes (win/loss). If you look at real world magic a lot of quick decisions can be made based on the concepts of value and tempo. What if the parameters for such heuristics could be compiled from player ranked board states (in addition to "learned" board states based on the outcome of previous games?

It's just an idea for now but I'd like to start this by offering firemind.ch as a central database where "this is bad"-feedback like you proposed could be stored. And while we're at it: Why not store the player and AI's taken actions/resulting game state at the end of the duel too?

[ShawnieBoy]

This is all really good

I've still got a mental image of slapping the AI on the wrist with a ruler, "No, bad AI!"

Negative reinforcement FTW

[muppet]

If you are going to alter the ai I am fairly sure the best thing to change is the evaluation method. As I understand it currently life total is used as a measure of success. This might account for the suicidal blocks to preserve life far too early in the game e.g. ai casts a bird of paradise and then blocks pretty much anything the next turn. where clearly the one or two life saved are not worth the bird. I can't account for the suicidal attacks in simple positions admittedly e.g. ai casts a 1/1 player casts a 2/2 ai sometimes attacks with the 1/1 into the 2/2 .
Anyway I think an eval function that calculates the state of play is the way forward even if its as simple as cards in hand, lands in play plus power/toughness of creatures (for both sides). It might be a good idea to even have this function be deck specific e.g. reanimation wants creatures in gy but that could come later.
currently playing the ai with it using firemind decks mostly it has about a 7% win rate which it shd be possible to improve. Its current standard of play really makes trying to determine "good" decks nothing more than a weird sub game, and I think it shd be possible to make it at least slightly relevant to the real world. It took chess a long time to be better than people, and bridge is now roughly on a par with all but the best in the world.

[VladVoivode]

I suppose I am quite stupid then as I find the AI, especially Monte Carlo - almost "human" and a smart one at that.

[muppet]

I do have cheating disabled if that makes a diff.

[muppet]

ok so I started a game and thought I wld just report the first non smart thing it did.

cast Goblin Rabblemaster in its second main phase. This is 99.9% of the time bad and was in the case involved.

[VladVoivode]

ok so I started a game and thought I wld just report the first non smart thing it did.

cast Goblin Rabblemaster in its second main phase. This is 99.9% of the time bad and was in the case involved.

Let's face it, AI isn't perfect and even if it was, humans make mistakes and we can learn from them. AI doesn't have memory as we do and without long term memory and adaptive learning through experience, even the BEST AI will make mistakes. If you read some articles about IBM's Deep Blue, it was/is a remarkable AI but it still lost to Bobby Fischer.

I don't mean to seem factious but, I think we are at least a century away from an AI that could learn through adaptation and possess what we would call long term memory. Who knows, we may even have Data by then. That said, I still contend that overall, Magarena's Les Vegas and Monte Carlo AIs are pretty good and I know from posts by others that I am not alone in thinking this.

Best,
Vlad

[jerichopumpkin]

ok so I started a game and thought I wld just report the first non smart thing it did.

cast Goblin Rabblemaster in its second main phase. This is 99.9% of the time bad and was in the case involved.

Guess this is an hyperbole, since even the most mindless goblin deck can face situation where simply yelling "CHARGE!!!" won't cut a path to victory...