Avoid AI repeating actions with no effect

[sponeta]

So I'm not really sure how to approach this but one of the biggest issues with the AI to me seems to be repeating actions, especially ones with bad consequences.

Often it's giving a creature an ability it already has. For example, it will keep activating Viscid Lemures ability multiple times even though it already has swampwalk. So the AI just keeps reducing the creature's power until it's useless in combat. Not to mention that I'm not even controlling any swamps, but that's specific to the card really.

Other examples:
- Rise of the Hobgoblins ability many times in a row.
- Regenerate a creature outside of combat when there's nothing dangerous on the stack.
- Cast Flight on a creature that already has flying permanently. I know casting Flight of Fancy on a creature with flying is actually beneficial because you also get to draw (it's just not the best use of the card), but Flight has no other benefits.

From what I've been reading I understand that you're not keen on denying actions from the AI because there might be a scenario where doing something counterintuitive is beneficial. Which is why the AI keeps changing equipment around. But adding an ability to a creature that already has it isn't beneficial in any other scenario than where there's something on the stack that's removing that ability.

The AI is also very keen on paying sacrifice costs for activated abilities, even when they're not useful. It's not uncommon to see it sacrifice all of its creatures to give Dross Hopper flying multiple times in one turn. Then when all other creatures have been sacrificed, it will also sacrifice Dross Hopper to itself. This might all happen during the turn when Dross Hopper comes into play so it has summoning sickness.

Is it possible to address this globally somehow, or would it mean tweaking every card manually?

Another example is anything with "firebreathing" (R: this guy gets +1/+0) like Furnace Whelp. I know this isn't quite the same situation as above, because the creature's power does increase with each pump, and a higher power is better than a lower one. It's just that having a higher power has no use in most situations.

Any time this kind of card is in play I know the duel will last much longer than usual, because the AI will spend every available opportunity on pumping the card. It's especially annoying when it happens after the combat phase, because there's no point to it. It would make sense if there was something in play or on the stack that made use of high power (like Wall of Reverence), but mostly it just slows the game down to a crawl.

This is especially bad with Wall of Fire, since it has defender and the number of situations where pumping it up are even more limited.

If this can't be fixed in a simple way, would it be possible to put repeated actions into a single AI decision where it first decides whether to use an ability, then decides how many times? Like in Duels of the Planeswalkers, the AI could tap 5 lands at once. This would at least reduce the amount of waiting.

[sponeta]

If this is something that doesn't have an easy fix, how about giving the AI an undo button? I mean the player gets one, why not the AI?

Whenever the AI makes a silly decision, I'd like to be able to go scroll back in the log, find the log entry where the decision was made, and click a button next to that says "That was dumb, try again". The game would undo back to that state, just as if I'd pressed my own undo button, and the AI would retry the decision, but the list of choices would exclude whatever it did last time.

The AI doesn't actually have to learn anything for the future. Although that would be cool. But it's enough if I could just get the AI to redo one decision at a time.

Of course you'd need to be able to go back to a sensible point, not just the latest decision. Like, if the AI decided to sacrifice a creature to Dross Hopper after combat, it's not enough to undo the target choice. The AI would just choose a different creature to sacrifice, until I'd clicked undo enough times that there were no creatures left. I'd need to go back two steps and tell the AI not to activate Dross Hopper's ability in the first place.

Might also need to figure out how to let the player pause the game in more places than now. Right now the AI can play an entire turn without player input if the player doesn't have any activatable abilities or instants to cast.

[Lodici]

If this is something that doesn't have an easy fix, how about giving the AI an undo button? I mean the player gets one, why not the AI?

See viewtopic.php?f=82&t=15969&hilit=stupid for a related discussion.

[sponeta]

Aw, and I was so stoked about the undo button idea too...

[Lodici]

I don't think any conclusion was reached in the discussion. The impression I get from our resident AI expert (@melvin) is that it would not work well with MCTS. Perhaps other AI types might be more amenable - @ubeefx might be able to help here. Utimately, I am firmly of the belief that where there's a will there's always a way.

[melvin]

I don't recall mentioning that it wouldn't work, it does work. My issue with this is that it a very complex solution that doesn't solve the underlying problem of why the decision is chosen in the first place. I'd much rather spend the same effort in making sure these moves are never or hardly even chosen in the first place.

The main issue I have is that there hasn't been a consistent way to replicate these bad actions, so we have not been able to diagnose the reason why this is occurring. If possible, please try to include a detailed setup of the game state or even better a screenshot so that we can replicate the problem. The game state saving feature would come in handly here, unfortunately it is only available in dev mode.

[sponeta]

Well, I wasn't really talking about the AI learning anything. I mean, sure, if it was possible, it'd be cool... but an AI that can learn is like a whole different beast.

As a player, I think I'd actually enjoy simply correcting the AI's actions now and then. It'd be a bit like playing with a novice! I see it more like a way to to avoid an unnecessarily easy win. Sure, it wouldn't make the AI better, but it would make the game more fun in those cases.

[melvin]

Well, I wasn't really talking about the AI learning anything. I mean, sure, if it was possible, it'd be cool... but an AI that can learn is like a whole different beast.

Neither was I, I was talking about telling the AI "do not do X this turn". The "this turn" part is necessary otherwise you'll potentially have to repeat this every time the AI has priority, which could be quite a lot of times in a turn for an instant effect.

Even then, you may have to repeat this every turn, while the bad move is possible. Say the suboptimal move is casting Flight on a creature with flying, as long as Flight is in hand and the creature with flying is on the battlefield, the AI may try to cast Flight on it. I think this quickly gets boring and tedious.

I'd much rather figure out WHY the AI is choosing such a move and then prevent it from happening. To do that, the first step is to be able to get the AI to do the bad move consistently, which is where we are stuck at.

We plan to include in the next release a debugging feature that can record the current game state to a file, that will allow us to open this file in the game and hopefully be able to replicate the bad move.

[Lodici]

...Say the suboptimal move is casting Flight on a creature with flying, as long as Flight is in hand and the creature with flying is on the battlefield, the AI may try to cast Flight on it...

That is an interesting example because if the AI did that during a game I would think, "dumb AI what a stupid move!". It breaks the immersion somewhat because I simply cannot imagine a real person doing it. This seems such an obviously wrong thing to do which in my mind as a non-AI expert could be alleviated with a simplish hard-coded rule - if permanent has X don't even consider casting X on it unless there is an immediate benefit in doing so.

[sponeta]

Well, I wasn't really talking about the AI learning anything. I mean, sure, if it was possible, it'd be cool... but an AI that can learn is like a whole different beast.

Neither was I, I was talking about telling the AI "do not do X this turn". The "this turn" part is necessary otherwise you'll potentially have to repeat this every time the AI has priority, which could be quite a lot of times in a turn for an instant effect.

Oh I see what you're saying now! Yes, that could become annoying... and certain actions could become valid later in the turn, too. Like regenerating a creature, which is another thing the AI does multiple times in a row.

...Say the suboptimal move is casting Flight on a creature with flying, as long as Flight is in hand and the creature with flying is on the battlefield, the AI may try to cast Flight on it...

That is an interesting example because if the AI did that during a game I would think, "dumb AI what a stupid move!". It breaks the immersion somewhat because I simply cannot imagine a real person doing it. This seems such an obviously wrong thing to do which in my mind as a non-AI expert could be alleviated with a simplish hard-coded rule - if permanent has X don't even consider casting X on it unless there is an immediate benefit in doing so.

Hehe that's exactly what I was saying in the first place.

Flying is one example, but I've seen the AI do this for all passive abilities, First Strike, Trample, and so on, as well as nonstandard ones like Can't Be Blocked.

The AI will also add combat abilities like First Strike at any point after combat has been resolved, or Can't Be Blocked after blockers have been declared.

Regeneration is similar but not quite the same, because it stacks. So in theory you could regenerate multiple times if there's several damage spells on the stack. But even then, pre-emptive regeneration doesn't make sense because you can always do it as a last-minute response to something on the stack.

When you add in the costs for activating these abilities, that's where the AI starts to feel really un-human. It seems it'll just as likely pay 1 mana to regenerate, or sacrifice another creature to regenerate. You'd think that if you sacrificed another creature, then the one you're keeping alive would have to be really worth it.

I'm not sure what to do with firebreathing out of combat though... maybe that could be fixed with bundling all of the firebreathing pumps into one decision like I described, that would at least avoid really long delays.

[jerichopumpkin]

I was reading the discussion, and an idea struck me. There is a common element in all of this: the AI does something that has no benefit at all, and usually expends mana to do it (or worst he sacrifice a permanent or discards a card). That should be possible to avoid, adding a check in the AI to make it choose the path with the most resources avalaible (even if they are not needed).
Could this be a way to at least reduce the bad moves done by the AI?