Project Firemind is back

[Lodici]

melvin is the ai expert. As far as I can tell monte carlo is the most recently implemented and actively developed AI and (I think) melvin's speciality. I only really play monte carlo because (I think) it offers the strongest challenge (the wiki page does not lie!). Ultimately what I would like to know is which decks does each particular AI play the best so we could start looking at creating preconfigured duels/challenges that we know would give a player a good game. Very useful for new players new to Magarena/MTG.

[mike]

I deployed a new worker with the Monte Carlo AI at strength 4 like the wiki suggests. It takes a lot longer to process games with 5 to 10 minutes instead of the < 1 minute results I observed before.

Obviously I prefer to use the strongest AI to have the most accurate results but the AI switch means that firemind.ch will be processing a lot slower than before.

I looked at some data and found a good solution to allow deletion of decks. The button is on the bottom of the right side on the deck view page.

[ShawnieBoy]

melvin is the ai expert. As far as I can tell monte carlo is the most recently implemented and actively developed AI and (I think) melvin's speciality. I only really play monte carlo because (I think) it offers the strongest challenge (the wiki page does not lie!). Ultimately what I would like to know is which decks does each particular AI play the best so we could start looking at creating preconfigured duels/challenges that we know would give a player a good game. Very useful for new players new to Magarena/MTG.

I'm mainly a Monte Carlo user too - however I did discover a strange occurrence with it if you are playing a deck that either has a chance to kill yourself, or no win condition, it can get a bit confused. It determines possible actions at random, so if most of those actions could be the opponent losing on their own, it declines to do anything. Very niche situation but something I encounter sometimes when testing cards.

I know that melvin was playing with the thought of using Minimax to determine possible moves then having Monte Carlo decide the best option. A super AI MonteMax (Better than MiniCarlo)

[PalladiaMors]

Mike, when/if you see this, I'd like to ask a couple things:

a) Would you consider implementing some kind of way to "report" cards with missing pictures? I don't find this all that important, but yeah there's quite a few cards with broken pics right now and it can get in the way a little bit.

b) Would it be possible to include some extra formats, such as the Block Constructed formats and Block Party (any legal block deck)? These are some of my favourite formats, I prefer to use more limited card pools. I do realize that they aren't that popular and right now might not attract too many players, though. By the way, I'm pretty sure that if Firemind is advertised a little bit in the big in the big MtG forums, it'll bring a lot of players in - it's a great, great tool.

[mike]

Sure, happy to answer your questions.

a) I am aware of the missing images problem. I'm currently working on some scripts that will check if all enabled cards have an image linked to them and I'm also working on a fallback system if the HQ pics are not yet available.

b) Yes, I am planning on adding more formats as time progresses. Right now the limiting factor is the number of duels I can process since I have to make sure that the ratings in each format are kept up to date. Block Constructed is high on the list yet since it's such a short lived format it requires more attention than eternal formats. The Block Party format seems like it would be more suitable since it is very close to an eternal format in terms of stability. I put that on the short list to implement. https://github.com/firemind/ProjectFire ... /issues/16

[Agetian]

I'm trying to register at Project Firemind. I have created an account and I've received a confirmation e-mail, but clicking the link in the e-mail results in "Invalid confirmation token" message. I tried several times but to no avail. Please help.

- Agetian

[mike]

I'm trying to register at Project Firemind. I have created an account and I've received a confirmation e-mail, but clicking the link in the e-mail results in "Invalid confirmation token" message. I tried several times but to no avail. Please help.

Sorry about that. Last update introduced a change to the way confirmation tokens are sent. It's fixed now. You can use the "Send confirmation again" function to get a new email with the fixed token.

[Agetian]

I'm trying to register at Project Firemind. I have created an account and I've received a confirmation e-mail, but clicking the link in the e-mail results in "Invalid confirmation token" message. I tried several times but to no avail. Please help.

Sorry about that. Last update introduced a change to the way confirmation tokens are sent. It's fixed now. You can use the "Send confirmation again" function to get a new email with the fixed token.

Thank you, this worked!

- Agetian

[PalladiaMors]

Maybe we could discuss a little how representative the results from Firemind have been? One thing that can't have escaped anybody's attention is that nearly every top deck in every available format is aggro. At first it might look like Magarena's AI is biased towards aggro - maybe it plays such decks better than control decks. However, everyone seems to agree that combat is in fact a weakness in the AI. I suspect that the advantage that aggro decks seem to currently have is more related to the AI being able to choose how to allocate damage in multiple blocker situations as if its creatures had banding in defense. Of course, getting free banding in defense on every creature is a massive advantage for creature-heavy decks.

The most competitive current format seems to be Modern, with over 125 rated decks. Let's have a look at what the top decks in the format have in common:

P1: 31 creatures, 2 spells, 27 lands
P2: 28 creatures, 10 spells, 22 lands
P3: 28 creatures, 9 spells, 23 lands
P4: 28 creatures, 9 spells, 23 lands
P5: 28 creatures, 10 spells, 22 lands
P6: 21 creatures, 17 spells, 22 lands
P7: 28 creatures, 10 spells, 22 lands
P8: 30 creatures, 9 spells, 21 lands
P9: 28 creatures, 8 spells, 24 lands

The average number of creatures in competitive decks seems much higher than in the actual live Modern format. Now, it's hard to really tell what's going on in the actual games because the game logs have been very confusing since the last update - I don't really know what's going on with that, but it seems like some turns are skipped or get mixed up in the logs. However, it does look like creature heavy decks are getting a significant edge by using multiple creatures to block and then spreading damage among the defenders, killing attackers without losing creatures and gaining card advantage. In the other hand, the AI handles mass removal poorly - it consistently uses cards like Wrath of God, Death Cloud or Barter in Blood to kill single creatures.

Right now it looks like decks with a huge number of creatures, particularly with high toughness, clearly have an edge because they take better advantage of the damage allocation in combat. Evasion abilities are even more important than normal, because they help in avoiding those situations. Vigilance is also more valuable than normal, as you can take advantage of the blocking damage allocation more often. Since a lot of the acceleration is missing because of the absence of the mana pool, every format is significantly slower and cards with higher casting costs look like they're seeing more play than normal. In the other hand, heavy control decks nearly always perform poorly.

This ended up being a long post, but I was trying to make sense of the things I observed after using Firemind for about two months. How do you guys feel about the "Firemind metagame"?

[ShawnieBoy]

Maybe we could discuss a little how representative the results from Firemind have been? One thing that can't have escaped anybody's attention is that nearly every top deck in every available format is aggro. At first it might look like Magarena's AI is biased towards aggro - maybe it plays such decks better than control decks. However, everyone seems to agree that combat is in fact a weakness in the AI. I suspect that the advantage that aggro decks seem to currently have is more related to the AI being able to choose how to allocate damage in multiple blocker situations as if its creatures had banding in defense. Of course, getting free banding in defense on every creature is a massive advantage for creature-heavy decks.

The most competitive current format seems to be Modern, with over 125 rated decks. Let's have a look at what the top decks in the format have in common:

P1: 31 creatures, 2 spells, 27 lands
P2: 28 creatures, 10 spells, 22 lands
P3: 28 creatures, 9 spells, 23 lands
P4: 28 creatures, 9 spells, 23 lands
P5: 28 creatures, 10 spells, 22 lands
P6: 21 creatures, 17 spells, 22 lands
P7: 28 creatures, 10 spells, 22 lands
P8: 30 creatures, 9 spells, 21 lands
P9: 28 creatures, 8 spells, 24 lands

The average number of creatures in competitive decks seems much higher than in the actual live Modern format. Now, it's hard to really tell what's going on in the actual games because the game logs have been very confusing since the last update - I don't really know what's going on with that, but it seems like some turns are skipped or get mixed up in the logs. However, it does look like creature heavy decks are getting a significant edge by using multiple creatures to block and then spreading damage among the defenders, killing attackers without losing creatures and gaining card advantage. In the other hand, the AI handles mass removal poorly - it consistently uses cards like Wrath of God, Death Cloud or Barter in Blood to kill single creatures.

Right now it looks like decks with a huge number of creatures, particularly with high toughness, clearly have an edge because they take better advantage of the damage allocation in combat. Evasion abilities are even more important than normal, because they help in avoiding those situations. Vigilance is also more valuable than normal, as you can take advantage of the blocking damage allocation more often. Since a lot of the acceleration is missing because of the absence of the mana pool, every format is significantly slower and cards with higher casting costs look like they're seeing more play than normal. In the other hand, heavy control decks nearly always perform poorly.

This ended up being a long post, but I was trying to make sense of the things I observed after using Firemind for about two months. How do you guys feel about the "Firemind metagame"?

I have noticed what you have very nicely put together (Reports like these will be -really- handy). I also noticed the prevalence of particular colours - the vast majority of decks utilise white, with blue having hardly any appearance.

The free-banding on block, and creature-heavy environment has influenced the cards I spend more time working on (and also putting together a black, prot-white heavy deck just to see what happened!), and trying to give blue a bit of love (counting only mono-coloured cards, it currently has a card pool 200 cards lower than the highest card pools, white and green).

The metagame is currently very one-dimensional, this environment would normally breed more control-type decks and creature-hate, but without any real effective card manipulation (scry > 1, 'peek' cards, and reordering library), there's no consistent way of getting hold of cards needed. (Assuming the AI would be able to rate cards well enough to identify which cards would be useful). Without these cards and AI edits, combo decks are also not going to be viable. Along with effective use of board-wipes, I don't see the current metagame changing much. But would be interesting to see if a creature-hate deck could buck the current trend

Another interest would be how small changes to AI, and increasing card pool, would effect the current high-ranking decks. Putting up some Modern decks from actual tournaments (if there's any we have all the cards for) would also be interesting.

edit: Got my numbers wrong

[mike]

Thank you for sharing your observations.

I have noticed the prevalence of aggro/creature-heavy decks. Interesting is the fact that Soul Sisters (and its variations) are doing so well. In Modern this is not a T1 deck, yet it seems to beat almost all the decks on firemind.

I would attribute blue's lack of success to two things. On one hand it is missing key deck manipulation like serum visions. On the other hand though it's strength are counterspells and tempo plays both of which are very dependent on the deck pilot knowing the opponent's deck and how to use these cards. I would not be surprised if blue is the last color the AI will master.

On the topic of "real" Modern deck on firemind. I have a dedicated account for exactly this purpose. I try to add all the Modern decks I come across to https://www.firemind.ch/users/339
10 out of 36 are playable (with very minor tweaks) by Magarena.

I also started working on a feature to analyze a meta game. https://www.firemind.ch/metas/1
It compares duel stats between a few hand-picked decks and ignores all of the "noise".

Let me know if anyone has an idea for special calculations or overview pages that could help make sense of the firemind data.

Edit: The scrambled log entries were caused by all 4 AI workers writing to the same log file. This should be fixed for future duels now.

[PalladiaMors]

No longer relevant.

[mike]

Thanks for listing the potential format breakers here.

I started trying some of them out by modifying the Mono Red Burn deck in Modern and enabling a Ur Delver Deck in Modern.

For anyone interested in how they're doing here are the links:

https://www.firemind.ch/decks/17277
https://www.firemind.ch/decks/4315


Squadron Hawk, Ranger of Eos and Bob should also make a splash in Modern pretty soon.

I'm interested to see if Astral Slide can have some impact on the current Legacy meta.

Standard still needs to be rotated. I'll do this soon.

[PalladiaMors]

When I try to start my Firemind Worker, it says my version of Magarena is outdated. I had done a couple new cards and thought that could be a problem, so now I'm using the 1.54 version downloaded from the main page (8519 cards), but it still won't work.

[mike]

Unfortunately you get the same message even if the verification of the Magarena Version fails. This can have many causes including missing internet connection or a server error. Most likely it is because of the StartSSL Certificate not being trusted on all OS Versions (see more on this in the "Crowdsourcing Project Firemind Duels" thread).