Using Forge for Playtesting DotP 2014

[JarmenKell]

I have recently gotten back in to MtG, thanks to DotP. With the new DotP 2014 just around the corner, I started to wonder how I could better tune my decks this year.

I decided it would be best if I could conduct multiple AI vs AI matches and collect data about the matches to determine what cards worked better than others. Since DotP decks have limited variability in their construction, you could compare various deck builds and collect information quickly.

After browsing the forums a bit, I decided BotArena was the only program I could use that would simulate multiple AI matches and produce log files that I could parse to collect information. Now that Forge allows for AI to AI simulations I have adapted my code to parse Forge files. The only problem being that I must manually start every simulation, which can be tedious when simulating 50-100 games. However Forge has a wealth of cards and a great AI compared to BotArena. So far, everything is working well. I have started a Google Site to detail my work and provide the results of the simulations. The latest data contains a breakdown of all currently leaked decks in DotP 2014 in 1 v 1 simulations. Since the new sliver cards aren't up on Forge I can't simulate that deck nor the "Hunter's Strength" deck, since I don't have a true card list for it yet.

So far the information gleaned looks okay. I am conducting 50-100 head to head battles of the different decks and parsing the log files using Python. The AI plays aggressively and without prior knowledge of the other deck e.g. it frequently plays removal on 1/1 cards in the Eldrazi "Chant of Mul Daya" deck. "Avacyn's Glory" seems to have a clear advantage on all decks most likely due to it's copious amounts of removal.

What I would like from the community is some discussion on this. Does this seem like a good way to playtest decks? Is there some data that would be helpful in determining if a card is "critical" to a deck? What would be some good builds to test after the first basic builds? What am I doing wrong?

The spam filters are blocking the website url, simply google "Magic the Shattering" (in quotes). It should be the first one. Go to the "Simulation Data" tab for the latest mined data.

[Max mtg]

JarmenKell, the AI vs AI matches could potentially (though I've never tested it) be ran without user interface attached at all. So the problem of manual launches can be solved with some knowledge of Java (it's all about not calling attachToGame() from forge.game.Match.startRound() method)

Don't know what else to add besides that any AI could be an unreliable measure to estimate deck's power.

[JarmenKell]

I'll have to look into the code more and try to see what I can do. More information in the log files would be advantageous too, so I will look into getting that up and running. I originally wrote everything for BotArena but switched gears for Forge. Running without the UI would be good but initially it was helpful for checking the AI's decision making.

I agree it's not the best. A human simulations would be better, but the ability to run massive simulations with a decent AI should render some information. The overwhelming majority of plays I am seeing make sense.

[moomarc]

As long as you're not trying to get the AI to play combo decks it shouldn't be too bad.

[JarmenKell]

Here're the links to both the main page and the simulation data.

Main Page:
https://sites.google.com/site/magictheshattering/home

Data:
https://sites.google.com/site/magictheshattering/documents