Page 1 of 1

Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 05 Jun 2015, 08:32
by quatrocentos-e-vinte
I'm not sure if this is the best place to make some of these questions, but since there isn't a Shandalar Development section...

1) What is the difference between Manalink.csv and ct_all.csv? Manalink.csv has about 30% more rows than ct_all.csv... is one generated based on the other?

2) Where can I get the most recent version of those two files? (I'm particularly interested in having up-to-date values for "AI Base Value"). Do I need to fetch them from a repository? (And, if yes, can I get read access? :P)

3) Is there any flag in any of these files that tells me whether a card is "Mythic Rare", whether a card is "Shandalar Special" (i.e. cannot be bought and can only be obtained through dungeons, in Shandalar), whether a card is of the "Planeswalker" type and whether a card has special restrictions IRL (Restricted or Banned in a format)?

(I apologize if any of these questions are trivial; I did try to find out the answers by myself, but was unable to.)

Thanks.

(Note: I'm asking this because I'm trying to make a small online app to help people make Shandalar decks, so I need to be able to determine every card's rarity and power level)

Re: Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 05 Jun 2015, 10:18
by stassy
2) The latest version of Manalink.csv is located in the magic_updater folder (look for latest modified file). Unfortunately, AI base value only affect Draft Mode for Manalink 3.0 (though Korath might be working on AI behavior in Shandalar based on Manalink.csv AI base value colummn).
I am currently the one in charge to filling this column for every new expansion, based on the relevant thread.

Re: Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 05 Jun 2015, 14:47
by Aswan jaguar
1- Ct_all.csv contents are in Magic.exe and you can edit easily main functions of it.It doesn't have uncoded card entries like Manalink.csv which lists all cards WOTC has released so far (apart Un-sets if I remember correctly).
3- About the plainswalker type in Manalink.csv you can find it in the "New types 1" column the 1006h value which represents the plainswalker type.

I don't know about the rest.

Re: Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 05 Jun 2015, 18:48
by Korath
quatrocentos-e-vinte wrote:1) What is the difference between Manalink.csv and ct_all.csv? Manalink.csv has about 30% more rows than ct_all.csv... is one generated based on the other?
Manalink.csv contains all cards, and (mostly) contains only data that can be determined mechanically from the physical card. ct_all.csv contains only cards implemented in Manalink, and (mostly) includes only data specific to the computer game.

Internally, ct_all.csv values are slightly more accessible than Manalink.csv values; except in the card-drawing functions now residing in Drawcardlib.dll, where ct_all.csv values are vastly harder to get at (and are totally inaccessible when running from the standalone deck editor).
quatrocentos-e-vinte wrote:2) Where can I get the most recent version of those two files? (I'm particularly interested in having up-to-date values for "AI Base Value"). Do I need to fetch them from a repository? (And, if yes, can I get read access? :P)
The ones in the latest Manalink patch are the most up-to-date that exist. They typically only get committed to our repository immediately before a patch is released. It's one of the things that drove me absolutely nuts while I was actively developing Manalink.
quatrocentos-e-vinte wrote:3) Is there any flag in any of these files that tells me whether a card is "Mythic Rare",
As far as Shandalar is concerned, it (mostly) uses the printed rarity of the expansion the card's displayed with. Which expansion it's displayed with is nontrivial, and depends on the end-user's settings in the [Expansions] section of DuelArt/Duel.dat. Once you have the expansion, it's a matter of looking up that expansion in the individual expansion columns in Manalink.csv; Mythic Rare is "M" there. (The plain "Rarity" field, as you've no doubt noticed, only contains 1 for common, 4 for uncommon, and 2 for rare, with other rarities like land and mythic rare and special inconsistently pigeonholed into one of those.)

Shandalar also bumps the rarity of almost all cards in the current Vintage and Legacy banned/restricted lists to mythic rare, regardless of their printed rarities. Those card lists are hardcoded, and are in src/shandalar/Shandalar.cpp in function compute_rarity(). (Note that the last paragraph in the long comment at its start is out of date; in particular, non-restricted mythic rares no longer get moved to rare, and it's usually the first, not last, printing that determines a card's expansion.)
quatrocentos-e-vinte wrote:whether a card is "Shandalar Special" (i.e. cannot be bought and can only be obtained through dungeons, in Shandalar),
Not in the csvs, though it originally was one of the columns currently marked "Flags: Unknown (Shandalar)" in ct_all.csv (the second from the right). Cards marked with the leftmost "Flags: Unknown (Shandalar)" also weren't available in towns/nomad's bazaar/etc., but weren't placed in dungeons; it's unclear if that was the only reason for the flag or a side effect of its true purpose. Either way, both those flags are overwritten at runtime in Shandalar; the list of cards that get put in dungeons are in src/shandalar/Shandalar.cpp, function fix_data(), in the dungeon_cards[] array.
quatrocentos-e-vinte wrote:whether a card is of the "Planeswalker" type
In Manalink, the card's flagged as an enchantment and has a cc[2] value of 9. cc[2]==9 has a thousand-and-one other uses, though, including "can activate from graveyard", so there's false positives. I'm probably going to use the slower-but-surer method of checking for SUBTYPE_PLANESWALKER (0x1006) in the New Types 1 column of Manalink.csv once I start making planeswalkers in Shandalar, just like card drawing already does.
quatrocentos-e-vinte wrote:and whether a card has special restrictions IRL (Restricted or Banned in a format)?
Again, not in the csvs. Seems like it would be handy to have, maybe as different rarities in the fake "format" expansion columns like (T1) Vintage and (EDH) Commander.

Re: Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 05 Jun 2015, 19:18
by quatrocentos-e-vinte
Wow. Thanks for all the useful information, everyone.

Now that you've clarified things for me, I think I will mostly work with ct_all.csv (getting whatever needed from Manalink.csv).

stassy: I am mostly interested in "AI Base Value" as a way of quantifying how good a card is and, therefore, how much gold it should cost inside Shandalar (so, for purposes other than fine-tuning the AI); Thanks for yours and Aswan's work here (and everyone else involved): the data seems extremely accurate to me. I'm currently working on scraping IRL prices for MTG cards and seeing how consistent those are with your "AI Base Value" estimates (under the assumption that "card price" mostly depends on "card value in the game" and "rarity").

Aswan: Thanks :) that was quite helpful. (btw.. I thought it was funny you said "plainswalker", rather than "planeswalker" :P everyone loves plainswalkers!)

Korath: Regarding getting the "real" rarities and the card's status as banned/restricted, I found some online APIs that can be used to fill in that information, so it should be ok after all. Thanks for pointing out the location of the special lists of cards: that will come in handy.

My idea is making an app where people can upload the Shandalar decks they make, and get information on how many commons/uncommons/rares/mythic they have in the deck, along with an estimate of the deck's power level (normalized by the original decks' power levels). I think this could be useful by mitigating the annoying parts of making decks while following restrictions. As a side effect, I'm also trying to figure out better approaches to calculate Shandalar card costs, as well as possible methods of automating the annotation of "AI Base Value" (which I imagine to be very boring).

Getting the IRL sale price of many cards doesn't have to be a boring job ;)

Code: Select all
> getPrice("Mox Pearl")
[1] 129999
> getPrice("Storm Crow")
[1] 49
> getPrice("Oubliette")
[1] 1299
> getPrice("Muscle Sliver")
[1] 99
> getPrice("Dark Confidant")
[1] 5499
(prices in USD cents)

EDIT: Also, Korath... that thing you said about cc[2]... is that why some cards (like Fireball) have crazy casting cost values?

Re: Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 05 Jun 2015, 20:28
by Korath
No, 40 in Manalink.csv and 255 in ct_all.csv just mean "X". (But "X" doesn't necessarily mean 40/255, especially where X is predetermined instead of being "any amount of mana you want to pay", like for Spoils of War.)

Re: Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 06 Jun 2015, 12:06
by stassy
quatrocentos-e-vinte wrote:I am mostly interested in "AI Base Value" as a way of quantifying how good a card is and, therefore, how much gold it should cost inside Shandalar (so, for purposes other than fine-tuning the AI); Thanks for yours and Aswan's work here (and everyone else involved): the data seems extremely accurate to me. I'm currently working on scraping IRL prices for MTG cards and seeing how consistent those are with your "AI Base Value" estimates (under the assumption that "card price" mostly depends on "card value in the game" and "rarity").
Well, "accurate" in the context of the card power alone (but Tribal), because those values are usually set before the expansion is released, so I have no idea of the cards usefulness in combo or meta game.

When I read your monster deck proposal, I though at first "oh well, this is going to be hell to find the rarity of each card", but then I read you here and I am now more than interested in the use of a program that can find rarity automatically :mrgreen:

Re: Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 06 Jun 2015, 22:55
by quatrocentos-e-vinte
Ok. I guess I got most of the information I need to start :)

In case it's useful for someone, here is some additional annotation for ct_all.csv. The row order of the cards is exactly the same as the last version of ct_all.csv I could find. The meaning of the columns is:

- csvid: same as the "ID" field of Manalink.csv
- name: card's full name
- colors: card's colors (comma-separated list)
- types: card's types (comma-separated list)
- rarities: card's rarities (one per print, comma-separated list)
- edh: whether card is restricted/banned in commander (otherwise, it's considered "legal")
- legacy: whether card is restricted/banned in legacy (otherwise, it's considered "legal")
- vintage: whether card is restricted/banned in vintage (otherwise, it's considered "legal")
- rarity: "maximum" rarity (i.e. if a card was ever printed as mythic, then it's "mythic"; else, if a card was ever printed as rare, then it's "rare", etc.)
(these last four fields could be useful)

In the case of cards which are not real cards (e.g. "* Draft") or stuff that otherwise doesn't exist in online databases (e.g. Astral Set), the information was taken from Manalink.csv (but none of those cards are banned/restricted/mythic, I guess).

Re: Some questions regarding Manalink.csv and ct_all.csv

PostPosted: 07 Jun 2015, 21:13
by quatrocentos-e-vinte
stassy wrote:When I read your monster deck proposal, I though at first "oh well, this is going to be hell to find the rarity of each card", but then I read you here and I am now more than interested in the use of a program that can find rarity automatically :mrgreen:
Maybe this helps? ;)

(after playing a bit around with it... yeah, the rarity guidelines seem a bit too strict :P)