New way of editing card_data_t (after merging the cards)

[Mok]

As you may have noticed already, merging and editing card_data_t from several old game versions using sky editor would take forever. There's also the problem of "unknown" entries that are filled with rubbish/random data because it's easy to change by mistake and not knowing it's proper value makes it hard to notice. I propose switching to a format similar to cards.dat ie. to xls/csv and expanding the fields. While you lose the ability to edit using known tool (as editing with sky editor wouldn't be possible after switch, I'd like to clean the exe a bit to prepare for it, and sky editors seems to be all hard-coded for old exe), it shouldn't be hard to switch as you already use the csv in case of cards.dat. Attached is an initial example version in excel format (for now). Some fields are split into a bit values, where previously there was only one combined value, and all unnamed fields were given some name. These are based on my old knowledge of the engine and not on your current expanded version in C sources. It means that some fields are used differently now in your code, while still being used in the old way in magic.exe. We'll have to resolve it somehow, but that's not important now. If you agree to continue with this we could go by all column names and decide what to call it so it's easy to edit later in case of new cards. I'll try to check the exe code to see exactly where given entry is used to get better idea what it does with card examples if possible.

Edit: v2 is uploaded, with cc[2] column added.

[Gargaroz]

What I could say ? I'm gar for Mok

Edit : In your file, our "cc[2]" should be "Extra Flags", right ? We're using them for the Rules Engine, (for example, "Force of Will" requires "1", while "Garruk the Wild Speker" requires "9"), they're bitfields ? So the leftmost would be 1, the next 2, the next 4 and son on ?

[Mok]

Ok, I'll slowly start to decribe the columns, proposals for better names (more suited for your work) are very welcome. Part 1.

The "secret" value is unused. I removed it completely as it's just a waste of space. The exe uses it to know how many extra slots are free, but that's only -1 value and set automatically anyway.

"Sort order" <- that's just a cosmetic field to help sorting the cards. There are only a few order requirments here: 1) basic lands and original lands that count as basic lands are first. Some code definitely used it, I'll try to find it but that might be hard. 2) effect cards and 16 free slots are last. Yes, there's only 16 slots for extra types created while playing and it cannot be increased without major changes in exe. For example only these 16 slots are copied when AI starts calculating and so on. In any case, sorting in excel by "Sort order" and "Full Card Name" is what I used for initial version. Blue entries are coming from the old 2005 edition (updated code address whenever it changed) so the other fields should be ok if you want to look how anything was set by the original programmers (or me at the time).

"Name (34 letters)" <- almost cosmetical field. It's used only for ease of debugging and first letter of name is used as a sort order whenever the game adds new card to some sorted list. The only one that must stay the same is "DataCard" as it's used to scan the struct for effects block at the end. This field can probably be truncated to make space for extra entries whenever you need to expand this struct. 35 bytes at offset 0x01 in destination struct. "char name[18];" in your header.

"Full Card Name in CSV" <- card name from CSV is used to get a CID from Cards.dat. DWORD at offset 0x24h in destination struct. "uint16_t id;" in your header.

"Type: Effect, Type: Artifact, Type: Interrupt, Type: Instant, Type: Sorcery, Type: Enchantment, Type: Creature, Type: Land" - combined makes a card type entry. Byte at offset 0x28. "uint8_t type;" in your header.

"Family" - a value for specific group of cards. This isn't really same subtype as in csv but close. It's used to identify some cards for certain effects that look for them. An example is "wall", "land that counts as a basic land" etc. It's not very flexible as you cannot combine 2 families for 1 card. I think you expanded it but in the past I had it defined like below. It's a byte at 0x29. "uint8_t subtype;" in your struct.

FAMILY_WALL = 0
FAMILY_MERMAN = 1
FAMILY_ZOMBIE = 2
FAMILY_GOBLIN = 3
FAMILY_DWARF = 4
FAMILY_DJINN = 5
FAMILY_EFREET = 6
FAMILY_RAT = 7
FAMILY_URZALAND = 8
FAMILY_ORC = 9
FAMILY_ELEPHANT = 10
FAMILY_MAMMOTH = 11
FAMILY_COUNTSASBASICLAND = 12
FAMILY_BASICLAND = 13
FAMILY_LEGEND = 14
FAMILY_LEGENDARY = 15
FAMILY_ENCHANTWORLD = 16
FAMILY_NONE = -1

"Color Unused, Color ArtMana, Color White, Color Red, Color Green, Color Blue, Color Black, Color Colorless" - combined into a card color value. Byte at 0x2A. "uint8_t color;" in your header.

Required Color Mana (total) - a sum of all color mana required to play the card. This is used by the code calculating available mana and playable cards. Byte at 0x2B. Part of "uint8_t cc[3];" in your header.

Required Colorless Mana (total) - like above, except sum of colorless cost. X is 255. Byte at 0x2C. Part of "uint8_t cc[3];" in your header.

"Extra Flags (Unused) * 6, Modifies Produced Mana, Has Self Cost" - an extra flags field. Mostly unused, not sure if I added it in the past or it was there from the start. It was originally at 0x2D but as your cc[2] replaced it, I moved it now to 0x32.

In any case "modifies produced mana" signals that a card will alter mana produced by other cards. The cards that use it are Fertile Ground, Gauntlet of Might or Wild Growth. Lots of new cards have trash value here. The subroutine that calculates potential mana from available cards checks this flag and if it's set it calls the card with an event type 0xA6 (no entry in your header, CP_MODIFY_PRODUCED_MANA in my old stuff). Checked/modified mana source card is set in _affected_card and _affected_card_controller.

"Has self cost" - means the card has an extra cost to pay. If this flag is set the card is called with an event 0xA4 (EVENT_MODIFY_COST in your header, CP_ADDITIONAL_SELFCOST in mine). I believe I added this flag and Avatar of Hope was the only card that used it at the time.

Edit: Now that I check it, you seem to have expanded this field quite a bit... I'll need to redo the xls and re-add the old values. I just hope you used it as a bitfield and not as an enum...

[Aswan jaguar]

Mok wrote:

2) effect card and 16 free slots are last. Yes, there's only 16 slots for extra types created while playing and it cannot be increased without major changes in exe. For example only these 16 slots are copied when AI starts calculating and so on.

Does this possibly have to do with AI not playing a single card(after some turns at many games)?If it has to calculate more 16 slots?
You beat me to it,I was about to ask if you have a csv definition guide or you remember how the fields are explained.(There are 2 current csv definitions in wiki from Orion79 and CirothUngol,If you want to take a look.)
http://www.slightlymagic.net/wiki/Editi ... .csv_files
You said it is an example version so you might have missed some values?(type token for example)

[Gargaroz]

Could you compact "Extra flags" in an unique field, so it'll be more easy to re-add the values we're using for the Rules Engine ?
And I suppose "modifies mana producers" is bit 6 and "has self cost" is bit 7 of the struct, right ?

[Mok]

Mok wrote:

2) effect card and 16 free slots are last. Yes, there's only 16 slots for extra types created while playing and it cannot be increased without major changes in exe. For example only these 16 slots are copied when AI starts calculating and so on.

Does this possibly have to do with AI not playing a single card(after some turns at many games)?If it has to calculate more 16 slots?
You beat me to it,I was about to ask if you have a csv definition guide or you remember how the fields are explained.(There are 2 current csv definitions in wiki from Orion79 and CirothUngol,If you want to take a look.)
http://www.slightlymagic.net/wiki/Editi ... .csv_files
You said it is an example version so you might have missed some values?(type token for example)

No, I think not. These 16 slots are only used rarely if a new type of card is defined. Some clone type cards usually do it. And what I meant is increasing it is hard because whenever an AI starts calculating, the game makes a snapshot of crucial data that is restored before each round of virtual "play". And expanding that memory area is non-trivial. From the results of all these virtual rounds the real "move" is chosen.

Hanging AI might be a result of simple bugs. Just yesterday I spotted a couple of embarrassing bugs in the cards I added years ago hehe

As for your second question, this is not the same file as Manalink.csv. The contents of this file are inside Magic.exe and up to this point were edited using skymagic editor.

[Mok]

Could you compact "Extra flags" in an unique field, so it'll be more easy to re-add the values we're using for the Rules Engine ?
And I suppose "modifies mana producers" is bit 6 and "has self cost" is bit 7 of the struct, right ?

So you used it as an enum and not bitfield? And no, the order of columns is backwards, so self cost was bit 0 and modify produced mana was bit 1. I'll restore all values for new cards, as I thought it was unused before. Ugh, how can you write lots of code using only magic numbers? I'd be confused a week after writing that hehe BTW: If you used that field differently and want to keep it that way, let me know and I might just kill all the code that uses it the "old" way. Assuming your code didn't use any of it.

Edit: I checked the C sources a bit and you really use it as an enum and not bitfield. Assuming you aren't using the special events produced by the old values of bits 0 and 1, id'd move your field cc[2] to another spot in the same structure (there's a couple of free places that can be used for that). But it seems you are using EVENT_MODIFY_COST quite a lot, so either you redefined it somewhere or you are using that field as both enum and bitfield at the same time... crazy

Edit2: After thinking about this mess a bit, I think that moving the extra flag field somewhere else is better idea while leaving your new enum in it's old place. Only 2 bits of old value are used so there will be only a few places in the exe that I'll have to modify. I'll use one of the 2 bytes from reserved3; (0x32 in the struct, marked as unused also in my notes). That means that cards that use your EVENT_MODIFY_COST will require setting the new field, otherwise it should work unchanged. What would you like as a name of your cc[2] field column (I'll make it normal 0-255 type column)?

[Mok]

One more thing... if there's more than one value of cc[2] for a card in a different exe, which one should be used?

And also... any other fields of card_data_t that you used your own definition for? This will help me not destroying the data there like initially I did for cc[2].

[Mok]

Updated first message with v2 of the xls file that now has a new column "cc[2]" and extra flags columns are moved to one of the other, unused fields right after power and toughness.

[Gargaroz]

One more thing... if there's more than one value of cc[2] for a card in a different exe, which one should be used?
And also... any other fields of card_data_t that you used your own definition for? This will help me not destroying the data there like initially I did for cc[2].

Not, it should be only cc[2].
Dunno if this is important for you, but some cards are coded with 0x80 in the "type" field : they're special cards that aren't really in play, like the Rules Engine, and using that value is important as they're not counted as permanents by our functions (and it seems that the hardcoed targeting functions ignore them).

The curret values for cc[2] range from 1 to 14, the complete list is here (scroll down a bit) : http://www.slightlymagic.net/wiki/Guide ... xe_editing

If a card requires more than one "value", it's adjusted by the function "check_decks_for_rules_engine", that employ that "get_id_by_name" for the special cases. I'll obviously modify that after the changes are done.

I'm quite surprised by your information : I was convinced that "free spell" could be only obtaioned thru the Rules Engine, but I see this was already implemented in the main engine by you. I wonder if cards like Force of Will are working without the Rules Engine too, as soon as the needed flags are set.

[Aswan jaguar]

Mok is there any chance you finish explaining the columns?