Defining Pre-2K-Limit Cards.dat

[CirothUngol]

A few months ago (before Manalink 3.0) I disassembled the Cards.dat file to try and identify the location of all the values in the Manalink.csv, with the goal of defining the Cards.dat file and building a DAT2CSV converter.
I've cleaned up my chart so that hopefully others can read it:

Code: Select all

StartOfCards.dat v3.x
---------------------
This chart was constructed by comparing the Manalink.csv and Cards.dat
from the earliest update from which I had both files (20090427).

The Cards.dat file and this chart are little-endian.
All but the BitFields are broken into 4-byte words.

HeaderSize = 8 bytes
DataFieldSize = 152 bytes
NewDataFieldSize = 20 bytes

StartOfHeader
-------------
byte 00 = Number of DataFields
byte 04 = Size of StringData
-----------
EndOfHeader

StartOfDataFields
-----------------
byte 00 = CardID
byte 04 = Offset for FullCardName
byte 08 = Offset for ShortCardName
byte 12 = Expansion
byte 16 = CardColor
byte 20 = CardType
byte 24 = SubType : SubType2
byte 28 = Offset for CardType
byte 32 = DB Card Type 2
byte 36 = Rarity
byte 40 = Req.Mana, Colorless : Black : Blue : Empty
byte 44 = Req.Mana, Empty : Green : Empty : Red
byte 48 = Req.Mana, White : Empty : Empty : Empty
byte 52 = Mod Dummy 1
byte 56 = Mod Dummy 2
byte 60 = Mod Dummy 3
byte 64 = Mod Dummy 4
byte 68 = Num Pics
byte 72 = Empty

Start of BitField
-----------------
byte #    bit#

byte 76 = 0-2 = Mod Combat Manipulator
          3-5 = Mod Control
byte    = 6-0 = Mod Counter Spell
     77 = 1-3 = Mod Recycler
          4-6 = Mod Damage Prevention
byte    = 7-1 = Mod Gain Life
     78 = 2-4 = Mod Discard
          5-7 = Mod Draw Cards
byte 79 = 0-2 = Mod Land Destructor
          3-5 = Mod Direct Fire
byte 80 =  -  = Empty
byte 81 = 4-6 = Mod Library Manipulation
byte    = 7-1 = Mod Damage Opponent
     82 = 2-4 = Mod Creature Generator
          5-7 = Mod Creature
byte 83 = 0-2 = Mod Artifacts
          3-5 = Mod Enchantments
         
byte 84 = 0-7 = AI Inc Power
byte 85 = 0-7 = AI Inc Toughness
byte 86 = 0-7 = AI Power
byte 87 = 0-7 = AI Toughness
byte 88 = 0-7 = AI Base Value

byte 89 =  -  = Empty
byte 90 = 0-2 = Dependent Wall
          3-5 = Dependent Creature
byte    = 6-0 = Dependent Artifact
     91 =  -  = Empty
     
byte 92 = 0-7 = Sleighted Color
byte 93 =  -  = Empty
byte 94 =  -  = Empty
byte 95 =  -  = Empty

byte 96 =   2 = VS White
            4 = VS Red
            6 = VS Green
byte 97 =   0 = VS Blue
            2 = VS Black
byte 98 =   2 = For White
            4 = For Red
            6 = For Green
byte 99 =   0 = For Blue
            2 = For Black
byte100 =   2 = Count as White
            4 = Count as Red
            6 = Count as Green
byte101 =   0 = Count as Blue
            2 = Count as Black
byte102 =   2 = VS White Land
            4 = VS Red Land
            6 = VS Green Land
byte103 =   0 = VS Blue Land
            2 = VS Black Land
byte104 =   2 = For White Land
            4 = For Red Land
            6 = For Green Land
byte105 =   0 = For Blue Land
            2 = For Black Land
byte106 =   2 = Count as White Land
            4 = Count as Red Land
            6 = Count as Green Land
byte107 =   0 = Count as Blue Land
            2 = Count as Black Land
byte108 = 0-2 = AI Landhome
          3-5 = AI Wall
byte    = 6-0 = AI First Strike
    109 = 1-3 = AI Trample
          4-6 = AI Banding
byte    = 7-1 = AI Flying
    110 =  -  = Empty
byte111 =  -  = Empty
byte112 = 0-3 = Exp Rarity - Revised
          4-7 = Exp Rarity - 4th Edition
byte113 = 0-3 = Exp Rarity - 5th Edition
          4-7 = Exp Rarity - Chronicles
byte114 = 0-3 = Exp Rarity - 8th Edition
byte115 = Empty
---------------
End of BitField

byte116 = Offset for CardRulesText
byte120 = Offset for CardFlavorText
byte124 = Power
byte128 = Toughness
byte132 = Num of BDA
byte136 = DBA 1-4
byte140 = DBA 5-8
byte144 = Empty : Empty : Mana Source Colors : Inflatable
byte148 = Hack Color : Empty : EnchantTypeOPP + EnchantTypeOWN : Hack Mode
---------------
EndOfDataFields

StartOfTextFields
-----------------
TextFields start at HeaderSize + (DataFieldSize * NumOfDataFields)
TextFields use a NULL as an EndOfField character

Each card has 10 TextFields in the v3.x Cards.dat:

FullName
Name
Type Text
Rules Text
Flavor Text
Damage Text
Effect Title Text
Effect Text
Legacy Title Text
Legacy Text
---------------
EndOfTextFields

StartOfCodedBitField
--------------------
4 byte value for Size of Coded BitField is at HeaderSize +
(DataFieldSize * NumOfDataFields) + Size of StringData
byte 00 = Size of Coded BitField

The Coded BitField immediately follows the Size of Coded BitField value
------------------
EndOfCodedBitField


StartOfNewDataFields
--------------------
NewDataFields start at HeaderSize + (DataFieldSize * NumOfDataFields)
+ Size of StringData + Size of Coded BitField + 4

byte 00 = Offset for Damage Text
byte 04 = Offset for Effect Title Text
byte 08 = Offset for Effect Text
byte 12 = Offset for Legacy Title Text
byte 16 = Offset for Legacy Text
------------------
EndOfNewDataFields

------------------
EndOfCards.dat 3.x

Previous Microprose Cards.dat contain no NewDataFields
and contain only the following TextFields:

dat1.0 = 325187 bytes, 5 textfields: WVL#; FullName; ShortName; RulesText; FlavorText
dat2.0 = 330587 bytes, 5 textfields: FullName; ShortName; CardType; RulesText; FlavorText
dat3.0 = 340009 bytes, 6 textfields: WVL#; FullName; ShortName; CardType; RulesText; FlavorText

If anyone could fill in those two blanks (where is the CodedStatus and what are the unidentified ~250 bytes before the StartOfNewData),Thanx snako! Now maybe I can remove dependency on the Manalink.csv from the PlayDeck Analyser and finish that DAT2CSV converter.
I removed dependency on the Manalink.csv from the PlayDeck Analyser and finished that DATtoCSV converter.

Edit -
Chart updated to reflect more complete\correct information.

Edit2 -
Updated chart with the final version I used to write the DAT converter.

[Gargaroz]

You really have to ask Mok, send him PM, I'm sure he'll answer.

[Snacko]

The old dat file is as follows (post mok pre2k):
int 4 bytes - Number of DataFields
int 4 bytes - string data size
152 bytes * Number of DataFields - data fields
string data size bytes - string data
int 4 bytes - card coded data size
card coded data size bytes - cards coded bitfiled
20 bytes * Number of DataFields - struct containing addresses of DamageText, EffectTitle, EffectText, LegacyTitle and LegacyText

[CirothUngol]

hmm... there was only one thing I hadn't found and only one thing I hadn't identified. *sigh*
Proof positive I'm completely worthless after 9:00 pm. ^_^

Thanx tons for the info Snako, it's really appreciated. I just disassembled the Coded BitField and it fit in the CSV like a glove.
Seriously, I can't believe I didn't see that.

[CirothUngol]

I've finished the DATtoCSV converter and added it to the PDAnalyser. Here's a RAR containing CSV conversions of the old Microprose MtG DATs (v1.0, 2.0 and 3.0) plus the early Manalink DATs.
I've also included the original and converted CSVs for the last Pre-2K update (20111201_v2), just to illustrate the validity of the conversions.

...I'm unsure if this will be helpful in anyway, but it was a fun way to waste a Saturday. ^_^

[gmzombie]

nice. Now if we could just figure out the cards1.dat for shandalar we may be able to do something with that old program..

[CirothUngol]

It seems that the Cards1.dat first showed up in Mok's original Manalink 2.0 version, and it's almost an exact duplicate of the Cards.dat from Manalink 1.3.2, with the only difference being the Number of Pics listed in the DAT. In turn, the Manalink 1.3.2 Cards.dat is nearly the same as the one in MtG_v3.0, with a few text corrections and some values changed in the "Mod" and "DB CardType" catagories. Apparently, Mok decided to just rename the Cards.dat from Manalink 1.3.2 to Cards1.dat for use with Shandalar.exe since he was altering the DAT so much for Magic.exe.

I believe that all useful info has been extracted from Cards1.dat, and if you'd like to alter the values you should be able to re-build the Cards1.dat from the CSV using csv2dat.exe (although I haven't confirmed that).

[Aswan jaguar]

CirothUngol I tool a look into the mtg_v3.dat converted to csv and there are a lot of columns unlike the ones we have now in Manalink 3.0 that are totally empty like:
Mod Direct Fire,
Mod Land Destructor,
Mod Draw Cards,
Mod Discard Cards,
and others

and others that the values are successive repeatedly indicating probably mistake like:

Mod Gain Life,
Mod Damage,
Mod Recycle.

Am I not understanding something correctly or the conversion is not so successful?

[CirothUngol]

The conversion is not so successful. ^_^

...at least not on the earlier Microprose Cards.dat files. The DATtoCSV converter was made by comparing the Cards.dat and Manalink.csv files from the earliest Manalink 2.0 update from which I had both files (20090427). I've converted and compared many of the Manalink 2.0 Cards.dat\Manalink.csv files, and they're exactly the same except for the "Expansion" column (the DATtoCSV shows all leading zeros, early Manalink.csv files often did not). I'm rather sure of it's accuracy on Manalink 2.0 Cards.dat. However...

If you view the Cards.dat from both MtG_V3.0 and Manalink 2.0 side-by-side in a HexEditor (discard the 8-byte header and use an 8x20 grid), you can see that Mok added additional data to the Cards.dat that wasn't originally there (new values in new places). Of course, that would mean that any fields not present in the original MtG files (added later to ML 2.0) would appear empty. If you'd like to check it out, I've uploaded all of the old MtG Program Files to my MediaFire account (check the link in my signature, look under "Microprose MtG" for MtG_Original_Program_Files.7z). Much of the info from the old Cards.dat seems to be in the same place... the problem is I don't have a roadmap for the earlier files.

It seems likely that any values "missing" from the older Cards.dat files (i.e. zeros) was never actually there (being added later by Mok), but there are some values that are blatently wrong (look at AI Inc Power\Toughness, which only has values for Lands). Unless I decide to re-learn Assembly and try to track were the values go and what they do, this is the closest I'm likely to get.

Thanx for the interest AJ, sorry for not mentioning that the early Cards.dat conversions are only mostly correct. The DATtoCSV converter will be an option on v0.3 of the PlayDeckAnalyser. If I have some time after releasing that I'd like to flesh-out all of the incorrect values from the early Cards.dat.

[Ciroth returns after ~10 minutes]
Well, let's hear it for trial-and-error. I was looking at the odd repetitive values you mentioned (Mod Gain Life\Damage\Recycler) and they don't just repeat, they increase. That's the sure sign of an offset pointer. Apparently, bytes 76 and 77 (the beginning of the main ML 2.0 bitfield) are a TextField pointer to the "WVL" number for each card (look at the Cards.dat TextField in a HexEditor, you'll see each card has a 0000.wvl, 00001.wvl, etc). I don't know what that little piece of text was used for, but it was important enough to warrant a pointer in the Cards.dat, yet is no longer used as of Manalink 2.0.
Go figure.

[Aswan jaguar]

I am most certainly interested in having a csv of the last cards.dat Microprose made and have 100% the original values of the cards.I am trying to define those values by comparing them and then testing them and it is very possible some values of original cards have changed through years of developing this project,as some values seem not to be right.On the other hand Mok found some of his files and from him not wanting changes on the original slots and cards I can assume that the values we have now are almost identical to the original ones.

Thanks for trying for something like that.

[CirothUngol]

The DATtoCSV converter is now an option on the new PlayDeck Analyser.