Deckbuilder update
Posted: 08 Jun 2016, 22:04
So yeah. One of Manalink's limits that's looming up in the very near future is a hard limit of 16,383 total cards, tokens, effect cards, alternate-versions-of-cards (like "Ghitu Encampment Animated" and "Heliod, God of the Sun Incarnate"), pseudo-cards (like "* Draft" and "* SPAT"), etc. "Very near future" in this case means that there's only 181 left, so unless Eldritch Moon in mid-July - announced as 205 cards, probably around a dozen of which will be double-faced and so require two slots each - has a ridiculously high number of reprints, we won't be able to fit it in. Conspiracy 2 at the end of August is out of the question.
There was a similar limit of 4095 cards that was very easily overcome - so easily, that I don't think anyone even bothered to mention it, compared to the more artificial and much more difficult 2000-card problem. (edit: Nope, was brought up here.) What was happening was that cards' internal id numbers were being truncated to 12 bits (2^12 = 4096), so that the 32-bit integers they're stored in can also accommodate other information. As it happens, nothing was being used in the 13th and 14th bits, so increasing the limit to 16383 (2^14 - 1) was simply a matter of changing a bunch of "AND <some address or register>, 0xFFF" instructions to "AND <the same thing as before>, 0x3FFF". It was trivial to search for such instructions; generally easy to be sure they were being applied to the right data, even without fully understanding the purpose of the surrounding functions; and trivial to update in-place, since the new instruction is always exactly the same size as the old one. I know this because it was one of the first things I had to do when updating Shandalar.
Bits 15 through 19, though, are used, so things would break if I just did the same thing again, even just widening by one bit to 32767 possible cards. They'd immediately break very badly in Shandalar: bit 15 stores whether an owned card is currently in the deck you duel with. Even in Manalink, though, it would require more work than expanding past 4095, since these bits are all used when exchanging data with the deckbuilder, and also internally in the deckbuilder.
So TLDR: among other things, we urgently need a deckbuilder update, and that's what I've been working on reverse-engineering for the last month and a half.
We probably could've gotten away with a more minimal patch if Manalink were our only concern, but it's really not anymore; patching all the changes to make it work with Shandalar would have taken at least as long, and I'm tired of it taking days to do things like make it legal to put more than four Snow-Covered Plains or Relentless Rats in a deck instead of seconds, so here we are. Rewriting it de novo would also have been an option; I decided against that because, unlike drawcardlib or cardartlib, it was unclear how its behavior has to change when invoked with different options. (It's still unclear, for that matter.)
At this point, it's more or less feature-complete, though I haven't yet fiddled with data interchange with Magic.exe - the whole point of the exercise! - and I've only given it very cursory testing. Everything seems to work, though, and I'm not going to be able to spend any significant amount of time on it in the next week and a half, so now seems like a good time for some wider testing.
This will not work with any but the earliest versions of Shandalar-2015/16 - RT*, LT1-3, or TH1-2 - since they patch the dll at runtime, and so are tied to the precise version we've been stuck with since the last time Mok updated it. It should work with original Shandalar just fine (though current drawcardlib.dll/cardartlib.dll as included in BFZ and later Manalink patches will not), as well as the standalone deck.exe and any version of Manalink.
Known differences from the previous deckbuilder:
Please assume anything else is a bug, and report it in this thread. You'll definitely want to keep a backup of your old DeckDll.dll for when this one inevitably proves deficient. Testing against compatible versions of Shandalar - see above for which - would be particularly welcome, since I haven't done that at all yet. (There's full installation archives for some of the RT versions floating around on freakin' Youtube. Some of them even bother to point back here.)
PS: This is probably futile, but if any stray programmers reading this have any experience at all working with Windows resource files at a low level, the one attached to the previous DeckDll.dll was corrupt. All the usual tools that normally make reverse-engineering them so easy just crash instead. I don't know whether MicroProse did this intentionally (I can't imagine why they would, compared to how little effort they put into obfuscating everything else), or Mok did it accidentally when he updated it; but it's seriously irritating, and it's going to make any updates to the dialogs way more difficult than they should be. I've included it in the attached zip. Any assistance appreciated. Never mind; tried enough tools and I eventually found one that gave me partial output, and I was able to reconstruct the rest.
There was a similar limit of 4095 cards that was very easily overcome - so easily, that I don't think anyone even bothered to mention it, compared to the more artificial and much more difficult 2000-card problem. (edit: Nope, was brought up here.) What was happening was that cards' internal id numbers were being truncated to 12 bits (2^12 = 4096), so that the 32-bit integers they're stored in can also accommodate other information. As it happens, nothing was being used in the 13th and 14th bits, so increasing the limit to 16383 (2^14 - 1) was simply a matter of changing a bunch of "AND <some address or register>, 0xFFF" instructions to "AND <the same thing as before>, 0x3FFF". It was trivial to search for such instructions; generally easy to be sure they were being applied to the right data, even without fully understanding the purpose of the surrounding functions; and trivial to update in-place, since the new instruction is always exactly the same size as the old one. I know this because it was one of the first things I had to do when updating Shandalar.
Bits 15 through 19, though, are used, so things would break if I just did the same thing again, even just widening by one bit to 32767 possible cards. They'd immediately break very badly in Shandalar: bit 15 stores whether an owned card is currently in the deck you duel with. Even in Manalink, though, it would require more work than expanding past 4095, since these bits are all used when exchanging data with the deckbuilder, and also internally in the deckbuilder.
So TLDR: among other things, we urgently need a deckbuilder update, and that's what I've been working on reverse-engineering for the last month and a half.
We probably could've gotten away with a more minimal patch if Manalink were our only concern, but it's really not anymore; patching all the changes to make it work with Shandalar would have taken at least as long, and I'm tired of it taking days to do things like make it legal to put more than four Snow-Covered Plains or Relentless Rats in a deck instead of seconds, so here we are. Rewriting it de novo would also have been an option; I decided against that because, unlike drawcardlib or cardartlib, it was unclear how its behavior has to change when invoked with different options. (It's still unclear, for that matter.)
At this point, it's more or less feature-complete, though I haven't yet fiddled with data interchange with Magic.exe - the whole point of the exercise! - and I've only given it very cursory testing. Everything seems to work, though, and I'm not going to be able to spend any significant amount of time on it in the next week and a half, so now seems like a good time for some wider testing.
This will not work with any but the earliest versions of Shandalar-2015/16 - RT*, LT1-3, or TH1-2 - since they patch the dll at runtime, and so are tied to the precise version we've been stuck with since the last time Mok updated it. It should work with original Shandalar just fine (though current drawcardlib.dll/cardartlib.dll as included in BFZ and later Manalink patches will not), as well as the standalone deck.exe and any version of Manalink.
Known differences from the previous deckbuilder:
- These are all pretty underwhelming | Open
Please assume anything else is a bug, and report it in this thread. You'll definitely want to keep a backup of your old DeckDll.dll for when this one inevitably proves deficient. Testing against compatible versions of Shandalar - see above for which - would be particularly welcome, since I haven't done that at all yet. (There's full installation archives for some of the RT versions floating around on freakin' Youtube. Some of them even bother to point back here.)