Manalink C/ASM Dll

[Ghost]

Ghost to me that sounds like a great idea..although im no coder by any means although i do hack stuff up and attempt anything i think i can do.

Ya, the good thing about this is that we can bypass our utter ignorance about the code. In our current perspective there is a black box of code accessing another black box of cards (PROBABLY, but not necessarily, simply an array), the idea is to be the "man in the middle" and serve either without they noticing it, that is, without breaking any boxes =)

We dont necessarily need this, but it is safer to avoid mysterious crashes later, since we can simply append code and not mess with any already there (only changing [] to () in a C context, or stacking the parameter and adding a call in asm. btw in C we wouldnt even need to change the name of the vector if we named the function the same =) but I digress).

I have good news though, the limit being 2000 means the ID variable has at least 11 bits.. being greater than 256 it is larger than 1 byte, but since it's highly unlikely that they broke C types for any reason (like memory economy or using the upper bits for something else) it means that variable is AT LEAST a short int, i.e. it is at least 2 bytes in size, meaning it can address up to 65536 individual cards! Which is like 5 times as much we need =)

This is great not only because the infrastructure is already in place (differently than the Unicode problem with ASCII), but increases the chances of the current approach work without breaking anything or, if it is indeed a simple matter of redimensioning an array, that it wont have no effect whatsoever (if the array was 256 and you simply increased, the variable wouldnt be able to go beyond 256 anyway, and other parts of the code might expect a byte and break). I still dont recommend trying to fix that C code, but, since we got an asm version working without problems it seems.. then perhaps pholosopher can change the values of 2000 he already found (use the c code as inspiration) but change the asm code. Allowing to test right away. BTW do you have a version with 2001 cards to test?

the one thing im not sure about and maybe some of the coders can chime in is if the AI reactions and movements are coded with the cards or not. not being a coder i dont know.

Since Im new to the project I can't really assert much, I can however give an informed guess based on what I know about AI. I didnt read much of the forum yet, so you guys might point me to appropriate threads where I can learn more, but it seems there were complaints about sloppy card coding in relation to what you say, that is, how the AI uses such cards. It does make sense, but from what I know of AI, this would be a very limited technique. Yet, since the game probably had limited development time devoted to the AI (also based on Gargaroz compliments to it ) it could very well be the ONLY technique employed! But we cant really know before studying the code, we can only guess. The AI could use this limited technique as in "oh, this creature has banding, Im banding"... i.e. every card mod would enable a new calculation comparing other actions by the AI. But this seem to be located in a bit pattern in some struct representing the card, not in the code I've seen around, which seem to already use those attributes for more complex stuff)
But I find VERY HARD to believe the AI could really understand any card code AT ALL, even if you coded it very simply. My guess is that every card that has any fancy mechanics would have a hardwired little rule somewhere (which could be in the card code itself, but from what I've seen of card codes, it's probably on the AI code, though perhaps I only seen people's code so...) saying stuff like "oh, you got channel. Set a goal for having life+mana>opponent life then cast fireball". Do you agree it would be much easier than make an AI that really figured what each new code would make? It would just need a table of card ID's (which might be one of those arrays pholosopher found btw =D) mapping into actions/goals.
Another way to do AI would be to simply play the card on what I like to call "the AI's imagination" and simply compare the result with multiple such imagined scenarios based on each possible action/card play the AI has at hand. But this wont be influenced by messy card programming either, since it will just wait for it to complete, the max it could cause would be slow down the pc, but nowadays it wouldnt make a difference and even earlier, the card code wont vary much in size or complexity anyways. The advantage of good coding is being readable for us humble humans, having 12K+ cards to program in is challenge enough without trying to show off coding virtuosi at each and every one of them. =) We need to be able to fix any bugs quick so we can continue to program them quick, thus we need clear code, not messy but not optimized either, which only benefits the machine really which should be our slave not the opposite.

but i also had an idea to eliminate the clutter of card code in the program exe and that was to find all cards still coded into the exe and move them to the DLL maybe even a new dll that is just original magic cards or just added to the current dll file too. either or really and that would eliminate code that we would have to go through..just a thought.

Exactly, I agree completely. Thats what I meant when I said "prune whole sections of the search space". I didnt quite mean it physically move code =) but work either way, might help unclutter but also may take away addresses that we need to understand the rest, so it's a trade off. I mean, imagine there is a function that refers to that part of the code, if we moved it a month earlier, we might think it's referring to something else and confuse our already fragile understanding. My idea is to do it non physically, color coding stuff, or, if you want to get fancy about it, even make a conectome of the code http://3.bp.blogspot.com/_CNrCaeifl0s/SVUDDT0mbqI/AAAAAAAAB38/wWmoJhwbJg0/s400/chaos_cancer.jpg
hehehe Now expand that idea to stuff we would rather postpone understanding, like the AI, Shandalar, monster movement, dungeon generation, deck randomization and sorting, programming chores.. we could, in theory, color code those out(or take note of start/end address) as soon as we notice some section is part of one of those and eventually isolate our priority, the 2K limit. And is something, I believe, anyone can help which means massive parallelism/many eyes looking.. But Im with you, thats pretty much the same thing.

Ghost.

[Gargaroz]

As for the AI, it seems that the program "knows" what a card coul do checking a string of numbers known as "Reserved information" on our editor.
Also, we know that the AI uses some values from the CSV id, but we're not sure it uses them all.
However, I think the best way to improve AI, when we'll reach the point we could do that, is rewrite it from scratch and use a system similar to MagArena, that is very impressive, although limited on what cards it's able to understand (no multi-effect cards like Planeswalkers, for example).

[gmzombie]

ok so putting our heads together on this, (which i will again remind peeps in no coder more of you could say a hacker with some knowledge of the system) i think the first step would be to make sure we are on the same manalink revision. i mean if anybody needs updates i have them from when it first started back in the mok days. second we would all i think have to have the same program to use when looking at this code due to some programs changing the start point or some stuff if i recall. Also the wiki will be a great resource to add this too after things get rollin, and finally how. we can have alot of eyes looking but if the peeps dont know what there looking at..(ie me) im just guessing and that isnt good. im all for this to really get started on finding the solution to this 2k problem. i will just need some guidance in the begining on what to do.

[Ghost]

As for the AI, it seems that the program "knows" what a card coul do checking a string of numbers known as "Reserved information" on our editor.
Also, we know that the AI uses some values from the CSV id, but we're not sure it uses them all.

Hmm, this seems more evidence to the id->action hypothesis =) but Im aware it doesnt say much at this point. It would be useful if we could gather a repository of all those little bits of knowledge and where did we get it, like, was it purely empirical (programming different cards and playing the game) or there is some basis on code, etc..
Oh, I believe I know what that 3rd vector is probably for.. the loaded images of each card =)

However, I think the best way to improve AI, when we'll reach the point we could do that, is rewrite it from scratch and use a system similar to MagArena, that is very impressive, although limited on what cards it's able to understand (no multi-effect cards like Planeswalkers, for example).

Yes, I agree, there is also MagMa and just found out about Forge too.. I do like (as I guess all here do =)) the Planeswalker front end though =) it would be nice if we could attach the best of those projects (perhaps we could even merge them in the future) and call them as a function from inside the game, hopefully even in Shandalar. I do like AI programming, we could try to improve it or at least add more cards to them which seems to be where all those projects are most at fault, but I would only delve into it after Im comfortable with my planeswalker version. =)

ok so putting our heads together on this, (which i will again remind peeps in no coder more of you could say a hacker with some knowledge of the system)

No problem, this will be like a puzzle anyway, very (VERY) slow in the beginning and exponentially catch speed as progress is made (the less pieces left, the less of a combinatorial explosion, thus the easier to place them in).

At first we will be overwhelmed by functions (hopefully we at least have a reasonable C code to read) we have no idea what they do, with variables we will only have addresses for names. Eventually we will begin to name stuff (like card ID will probably be the first =)) and even comment the code. This also should be updated in every volunteer's version and in the wiki.

i think the first step would be to make sure we are on the same manalink revision. i mean if anybody needs updates i have them from when it first started back in the mok days.

The less stuff added the better, I even have the original cd if needed, but I believe there is a 80MB download floating around the net somewhere, or at least were. That would be best, as vanilla as possible. Most of the work will be reading the decompilation done (or redone on that version, which btw might generate less errors?=)) by pholosopher commenting on what you figure of any snippet of code you happen to be assigned to (even if its nothing at all, like "this function just take that variable and return a calculation on it"... Once we know where such a function is called, we can retrace "that variable" to some name we might figure somewhere else.

second we would all i think have to have the same program to use when looking at this code due to some programs changing the start point or some stuff if i recall. Also the wiki will be a great resource to add this too after things get rollin, and finally how.

Yes, in fact we could start there, placing all this protocols we are discussing and the goal, perhaps even the mysterious code we have now so anyone can read what we are up against lol

we can have alot of eyes looking but if the peeps dont know what there looking at..(ie me) im just guessing and that isnt good. im all for this to really get started on finding the solution to this 2k problem. i will just need some guidance in the begining on what to do.

I'll be glad to explain what I can, but rest assured, it WILL be guessing most of the time =) specially in the beginning, but remember this is all simple mathematical relations from the level we are looking from, so even translating that into English will help. Once a more experienced coder reads your translation it will be much faster to figure stuff like "oh, this can only be a sort algorithm" or "that interrupt is loading files, probably images here.." etc..

Finally, it would be nice if this program we are going to use could help us treat that code prior to beginning. For example, instead of having address for variables like i said, we could rename to variable_1, variable_2, etc in all places they appear, also with functions unless you have no problem with sub_Hex#####() which it seems to do (variables seem to be v1, v2, v3.. which is fine).

Also, I would REALLY like pholosopher to try and change the asm code directly and increase the array to 2001 and test =) before we begin hehehe... do you guys have 2001 cards ready, and not able to see the 2001st?

Ghost.

[gmzombie]

with reply to the original disc we cannot use that due to mok changing everything the first time. also for that i believe we should look at the first time the coders added the 2 manalinkex and eh.dlls. reason being is that a guy named skymarshall and harry pitfall was able to crack the array the first time and up the cards. see we were running out of room before and i believe it was around 1400 or so. they actually added the jump to the first dll manalinkex.dll later came the eh.dll and that was done by lonefox i believe which made it possible to write cards out in c.
im thinking ollydbg or ida with hex rays disassembler for the program.if i remember correctly someone already tried to do the 2001 card and it broke the game. i dont believe it could be loaded. i could be wrong though

[Ghost]

Since we dont know what led them to such a jump, then it's one more reason to try with both versions. Pholosopher is then looking inside an already modified version then why would they put 2000 arbitrarily there in 3 places without understanding (or explaining why) they did it? If we cant ask we must find out for ourselves either way, I believe the next step then is to set that wikipage with our tools, protocols and versions. The simplest ones, which can be both mok and the original planeswalker (also to have a base of comparison for what they did, since they arent here to explain, their modifications which have helped upto here might complicate understanding though).

We decompile both and upload the code, both C and ASM, and briefly explain how to do it, after we take a glance at it, we can perhaps start assigning sections or functions for everyone.

I'll be available to help with the wiki if needed.

Ghost.

[gmzombie]

http://home.comcast.net/~gmzombie/magicC.rar
this is moks 2.0 nothing added to it c decomp from IDA with hex rays disassembler. standard options. (edit) im looking at this code right now and there is alot of this
(//----- (0041AF50) --------------------------------------------------------
#error "41AFA5: invalid basic block (funcsize=31)")
but different sub routines...what gives how can i make that error go away and actually decomp this sub routine?

http://home.comcast.net/~gmzombie/magicpe.rar
this is same version but in asm form from PE explorer. it seemed to have it more organized than ida so i used that. i can give both if needed. default options in pe explorer.
Hope this helps as a start.

[Ghost]

Hmm, does this program has a manual or faq?

I had a quick look, a lot of the asm code seems dedicated to cards, like 66%, ~400.000 lines.. (though many of those are just the name strings "db". On the C code they seem to be used by functions closer to the beginning)
On the C code you can find a lot of functions dedicated to netplay (i already got some of those error strings hehe), and then what seems threaded functions (what seem to be semaphores with stuff like "while(result)return result").
There are lots of functions which seem to relate to the turn phases.. the cool thing is that they left logging functions around =) they pretty much commented the code for us! which also let me identify what must be some sort of "verbose" flag.. dword_790640 & 2 , that is, the second least significant bit of that dword there.. set that flag and you will probably get a file describing which functions the code is entering!

Even though this was just a quick, ~half hour analysis of spaghetti code =) Still, it doesnt seem we are at a stage we can begin assigning functions for people to look at.. any progress anywhere is welcome for now..

Until around line 7000 seem just initialization chores (stuff like while(v11--)set a bunch of variables) which comes after a lot of declarations.. i.e. variables, functions, constants..

Yet, I could not identify anything resembling AI code nor Shandalar related stuff.. this is probably only the net duel code..

Ghost.

[gmzombie]

http://home.comcast.net/~gmzombie/DUEL.rar
asm listing of duel.exe from pe explorer. possible ai in there?

http://home.comcast.net/~gmzombie/DECK.rar
deck.exe pe explorer asm listing
http://home.comcast.net/~gmzombie/DeckDLL.rar
deckdll.dll asm listing from pe explorer

[Ghost]

Hmm.. Duel.exe seems to contain a lot of shandalar related stuff.. and what first seemed a few strings of AI is apparently more of a tutorial of some kind, except for the video tutorials, I dont remember that in the game. even taunts seem to have been included lol.. (and not the ones after you win or lose a duel, seems like in duel stuff)
Funny, they can include sets like chronicles, legends (without the golden cards) without maxing the card count, in what seems to be called collections in that file and/or "card pallete" (wtf with the hardcoded path 'D:\Newmagic\sources\NedCard\Palette.c', LOL).
But specially the 5th edition, included without regard for cards from Fallen Empires, Ice Age, and Homelands which wasnt included in the game.. unless you guys added it?
http://en.wikipedia.org/wiki/5th_Editio ... _Gathering)
And also probably without regard for the different artwork.(which is ok for us that have no access to the code or compilation but seem quite unprofessional on their part).. not to mention the ruleset had "numerous changes" on that edition...
This asm seemed to contain more complex card codes.. which I havent found in the other file, but then I found this: D:\Newmagic\sources\sid\Ai.c Those might very well be the AI code to use such cards more cleverly =)
Hmm, did you guys added Fallen Empires? look "SSZ004F3370_CARDBK_FallenEmpiresLand:
db 'CARDBK_FallenEmpiresLand',0
"
Yet, for the most of this file, it was quite hopeless unless we get to decompile to C too, which speed things up considerably.

DECK.asm is pretty small I believe its just the interface and it calls deckdll.dll for most of the work. But haven`t you guys already understood that stuff? I mean, dont you have a custom deckbuilder? If not, it sure helps a lot to isolate the code dealing with card collections and probably with the limit on card quantity for those collections..
DeckDLL.dll would also be really helpful if we can understand it, it contains all filtering functions of the deckbuilder, by collection, type, rarity, even artist name.. in case it wasnt already understood. Those are the functions to study if you want to figure the card struct format...
Again, it would help having a C decompile, though as expected most of that file are windows runtime api stuff, dialog boxes, popup messages and menus, open file dialogs, scrolling, etc...

Ghost.

[Ghost]

Where is Foolosopher, I would like to know what he thinks of all of this. And also the following:

I found four tables with that number of elements in magic.c

Line 1256: extern int dword_56260C[2000];
Line 1691: extern char byte_5B924C[2000]; // idb
Line 56820: int v8[2000]; // [sp+18h] [bp-1F44h]@5
Line 60633: int v20[2000]; // [sp+4C1C4h] [bp-1F50h]@7

The problem is with v8 and v20 which have a lot of occurences (more than 1500 each),

Thats because the decompiler resets the variable count for each function ("sub"procedure), you just found a particular LOCAL variable (the 8th local variable from some function and 20th declared variable under another particular function) that happens to be related to the card limit, thats what I mean by being risky changing a constant that might be depended upon by many hardwired places, for example the 7d0 you looked for, ok, there isnt any 'tables' declared to that size, but you will notice that there are LOTS of memsets and memcopies that use that exact constant, which means that if we simply raise the limit, those 'tables' will not be initialized properly, and as i said, mysterious crashes will ensue, or crazy bugs..

That second vector you found, it doesnt exist on my version (got from gmzombies posts above), which probably means its already a change made by Mok or later... Thats what i meant when i said their changes might complicate our understanding on this. Thats why i suggest working with the original, until we understand the original limit. Unless someone can explain what they did to get where we are today. Otherwise we should have 2 versions each reaching one of the limits to be able to understand what they did (which might be very helpful actually).

One piece of good news though, the first vector you found is the same on my version and, most importantly, there is this extern keyword there, meaning it's declared and set (since it's promptly used there, once, and never else!) in some other file. If we can find that file, or where such a vector is declared WITHOUT that extern, that would probably be the place to change the limit. (tho we would still have all the initializations to fix).

Ghost

[gmzombie]

http://home.comcast.net/~gmzombie/deckdll.c
deckdll from asm to c standard options
http://home.comcast.net/~gmzombie/DUEL.c
duel to c standard options

[Ghost]

btw, that last variable foolosopher found, is in a function that seem to start the draft ("gauntlet") tournament, it calls another function passing that vector as the 2nd parameter (a2), and an int pointer as the first (a1).
That function seems specifically meant for loading the cards on a particular package, probably randomizing a booster first, for the sets chosen just before. Since it seems to be called only once, it's probably loading all cards from all those sets, perhaps randomizing happens later. But that function doesnt seem to account for the limit which means the calling function is the responsible for that, the 1st parameter a1 probably returns the number of cards loaded.

A first well informed test, change that variable you found, and check if the gauntlet can indeed load more than 2000 cards.. or better yet, change to 5 cards, and see what happens, since its randomizing and normalizing to the number of cards in the total #of boosters, it would never reach 2000... but if you can successfully limit it even further, then thats probably equivalent.

I'll check that v8(in your version) to see if it also calls that sub-procedure to load cards...

Ghost.

[gmzombie]

yea the deckdll file which i gave the c for is only a little bit known. in the original game there wasent an option for a new cardset so mok originally added the extra red button on the left. when he did this he recreated the whole file and thus changed the system so he could code it better to what he knew which was asm i believe. i helped some peeps change the .pic files of the deckbuilder to .bmps cause the game was already using .bmp's in it already but the code was in magic.exe. so the jump was added so that is also why you can no longer open up just the deck builder alone. it needs code from magic.exe to run the bmps. as for changing the deckbuilder there was only cosmetic differences changed. the fallen empires land and whatnot was already in the game just not enabled. fallen empires was later coded into the manalink.csv but never really coded except for some cards now.

[Ghost]

That v8 is on a function that seems quite simple, though the use of that 2000 vector is strange.
Its only called from 1 other function in the code, and only twice there, one after the other.
It seems to do the following:
Take a number(given by 2nd parameter a2, which that calling function always set to 11, is that even a booster card count?) of cards (which positions seem to be given by an index in the 3rd parameter a3 multiplied by 4, which means the card structure has 4 bytes (more probable) or its considering every 4 cards) starting from a random(0-4) position from that 2000 vector, and thats the strange part, the vector is never initialized, and it keeps cycling through only those 5 positions! (i.e. 2000 are never used)..
That seems to be half of the process used by that calling function, since the 2nd time it calls this function (again taking 11 cards) it does the same thing but this is done so it remembers where it stopped the last time.
I dont know where it might be used, perhaps shuffling the deck, but it does have a 60 hardcoded there =) and the only time you would arbitrarily limit deck size to 60 is in the construction stage LOL I believe thats how the AI "builds" it's deck for the gauntlet!

Ghost.