DTK - Preliminary manalink files (Updated 19th July)

[Sonic]

Seemingly the stampede to take up the post of Manalink files editor hasn’t yet materialised.
So as I've had a few spare moments here and there I thought it might be worth at least finishing off my tenure as file editor by completing the last set in 'Khans of Tarkir' block and creating some preliminary manalink/ct_all files files for 'Dragons of Tarkir'.

Whilst the files are fully formatted to be simply pasted into the current manalink/ct_all files, they only contain the basic card data for the new cards in the 'Dragons of Tarkir' set (the reprints are handled separately – see below), none of the manalink specific data has been added (code pointers, etc.). Also, as I'm unaware of the cards that will be activated in the patch release, the ct_all file contains entries for all the cards in the set; the non-activated card entries will need to be removed once the activated card list has been finalised for the patch release.

Obviously, being part of previous sets the reprints for 'Dragons of Tarkir' are already listed in the main manalink file. So it's only necessary to add the set rarity codes for these cards to the 'Dragons of Tarkir' expansion column in the existing manalink file before the new card entries are added to the file. I've also added the card rarity codes for 'Duel Decks: Elspeth vs. Kiora' and 'Modern Masters 2015', these columns can be added at the same time in order to activate these sets.
You'll find the completed columns for all three sets in the 'Expansion Reprints.ods/xlsx' files.

I've also included the edited artwork for the entire 'Dragons of Tarkir' set, including the new artwork reprints. The reprint image files are pre-numbered, but will need to have their 'Num Pics' entries in the Manalink file updated.
When compiling the artwork for the patch only include the images for the activated cards in the final release. Take it from me, having a whole bunch of unused artwork in the CardArtManalink folder is a very quick way to loose track of the artwork.

Updated Download 19th July
Corrected the files and also added the emblem and token entries and their artwork. That should keep you all busy for the time being.

http://www.mediafire.com/download/d39c7 ... Prelim.zip

All the best... Sonic

[Aswan jaguar]

Sonic there seem to be cards missing from Dragons of Tarkir set from your Manalink files,like Ainok Survivalist,Anafenza, Kin-Tree Spirit,Ancestral Statue and others and their numbers are skipped from ID value,too.eg Ainok Survivalist 's ID should be 14666 and this is skipped in the files that go from 14665 to 14667.

I tried to insert Dragons of Tarkir set to Manalink.csv to see how much work is involved few days ago and most important if I could manage to do it with my less than basic knowledge of editing files and without having to insert everything manually.But it took me very much time just to gather info and insert full name,coded card,short name,cost text (without the | symbol.How do you insert this automatically? I tried several functions without success),expansion and types text only.

Since noone that is able to edit files has stepped up to fill your shoes can you please make a guide for the whole work that needs to be done,sources of magic data,tools,functions you use for different tasks and whatever else is needed so maybe I or others can help or split up the work and help Korath.

By the way how much time did it take you (an expert in editing files) to edit the files for a new set?

[Sonic]

Sonic there seem to be cards missing from Dragons of Tarkir set from your Manalink files,like Ainok Survivalist,Anafenza, Kin-Tree Spirit,Ancestral Statue and others and their numbers are skipped from ID value,too.eg Ainok Survivalist 's ID should be 14666 and this is skipped in the files that go from 14665 to 14667.

I tried to insert Dragons of Tarkir set to Manalink.csv to see how much work is involved few days ago and most important if I could manage to do it with my less than basic knowledge of editing files and without having to insert everything manually.But it took me very much time just to gather info and insert full name,coded card,short name,cost text (without the | symbol.How do you insert this automatically? I tried several functions without success),expansion and types text only.

Since noone that is able to edit files has stepped up to fill your shoes can you please make a guide for the whole work that needs to be done,sources of magic data,tools,functions you use for different tasks and whatever else is needed so maybe I or others can help or split up the work and help Korath.

By the way how much time did it take you (an expert in editing files) to edit the files for a new set?

Sorry my bad - I'll need to construct the files again. That's the problem with not having more than 25-20 minutes here and there to do something. You tend to loose track of where you were.
I seem to have lost part of the manalink file when I copy and pasted it to the Ct_all edit file with a filter setting left on. Duh!

As I explained to gmzombie when I first mentioned I was going to give the editing a rest. I just haven't got the time at the moment to explain how to use my spreadsheets for editing the files. I'd already started on the DTK files before I decided to hand on the editing post - and it's taken me this long just to find the time to finish them. That's how bad the available free time has become. So finding the time to write instructions is going to be as bad.
I'll get there, I just don't know when.

I'll try and redo the current files ASAP.

Best... Sonic

[Korath]

If there's an easy way to get a card's raw Gatherer data (besides fetching the webpage and then trying to parse it), I can probably put together some perl scripts to convert it into a manalink.csv and preliminary ct_all.csv line.

[Sonic]

If there's an easy way to get a card's raw Gatherer data (besides fetching the webpage and then trying to parse it), I can probably put together some perl scripts to convert it into a manalink.csv and preliminary ct_all.csv line.

Getting the card data isn't a problem. I've always used Gatherer Extractor - viewtopic.php?t=4021 as this will output the basic card data in a number of different formats.
If you could automate the preliminary file construction without resorting to using spreadsheets this would be half the battle. Because it would greatly open up the number of people who could take on the file editing task with only a basic understanding of spreadsheets to do the remaining manual data input where the information in the rules text needs to be manually converted into manalink specific data.

[Aswan jaguar]

I tried a couple of tools for magic data but they weren't very helpful.I have never heard about Gatherer Extractor,so thanks for that.I will use this tool.

[Korath]

I'll take a look once I manage to get the next version of Shandalar released.

[Sonic]

I have never heard about Gatherer Extractor,so thanks for that.I will use this tool.

I am surprised you've never come across it. Definitely the best of the best, its pretty well indispensable when it comes to downloading the new card data - fairly intuitive to use as well.

One of its tricks is to be able to manually setup the delimiters for the exported data. So you can export the rules and flavor text fields pre-formatted with the manalink specific new line code '\n'.
Other useful functions are being able to calculating the cards current tournament legalities and selectively choose which data fields to export. You don't need to export the tournament formats we don't use in the game or the extra languages or card prices for instance.)

[Korath]

I'm at the point where I can autogenerate 90% of the manalink.csv line from Gatherer Extractor output, including hack and sleight marks, but not damage/effect/legacy text. ("Plains" will always need human disambiguation, too; there's no way to tell whether it should turn into "MOUNTAIN" or "MOUNTAINS" if it's Magical Hack'ed.) Two questions, though:

• Do you use any sort of automation to get the AI deckbuilding values (columns "Mod Direct Fire" through "Inflatable"), or do you do those manually?
• How do you generate the "DB Card Type 2", "Num of DB Abilities", and "DBA1" through "DBA8" columns?

[Sonic]

I'm at the point where I can autogenerate 90% of the manalink.csv line from Gatherer Extractor output, including hack and sleight marks, but not damage/effect/legacy text. ("Plains" will always need human disambiguation, too; there's no way to tell whether it should turn into "MOUNTAIN" or "MOUNTAINS" if it's Magical Hack'ed.) Two questions, though:

• Do you use any sort of automation to get the AI deckbuilding values (columns "Mod Direct Fire" through "Inflatable"), or do you do those manually?
• How do you generate the "DB Card Type 2", "Num of DB Abilities", and "DBA1" through "DBA8" columns?

Sleight and Hack were done via a manual 'search and replace' on the original Gatherer Extractor output (xls format). Checking for obvious things; like a named color or basic land type in the rules text isn't simply part of the cards name.

The rules for the Hack codes are:
|H - "a swamp", "an island", "a forest", "a mountain", "a plains".
|H1 – plural - "swamps", "islands", "forests", "mountains", "plains".
|H2 - singular - "swamp", "swampwalk", "nonswamp", "non-swamp" etc.
(Note: “plains” can use either |H1 or |H2).

Damage/Effect/Legacy - all manual entry.

Never touched most of the AI deckbuilding values apart from the abilities columns - AI Flying, AI Banding, AI Trample, AI First Strike, AI Wall, Num of DB Abilities and the DBA1-8 entries. All had to be done manually by filtering the Rules Text for each named ability individually, and then checked for the exact usage of the filtered name in the text. I then used these entries to generate the abilities data in the ct_all file.

[Korath]

OK, on the DB abilities, let me rephrase.

I can parse, say, White Shield Crusader's rules text well enough to tell that it has protection from black. I'll maybe be able to parse it well enough to see that it can grant itself flying. How do I translate that into "DB Card Type 2" = 12, "Num of DB Abilities" = 2, "DBA1" = 12, and "DBA2" = 24? Are the numbers documented anywhere?

Two other examples to demonstrate why I'm having so much trouble with this, separately from just having to figure out what the numbers mean -
Pixie Queen is "DB Card Type 2" = 0, "Num of DB Abilities" = 2, "DBA1" = 4, "DBA2" = 24. So I still have no idea what DB Card Type 2 does; and I'm guessing "Native (Color) Ward" ends up as 12, "Native Flying" 4, and "Gives Flying" 24.
Vodalian Knights, on the other hand, is "DB Card Type 2" = 0, "Num of DB Abilities" = 2, "DBA1" = 3", "DBA2" = 4. In the deckbuilder, it shows up for both "Native First Strike" and "Native Flying".

So one of the three cards is wrong, and it's not immediately apparent which - is it the convention for "Native" to mean "always has it", or "can grant it to itself"? And similarly for "Gives", is it specifically "gives the ability to another card", or should it include "can activate to give this ability to itself"?

[Aswan jaguar]

We have 2 different Guides to CSV editing done by Orion79 and by CirothUngol they are on wiki/Manalink2/Editing the .csv files http://www.slightlymagic.net/wiki/Guide_to_CSV_editing
& http://www.slightlymagic.net/wiki/Manal ... Definition

From both guides and from the position on exe just after card type and their context you can see that the values seem to be mostly irrelevant from deck builder values and abilities also Jatill said that DB stands for database in this case.
Orion79 said about DB Card Type 2:
DB Card Type 2: This category gives precious advices to AI about how to use cards. It may also have occasional consequences on deck editor.
also mentions: Since some cards may have multiple effects, when you code new cards you have to choose a priority. All previous lands have "provides mana" even if they have other effects, while X casting cost is often selected if there are no other effects.

So in your example
White Shield Crusader has DB Card Type 2 =12 because it is an inflatable creature. The others have DB Card Type 2 =0 because they don't have any of the special abilities values on this field.

Vodalian Knights of course has also wrong value "DBA2" = 4 instead of 24 as it is not a native flier but gains the ability.

I have all values of Manalink.csv sorted by values so the cards with the same values are all grouped together so they can easily compared and conclusions can be easier deducted. I used values from the earlier CSV file found and I name which ones were coded by Mok and which were coded later as those values could be wrong. I am attaching Manalink.csv file with DB Card Type 2 values sorted in case you want to check on your own.

[Sonic]

Actually the preliminary files I posted have some of the entry values listed in a roll-over drop-down comment box in the column headers.
If the header has a small red box in its top right corner it has a comment included.

Examples:

DB Card Type 2
0 - N/A
1 - Ante
2 - Control permanent
3 - Counter spell
4 - Direct damage
5 - Discard
6 - Draw card(s)
7 - Gain life
8 - Damage prevention
9 - Land destruction / alteration
10 - Mana producers
11 - Game stallers
12 - Creature enhancers / inflatable creatures

DBA Abilities
0 - No additional abilities
1 - Banding
2 - Desertwalk
3 - First / Double strike
4 - Flying
5 - Forestwalk
6 - Vigilance
7 - Islandwalk
8 - Legendary landwalk
9 - Mountainwalk
10 - Plainswalk
11 - Poison / Infect
12 - Protection from black
13 - Protection from red
14 - Protection from white
15 - Haste
16 - Rampage
17 - Regeneration
18 - Deathtouch
19 - Swampwalk
20 - Trample
21 - Reach
22 - Banding granting
23 - First strike granting
24 - Flying granting
25 - Forestwalk granting
26 - Vigilance granting
27 - Islandwalk granting
28 - Mountainwalk granting
29 - Plainswalk granting
30 - Protection from artifacts granting
31 - Protection from black granting
32 - Protection from blue granting
33 - Protection from green granting
34 - Protection from red granting
35 - Protection from white granting
36 - Haste granting
37 - Rampage granting
38 - Regeneration granting
39 - Deathtouch granting
40 - Swampwalk granting
41 - Trample granting
42 - Reach granting

Under the AI Abilities entries:
1=Has ability
2=Gains/gives ability
3=Special (no idea on this one)
4=Removes an ability (either temporary or permanent).

[Korath]

OK, hrm. csv2dat.exe is difficult to analyze, but DB card type 2 gets written to DBinfo.dat, so it's unlikely to be end up anywhere else too. It's definitely not in the data exposed to the main game, either during a duel or AI deckbuilding.

So far as I can tell, nothing reads DBinfo.dat except deckdll.dll. deckdll.dll ignores the DB card type 2 data almost entirely - it only cares whether it's 10 (Mana producer) or not, and only uses that to implement the right-click menu for the land icon. (Which doesn't seem to work very well. I can, for example, get it to display only those artifacts that produce mana, but can't get it to display Llanowar Elves - which also has 10 in DB card type 2 - without displaying all other green creatures too.)

So I'll output a 10 there if the card ends up flagged as a mana producer, but otherwise, I'm not going to worry about it. If someone expands the deckbuilder, we can do something about it then.

[Korath]

Output from converting DTK is attached. It hasn't been touched by human hands except to remove the already-existing cards (my converter merges directly into Manalink.csv and ct_all.csv).

Besides the new cards, it updated Battle Mastery's and Kindled Fury's rules texts, since they were reprinted in DTK; and added the flavor text to Spidersilk Net (which it only does when there's no existing flavor text). It also added all the reprints to the DTK set, and (for what difference it makes; I've never seen any use for them) added the following cards to various format-legality pseudo-sets:

• Pacifism to (B) Block, (T2) Standard
• Mind Rot to (B) Block
• Dragon Fodder to (B) Block, (T1.X) Extended, (T2) Standard
• Explosive Vegetation to (B) Block, (T1.M) Modern, (T1.X) Extended, (T2) Standard
• Duress to (B) Block, (T2) Standard
• Battle Mastery to (B) Block, (T1.X) Extended
• Kindled Fury to (B) Block, (T2) Standard
• Negate to (B) Block
• Spidersilk Net to (B) Block, (T1.X) Extended, (T2) Standard
• Evolving Wilds to (B) Block
• Gravepurge to (B) Block, (T2) Standard
• Death Wind to (B) Block, (T2) Standard
• Ultimate Price to (B) Block, (T2) Standard
• Summit Prowler to (T1.X) Extended
• Tormenting Voice to (T1.X) Extended

I used Gatherer Explorer's "legality_Legacy" field for the (T1.X) Extended column, for lack of better ideas.

It shouldn't be difficult to tweak the converter such that it doesn't do anything except update rules text and add flavor text.

The output still needs human post-processing; in addition to the |H1plains vs. |H2plains, AI deckbuilder values, and effect/legacy text issues already mentioned, it doesn't add tokens, emblems, or altered-type versions of cards (e.g. animated lands, noncreature Theros gods); set Num Pics to anything except 1; set cards that target spells on the stack to be interrupt speed; or pick particularly intelligent values for the crucial Activate/Play Before/After Combat/Blockers AI columns. The ct_all output also still needs most of its flags set when a card is programmed, though it does detect hybrid and planeswalker cards for CC[2], and can see most forms of inflatability.

Still, it does at least 90% of the work here. Any takers for the remaining 10%?

---

Edit: I'm midway through updating rules text, confirming each change manually. Trying to do it all at once was a mistake; Gatherer Explorer makes a 100-megabyte xml file if all cards are included, and Text::CSV runs out of memory. Cutting it down to something that can be read without dying was an adventure. It's turned up some useful corner cases, though, and highlit some more cases that'll probably always have to be dealt with by a human, such as making sure the reminder text for Magical Hack-like and Sleight of Mind-like cards themselves isn't hackable or sleightable.