Magic Data

[friarsol]

Hey Arch,

Looks like Gatherer has been updated.

http://gatherer.wizards.com/Pages/Searc ... anced&set=[%22Return%20to%20Ravnica%22]

[Arch]

Updated. Get it.

[Snacko]

Any chance to get other languages support ?
This would help immensely as it's the best formatted gatherer output available, without going trough hoops to get it myself.

[Arch]

The short answer is that the chance is about 'medium'.

I've thought about it in the past but since I am (was) focused mostly on the mechanics of it all there was no real reason to dig into it.

There are a couple of things that would need answers.

Data source

I tried clicking around on all the language related stuff I could find in gatherer but at most I managed to change the card-name and flavor-text. Is there some button I'm missing or does gatherer not have this information?

There are other sources such as magiccards.info that seem to have a more complete archive. Though not completely complete. Sometimes it lists languages it does not have translations for.

Language barrier

I, obviously, don't know these other languages so it will make it harder to maintain properly. A bigger problem though seems to be that there's possibly not a 1-to-1 mapping here. The typeline 'Elf Druid' turns into 'elfe et druide' in french for instance.
The rule-breaking (that is done for things like 'Trample, Haste') would of course not work as well.

Project objective

The original objective of this project was to get the 'raw' data for magic cards. Essentially things that a rule-enforcement engine would be interested in. I'm not really happy with having flavor and artist information integrated into the same structure. The text-data doesn't really belong in the same package either.

If I'm doing this I would definitely look into moving all of it to a separate project. This applies doubly if the data is not available in gatherer. The structure of the data should be updated to accommodate the thing I've mentioned above as well.

---

The biggest question for me is were to get the data. With that much of the other stuff will probably fall into place by itself.

[Ark]

I get all the foreign data with GathererDownloader, you have to click on "Printed" instead of "Oracle" on Gatherer's website to see most of the other language, which makes sense, because the definition of the Oracle text is the English version, even in other languages.

[silly freak]

Yeah, other language text is only "printed", so older versions of cards do not get updated rules text in foreign languages. If you tackle it, my suggestion would be to start with English "printed text", which would have to skip any processing that needs rules knowledge, e.g. that "Trample, Haste" are two separate abilities.

But it's your decision whether that's in your project scope

[Arch]

I get all the foreign data with GathererDownloader, you have to click on "Printed" instead of "Oracle" on Gatherer's website to see most of the other language, which makes sense, because the definition of the Oracle text is the English version, even in other languages.

Thanks. Printed does make sense.

Yeah, other language text is only "printed", so older versions of cards do not get updated rules text in foreign languages. If you tackle it, my suggestion would be to start with English "printed text", which would have to skip any processing that needs rules knowledge, e.g. that "Trample, Haste" are two separate abilities.

But it's your decision whether that's in your project scope

Yeah, didn't consider english originals, guess that should be included as well.

[Arch]

I decided that I should probably add languages. I've had a look at what needs to be done to add them. Unfortunately it includes a bunch of really boring code that must be written. This means that it'll take a lot of rainy days for me to complete. The current estimated delivery date for this feature is Dec 31st - give or take a few months.

[Snacko]

Better late than never, i have a crude way to extract those, however your output is so much nicer and easier to work with.

[gumgod]

Trying to build my own site. I was able to get quite a bit of information out of gatherer using the text spoiler option and parse it myself, but a few things I just can't find on Gatherer like the Deckmasters boxed set and some promos/rarities/oddities. What is a good source to get this information?

[Arch]

This is a beta version of the data with languages included. The old format has changed a bit as well. Was planing to drop this with Gatecrash but figured that giving some time for feedback and possible updates to dependent applications might be good.

Download beta

Items marked 'todo' are things I know about that will be fixed in next release.

card.xml
- Basic lands still have rules. (todo)
- Cards not in gatherer have not been added from the old app. (todo)

meta.xml
- Moved flavor text from meta.xml to language_*
- The card name is now an attribute instead of an element.
- The set is given as the name of the set instead of the set-code. Set-codes, as they currently exist, are not really good identifiers. They could be consistent within a single release. The set-name is now also an attribute.

setinfo.xml
- Card information (number of cards) have been removed as this is derived data.
- Just noted that Timeshifted entry is messed up. (todo)

language_*.xml
- New files.
- Contains language specific card data. There's an entry for each card and that in turn contains an entry per set.

format.txt
- Not updated or included. (todo)

[Marek14]

Master of the Wild Hunt Avatar seems to have an extra "green" when describing the last token.

[Arch]

Master of the Wild Hunt Avatar seems to have an extra "green" when describing the last token.

Fixing

[Snacko]

You have to fix type bug in translated cards, where they always add - even if it should be there:
Chinese Traditional/Chinese Simplified "～ - " to "～" notice there is no space here
French ": - " to ": "
Japanese "― - " to "― "

There are some more bugs I don't remember right now, but as soon as I recall them I'll post.

There are also some problems with flip cards ex. Akki Lavarunner. It's 2 times and the same data is in normal and multi tags.

I suppose something more will pop up, but overall good work to get it all so well formated.

Feature request:
1) add number of cards to setinfo.xml, what I mean that it's the same as yy in xx/yy wizards numbering.

[Arch]

Fixing:
Remove '-' from languages for certain languages.
Lavarunner. This effects most multi-part cards.

Not going to add card count. Get this by counting occurrences in meta.xml. This was part of the old releases and it always slightly bothered me since it's derived data. The only reason I can find to add it again would be if meta.xml is not correct. (That would indicate another problem though.) I tried it with a couple of sets and got the right number. (Only seems to be the multi-part problem that's currently messing it up.)