Gatherer Downloader App (v1.6.8) [hq pics / card database]

[woogerboy21]

If I redo the path/file structure option so it is a free entry text box, then I'd likely want to offer a {lang} kind of thing so that if you pick a different language from the menu, you can automatically have the other language images go to the proper folder. Of course, I'd probably want to offer {lang} for ISO 639-1 and {lang3} for ISO 639-2 (the one you'd want here probably).

I still need to know what the point of this "Core Sets" vs "Expansions" thing is, and if there is a 3rd or more choice that could appear there.

This way, you could just enter:
{settype}\{setname}\{lang3}
into the box for the path structure.

My thoughts for the folder structure flexibility was simple future proofing the application for any application that may use a deeper or more shallow foldering structure as well as leave the option for flexibility in folder naming. Logically you would think the application designer would stick to a folder structure with some type of descriptor but nothing says they wouldn't use a folder stucture such as \e\1\f. Now I also understand you can't attempt to work around every "what if" scenario. Just curious why your leaning to go more complicated that what is already in place? Right now you allow the user to put anything in the set line after the "/" character to create a subfolder named what they specify. Adding the ability to go deeper seems like it wouldn't be alot of work to implement (but I am not familiar with the language your using.)

I do see your point in the alternate languages and having the application download the appropriate language specific image.

[Ark]

I'm using MFC/C++ and I think the CreateDirectory() call will create all parent directories for me as needed. If not, I split on the \ and loop through each part, creating them one by one. No big deal.

I'm not sure how else I can support paths like you suggest, with language strings and such, without just letting the user enter a free form string.

Can somebody let me know on the "Core Sets" vs "Expansions" thing. Is there a 3rd choice that can appear in that "slot" ?

[woogerboy21]

I'm using MFC/C++ and I think the CreateDirectory() call will create all parent directories for me as needed. If not, I split on the \ and loop through each part, creating them one by one. No big deal.

I'm not sure how else I can support paths like you suggest, with language strings and such, without just letting the user enter a free form string.

Can somebody let me know on the "Core Sets" vs "Expansions" thing. Is there a 3rd choice that can appear in that "slot" ?

Free form is the only way I can think of as well. Also I can tell you the image folder structure can start with one of the following:

vanguard
core sets
expansions
promo cards
special sets

Is that what your looking for? if so you can see the folder structure the magic album application uses by downloading the app and under the database folder open the sets.csv file.

[Ark]

Do all 5 of these types use the same format with a set name and then language subfolder?

GathererDownloader does not really have a way to identify which of these 5 groups a set belongs to, so I'm not really sure I can implement a {settype} tag if I did this now.

[woogerboy21]

Do all 5 of these types use the same format with a set name and then language subfolder?

Yes, each of these types use the same folder structure format of {set type}\{set name}\{language}\{card image} and each image is stored in its respective language folder. The magic album application supports the languages listed above and displays the appropriate image for each language based on user configuration and selection. A user can infact have all the languages selected so all the versions of the cards are visable in the main library or user inventory and as the user clicks on each one the respective image is shown in the image window.

[woogerboy21]

I really like this app due to its simplicity! But I believe I found an actual bug that you might want to correct. It appears that if you have an invalid file or folder character the application simply bombs when attempting to create the file or folder. For example if you use set name for the folder naming method and you have a set such as "Ravnica: City" {snip} when the application gets to that set and attempts to create the folder it crashes (atleast it does for me). Is there any way you could work in the ability to check for and remove invalid characters in the folder / files names before the application attempts to actually create them?

[Ark]

I've always removed : and " from the strings used to create the path names. I get a folder with the name "Ravnica City of Guilds" when I test. Did you change something in the set list to not be the default? If so, please let me know the exact line you used. I tried: RAV-Ravnica: City and did not get any crash.

Also in file names, I remove : " and the string " // " (minus the quotes) from the file names.

[woogerboy21]

I've always removed : and " from the strings used to create the path names. I get a folder with the name "Ravnica City of Guilds" when I test. Did you change something in the set list to not be the default? If so, please let me know the exact line you used. I tried: RAV-Ravnica: City and did not get any crash.

Also in file names, I remove : " and the string " // " (minus the quotes) from the file names.

Odd, on this xp box it worked fine using only the Ravnica set but the other day working on a different box (running win 7) when I tried to download from alpha, and all sets afterwords, it made it to the Ravnica line and failed out. And no I didn't change anything on the set's list. I'll have to try it again on the windows 7 machine later.

[woogerboy21]

Just an update. I ran the applicatoin again on both my boxes and was able to download all the set and card information so I have no idea why the application stop responding and killed over the few times prior that I tried it. I found it odd that it always stopped in the same spot (Ravnica set). Oh well.

One other question for you, I've created code to take and if you download the set information using the set full name, card full image, and card name that it parses the downloaded folder and moves the images and folders (renaming if required) into the proper format expected for the magic album application. Would you be interested in a script or compiled version of this or would you mind if I posted a "how-to" on using this tool in conjunction with something I post on how to collect the card images for the magic album application?

[woogerboy21]

Alright ran into some other things and curious if I am just the lucky one or if its actually a bug? If I attempt to pull the sets Unglued and Unhinged it appears that the application is not pulling out the invalid file name characters on the card names themselves. For example the card "Question Elemental?", the application simply says it cannot save the card image. But if I save the database as an .xml file and edit the card name removing the '?' in the name the card image is then saved properly. This also appears to happen in the card "Who/What/When/Where/Why" and "Which of You Burn Brightest?"

The second bug appears to be that if you attempt to add the set "Premium Deck Series: Fire & Lightning" the application does not format the character '&' correctly so when it attempts to connect to the gatherer it fails and gives an error.

Are you able to reproduce these problems?

[Ark]

There are some things there that I can work around in the next version:
I can add \ / * ? < > | to the list of characters stripped from paths and file names to fix things like Question Elemental.
I can also change the flag on the status line of text that changes &blah to blah and makes Alt+B a shortcut to that line or the next thing in the dialog.

However, the problem is that the correct name for the set is Fire and Lightning, not Fire & Lightning. Gatherer Downloader does urlencode the set name, and for that example, it tries to fetch the following URL:
http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=text&set=%5B"Premium+Deck+Series%3A+Fire+%26+Lightning"%5D&special=true

Due to bad programmers on Wizard's end, that & (encoded as the %26) causes a HTTP 302 redirect response to the Server Error page, and not a normal HTTP 200 response.

If you had tried Fire + Lightning instead, the URL would be:
http://gatherer.wizards.com/Pages/Search/Default.aspx?output=spoiler&method=text&set=%5B"Premium+Deck+Series%3A+Fire+%2B+Lightning"%5D&special=true
Where the + is encoded as the %2B. This results in a normal page where Gatherer Downloader can get the XML document and report that there are no cards found.

I was going to push a new version out today, but I want some feedback on this options screen I added instead of the menu checks for the file/path formatting. Basically, do the options for the MWS/MagicAlbum look correct?

ALSO! I forgot, but I still need more info on the 3-letter codes that Magic Album uses.
For the two-letter versions, I'm using ISO 639-1 codes, and the country type codes for the two Chinese ones: en, zh-cn, zh-tw, fr, de, it, ja, pt, ru, es.

I was going to use ISO 639-2/T codes for the 3-letter versions, but those are: ENG, ZHO, ZHO, FRA, DEU, ITA, JPN, POR, RUS, SPA
You listed ZHC and ZHT, neither of which I can find in any standard, and also you listed GER for German, which can be found in the older ISO 639-2/B standard: ENG, CHI, CHI, FRE, GER, ITA, JPN, POR, RUS, SPA
As you can see, 639-2/B means using FRE for French instead of FRA, and CHI for Chinese instead of ZHO.

I'm conflicted as to which set of 3-letter codes to use now, and I don't know how ISO 639-2 deals with language variants like 639-1 does either.

[woogerboy21]

As for the "Fire & Lightning" set I was aware the correct name included the "and" word and that worked but was just posting it in wondering if you had any thoughts about working in anything to the application to correct the name before querying the gatherer.

Also the screenshots look good to me atleast.
And as far as the language standard, that would be a question for Goblin Hero since he is the one that developes the application and hasn't released any of the code.

edit:
scratch the folder name question....

[Ark]

Ok, I posted v1.1.0, and here is what is changed:
Added work-around for Magical Hacker's Flavor Text causing invalid XML
Added \ / * ? < > | to the list of characters stripped from paths and file names
Removed flag from status line that changes &x to _x_
Moved several menu option checkmarks into a new Image Formats dialog window which also contains some new options to format the card alternate art suffixes, split cards, and per-language subfolders.

I'm trying to add options that are helpful to people who use programs that can use the images from Gatherer, but I can't spend all my time following Goblin Hero and tracking what nonstandard things he does in his program, if you know what I mean. The set names I use match the exact set names Gatherer uses, and the set codes do as well for the most part.
I can't really put a feature in that separates out folders for Core vs Expansions, because Gatherer doesn't provide me this information. I also choose to use the ISO 639-2/T standard for 3 letter codes, but I compromised on not having ZHO listed twice, so my list of 3 letter codes is ENG, ZHO, ZHT, FRA, DEU, ITA, JPN, POR, RUS, SPA. So for everything but German, I guess you're good to go for Magic Album, after you move the sets into the Core vs Expansions folders. With the suffix formatting and split card formatting options, I think I minimized the amount of manual work needed after a big download at least.

I saw your list of folders to be renamed before you removed it, and I think for the most part, his solution to sets with ":" in the name is to change it to " -", but that would make Ravnica: City of Guilds into Ravnica - City of Guilds, and that wasn't on your list. I'm a big fan of consistency, so I can't say if that option helps or hurts, because it would effect all set name folders created if I put something like that in.

[woogerboy21]

No problem on spending your time following other applications. I just didn't know what you had in mind for formating images to work with applications that use those images. It looked like from the earlier releases you had worked in Magic Workstation formating and was just trying to help users of other applications out. Also I took the folder names out of my post figuring that users can either define the folder names them selves or rename the folders.

Did you by chance want the few specialty sets that you don't include in the default set list to add to the default set list or are you not concerned with them? Sets such as the premium series, from the vault, or premium series for example.

[Ark]

I'd like to make it easy as possible to get the images into folders and filenames formatted the way users want them to use with other applications. That is the whole point of the program. However, I need to try to keep to standards and official naming, so unless I can make a future-proof and generalized rule to deal with something (for example, translate ":" into " -") then I can't see adding the rules. For example, the Core vs Expansion vs Promo thing, I don't know which sets in the future Goblin Hero is going to put into which category, or if he will make up a new category, so I can't follow it very easily.

I can change the Default button you see in the Set List Editor to be a dropdown combobox + Load Defaults button just like you see in the Image Formats dialog. That way I can try to put in all the other sets in other lists in a similar way. Please let me know the lists with set codes.