MTG Gatherer Downloader

[woogerboy21]

I know that most people come to this site for the HQ images created by the CCGHQ team (which are wonderful by the way) but recently I ran into individuals that wanted full card images but didn't have the space to store all the images in HQ. After posting the question about low quality images and searching around I found no real solution for obtaining full card images in any type of downloadable "pack" similar to what is available for the HQ images. So I created the "MTG Gatherer Image Downloader" HTA to parse and download the images from http://gatherer.wizards.com/. This should save you time from having to search, then click and save every image if you so want them locally. Feedback welcome.

Screenshot:


NOTE
If you recieve the following error (or similar)


Please choose "no".
If you do not wish to see this message please follow these steps: http://support.microsoft.com/kb/175500
Also, there is now a .reg file included in the distro that will disable this popup as well


Update Notes
Version 2.9 has had a minor testing on the image downloading routines. Please report if things are broken or wrong. See "Program Versions" below for what has been updated.

Known Bugs:

None

I strongly encourage users to download any new images that are available in HQ using the torrent file provided by http://www.slightlymagic.net at http://www.slightlymagic.net/tracker (and peered by many supporters).
mtg.gatherer.image.downloader.hta||2.9||450d3a8e3c050f57a6cad27ec6b5e19c

[woogerboy21]

Program Versions:

• v0.1 - Initial Release (probably buggy)
• v0.2
  
  • Added File Naming Options
    "plain" mode [by card name]
    "mws" mode [by card name with the .full. in the file name for the magic workstation application]
    
  • Added Temp File Cleanup Routine
    
  • Minor UI Changes
    
• v0.3
  
  • Corrected "basic land" naming issue
    
• v0.4
  
  • Added "All" Option (if you dare)
    
  • Corrected image set detection, now on older set it actually downloads the older card image!
    
• v0.5
  
  • Updated "release notes" formating
    
  • Correcte "resume" disabling error when trying to overwrite an existant image
    
  • Corrected invalid characters in card names issue
    
  • Minor UI Changes
    
  • Added "hta.timeout.fix.reg" registry key file to distrobution
    
  • Added "card image settings
  • set / newest" option, now you can select "set" which will download the card images from the selected set in there original form or "newest" which will download the card images from the most recent set available
    
• v0.6
  
  • Corrected issue with "all" placing each sets folder in a subfolder of the previous set
    
  • Corrected the number of pages in each set displayed / downloaded in the UI
    
• v0.7
  
  • More UI Changes!
    
  • Added folder structure support. There is now a third dropdown menu under "card image settings". Two options so far include "set name" and "magic album". "Set name" creates folders based on set names and places downloaded images into them. "Magic Album" creates the subfolders for each set in the form that the magic album application uses and stores images in that folder structure.
    
• v0.8
  
  • Added language support (english is the only option so far)
    
  • Corrected "magic album" folder structure path to include language subfolder
    
  • Minor update to UI
    
• v0.9
  
  • Added MWS Folder Structure Support
    
  • Corrected minor status box display issue when still showing last card downloaded when "complete" popup is displayed
    
  • Minor update to UI
    
• v1.0
  
  • Added sets verification from the gatherer on startup. Now the program locates all the available sets on the gatherer and creates a warning log in the event there are more sets online than in the known listing of sets.
    
  • Majorly reduced the amount of memory the application uses while running.
    
  • Corrected status box to indicate if a card is being verified or downloaded during resume (previously it always just showed downloading)
    
  • Minor status box formating updates
    
• v1.1
  
  • Bugfix - Error in sets array for set "Starter 1999", Corrected (the "all" option should now work again)
    
  • Updated the magic album folder structure routines to allow more than a single subfolder to be defined in the magic album set folder information
    
  • Created custom name formating routines to reformated "known" mal-formed named. (card names containing Aether in HTML form are not properly spelled)
    
  • Created routines to do a version check and inform the user if an update is available.
    
• v1.2
  
  • Corrected "Portal Three Kingdoms" display name
    
  • Corrected Incorrect Pathing Names for the magic album folder structure on a number of sets
    
  • Added "From the Vault: Relics" set to known set array
    
  • Corrected endless loop when more than 4 basic lands already exists
    
• v1.3
  
  • Corrected "folder does not exist" error when attempting to use "setname" as folder path
    
  • Added more languages to the available language list!
    
• v1.4
  
  • Added additional language options
    
  • Corrected existing language abbr. to correctly name folder structures to MA application expected folder names
    
  • Added Duels Deck: Elspeth vs. Tezzereth
    
  • Updated UI Size
    
• v1.5
  
  • Somehow managed to bunk up v1.4. Corrected Duel Deck: Elspeth vs. Tezzereth not downloading
    
  • Corrected error on line 436 about "file not found" when attempting to remove temporary files used during download process of card images
    
• v1.6
  
  • Corrected internal application name mispelling
    
  • Added Additional name correction character to better spell the word "AEther" (still working on special card name corrections)
    
  • Removed redundant language abbr. (the older abbr that were wrong, doh!)
    
• v1.7
  
  • Added the ability to export the card details listed from the gatherer to a txt file! Current options are "none", "text only", "all". "none" does not export data, "text only" exports only the card text in the textbox (no flavor text), and "all" exports all data collected (excludes expansions, and other sets).
    
• v1.8
  
  • Added the application "mode" drop down menu option. The three modes will allow you to either download just card images, just card text, or both from the gatherer's website
    
• v1.9
  
  • Added the export option "full text spoiler" menu option. This options will allow you to create a full text spoiler of the choosen set (i do not recommend you use this option with the "all" option under "set selection")
    
  • Updated sets listing to include scars of mirrodin gatherer information
    
  • Minor text capturing formating updates (such as trimming extra spaces between Casting Cost amounts)
    
  • Added power / toughness captureing to text portion of text data outputing
    
  • Added the export option "tab delimited spoiler" menu option. This option will allow you to create a full text spoiler with each cards components seperated in a single line by tabs for importation into other apps
    
  • Added the option to create a visual spoiler html page that is printable
    
  • Added the option to create proxies and be able to select the rarity of proxies printing to use. (this creates output similar to magiccards.info)
    
• 2.0
  
  • Corrected scars of mirrodin set information (now you can get more than the first page of images)
    
  • Corrected text conversion for the mana cost of 3. Now correctly identifies the image icon
    
• 2.1
  
  • Corrected internal application number
    
  • Updated "update check" routine to improve performance
    
  • Updated "gatherer check" routine to improve performance
    
• 2.2
  
  • Corrected "Red" Mana symbol formating in text output
    
  • Corrected "Black" Mana symbol formating in text output
    
  • Corrected "Green" Mana symbol formating in text output
    
  • Corrected "X" mana symbol formating in text output
    
  • Corrected Colorless 5-10 mana symbol formating in text output
    
  • Renamed spoiler names to include language abbr
    
• 2.3
  
  • Updated language text capture routine to capture the name of the card in the language rather than english
    
  • Added "single card number only" option to text output method, this option will create a text file with the card name and only write the card number in the text file content
    
  • Changed the "card export" descriptor to "text options" for better clarifications on the UI
    
  • Changed the text option "all" to "single card all details" for clearer definition
    
  • Changed the text option "text only" to "single card text only" for clearer definition
    
  • Added "mws text spoiler" option to text output method, this option creates a text spoiler that you can import into MWS to add card set information into your masterbase (**NOT FUNCTIONING PROPERLY, UNDER DEVELOPMENT**)
    
• 2.4
  
  • Major UI update! Fixed UI functionality for enabling / disabling features when various options / suboptions were selected
    
  • Added the customization options to the text spoiler output. Now instead of selecting from a drop down menu of various output options you select the type of text output and can checkthe fields you want to have located in the text output file
    
  • Minor internal code update to make code layout clearer (code still a mess in my mind but it works)
    
• 2.5
  
  • Major internal code update to the locate image routines (this will allow for additional features down the road)
    
  • Minor internal code update to the locate card details routine, now will return specific card information when called (most people probably dont care about this)
    
  • Added the drop down menu item "plain-num" to the image name option list, this will name the image downloaded starting with the cards number located in the set (if available) at the beginning of the card name
    
  • Added the drop down menu item "plain-color" to the image name option list, this will name the image downloaded starting with the cards color (if available) at the beginning of the card name
    
  • Remove the drop down menu option "newest" from the image version option list (work still in progress for additional set image identification)
    
  • Added card color to text outputing options
    
  • Corrected "tab delimited spoiler" option under "text spoiler format" menu to now work with spoiler option check boxes
    
  • Corrected text and flavor text when outputing to "tab delimited spoiler" option. (removed line return characters so lines are now properly written)
    
  • Linked the "image name" drop down menu with the "folder style" drop down menu so now when choosing various naming methods or folder styles the other drop down menu changes accordingly
    
  • Update the "plain" image name option functionality, now if you choose the drop down menu "plain" under the image name options it will name the image of the card locally with the spelling of the respective language (i have no idea if this works right since I don't know any other language other than english, but its sure naming things funny)
    
  • Added the "plain-eng" image name option, if you choose this option it will name the downloaded image with the english name form but the image downloaded will be of the respective language
    
  • Added the "magic album" image name option, this has the same functionality as the "plain-eng" option but changes the folder settings accordingly
    
  • Enabled basic error resuming
    
  • Update UI controls to not allow changes during application downloading
    
  • Added "enable local cache option", this option will save the set data as it is downloaded and as will use the local copy of the set information rather than redownloading it every time. (THIS OPTION MUST BE ENABLED EACH RUN OR THE SET INFORMATION WILL BE RECOLLECTED AT RUN TIME)
    
• 2.6
  
  • Updated Card Name routines to remove invalid filename characters (some how removed this from the recent release so cards such as "circle of protection" arent properly named / downloaded)
    
  • Updated Tab delimited routines to split card type and subtype out into there own column locations
    
  • Updated UI "Download Images" button to "Start Download"
    
• 2.7
  
  • Corrected Image Name naming when the MWS image option is selected (broken as of version 2.5 after rewrite)
    
  • Minor UI update (lowercased the option "enable local cache")
    
  • Created internal code function to identify and correct special case names for image names based on application
    
  • Implemented split card image naming
    
  • Implemented multiversion card image capturing (now you get all the land cards and all the different versions of older cards!)
    
  • Corrected issue were enabling local cache would cause card details to not be located if application was terminated prematurely
    
  • Corrected text formating for older SYMBOLS pre 6th edition (i think thats when the old T for tap was changed?)
    
  • Updated "exporting to single card" to create seperate .txt files for each version of a card (if multiples exist)
    
  • Enabled text export resuming when the option "image and text" is selected, no need to redownload all card details that you should already have
    
  • Corrected text exporting routines to resolve issue of text mana symbols that are side by side only showing a single symbol rather then multiples
    
• 2.8
  
  • Implemented error checking for image folder path name to verify string is formated as expected
    
  • Implemented minor run and error logging
    
  • Corrected subtype column alignment on tab formated exporting
    
  • Updated exporting card text routines to replace the unicode dash code with plain text dash's
    
  • Updated special case card naming routine for split name cards
    
  • Updated UI controls to keep "proxy rarity" disabled by default till the checkbox is checked
    
  • AEther name correction implemented
    
• 2.9
  
  • Minor internal code changes
    
  • Added the "wagic" application to the cardname and folderstyle
    
  • Added zip.exe application to distrobution to allow image zipping for the wagic application
    

[deurk]

Great tool! Thanks!

Any possibility to incorporate download of other languages?

Thanks.

[woogerboy21]

Great tool! Thanks!

Any possibility to incorporate download of other languages?

Thanks.

Absolutely. What languages were you looking for? Plus since I havent looked into how to get other languages from the gatherer are you familiar with how to do so? Is there a different URL for other languages etc?

[deurk]

I was trying to get FR so far but I was not able to get the gatherer to show me the checklist and/or spoiler versions of all the FR cards. I was able to see their french text on single cards using the following URL for example:
http://gatherer.wizards.com/Pages/Card/ ... eid=222305
(Note the printed parameter)

How is your tool getting the pics? Directly from each single card page or from a more general page like listing?

Thanks!

[woogerboy21]

I was trying to get FR so far but I was not able to get the gatherer to show me the checklist and/or spoiler versions of all the FR cards. I was able to see their french text on single cards using the following URL for example:
http://gatherer.wizards.com/Pages/Card/ ... eid=222305
(Note the printed parameter)

How is your tool getting the pics? Directly from each single card page or from a more general page like listing?

Thanks!

The way it works is it first queries the gatherer for a complete set listing using the standard view such as a url like this:
[url]http://gatherer.wizards.com/Pages/Search/Default.aspx?action=advanced&set=|["Alpha"][/url]

You can locate the URL listing used by going to the gatherer. Using the advanced button. Then simply adding the set name you are looking to query for. And clicking the search button.

It then parses the recieved data and locates each card listing (name and multiverse image number). It then goes back and downloads the image using the multiverse number and placing the image with the correct name into the save folder that is defined for each set in the array containing all the set information. Seeing as I was unfamiliar with the other languages I was hoping that you could query the gatherer and recieve a standard view listing in foreign languages. If so there would only need some minor updating to the code to reflect the different languages split location spellings. But if there is no way to view a listing of cards in a particular set in a particular language (other than english) there will need to be some major updates in the code to support other languages.

edit:
looking at the language tab there may be a way for me to determin the multiverse ID from there so adding the ability for each language would be possible but give me a bit to think about how to implement it. First thing that comes to mind is adding one more step in after identifying the card from the set list to aquire all the multiverse id's for that card from the language tab page and then downloading the associated picture based on selected language

[deurk]

Would be great. If your tool is parsing the card page and not just a listing, I'd be interested in it to also retrieve the text of that page, that would plain rock

[woogerboy21]

Would be great. If your tool is parsing the card page and not just a listing, I'd be interested in it to also retrieve the text of that page, that would plain rock

What data are you exactly looking for? It is possible to parse the page and its contents and save them. The way the script does it probably doesnt work much like your thinking (possibly) but can be adapted to fit probably easily.

[woogerboy21]

Could anyone tell me were to get the language to language letter abbreviations used?
Currently I have these defined but dont think the abbr are correct:

"eng" : "english"
"csa" : "chinese standard"
"cst" : "chinese traditional"
"fre" : "french"
"ger" : "german"
"itl" : "italian"
"jap" : "japanese"
"por" : "portuguese"
"rus" : "russian"
"spa" : "spanish"

[Huggybaby]

Nice, what a great tool. Thanks woogerboy21.

[deurk]

Could anyone tell me were to get the language to language letter abbreviations used?
Currently I have these defined but dont think the abbr are correct:

"eng" : "english"
"csa" : "chinese standard"
"cst" : "chinese traditional"
"fre" : "french"
"ger" : "german"
"itl" : "italian"
"jap" : "japanese"
"por" : "portuguese"
"rus" : "russian"
"spa" : "spanish"

I think those are correct:
cn 简体中文 Simplified Chinese
de Deutsch German
en English English
es Español Spanish
fr Français French
it Italiano Italian
jp 日本語 Japanese
ko 한국어 Korean
pt Português Portuguese
ru Русский Russian
tw 繁體中文 Traditional Chinese

[deurk]

What data are you exactly looking for? It is possible to parse the page and its contents and save them. The way the script does it probably doesnt work much like your thinking (possibly) but can be adapted to fit probably easily.

The printed card texts mostly. Putting gathered text within a .txt file next to the picture would be awesome

[woogerboy21]

Nice, what a great tool. Thanks woogerboy21.

No problem! I still need to correct the foreign abbr. But the code structure is there and it shouldn't be to difficult. Im just busy working on updating the magic album pricing scripts.

The printed card texts mostly. Putting gathered text within a .txt file next to the picture would be awesome.

Are you looking for the foreign text or the english?

[deurk]

Actually both, but each in it's own language pass, so I'd say foreign

[Huggybaby]

This topic is stickied.