Unified picture resource handling in MA and Forge

[LordHelmchen]

Greetings!

As I'm using both Magic Album by Goblin Hero and Forge, I currently have two folders with card pictures, which strikes me as a waste of space (and on updates, bandwith). Thus I'd like to ask if the two projects could envision to collaborate on unifying their expected folder structure and enable to set one picture folder for both programms to read their images from.
Crossposting this to both Forge and MA subforums.

regards,
LordHelmchen

[Goblin Hero]

Frankly speaking I do not want to change the way MA handles the pictures. I like current system. And it depends on my magic cards database.

[woogerboy21]

Do NTFS symbolic links not work?

[Nyth]

NTFS Symbolic links are not for the lighthearted... MA is purely an inventory and deck building tool so the picture folder structure is very important for collectors. Forge doesn't need all that. Forge doesn't need multi-language, foil/non-foil differentiation, and all the rest of it. Seems a very difficult task to unify those even if it made sense to.

[Qriist]

As a developer of my own projects, I've actually ran into the need for multiple duplicate files in multiple places... and it really doesn't make sense to double/triple/quadruple/more your space requirements for such reasons. Hence, lots of searching for a solution. A few years ago I came across this gem of a program, finddupe.

It is a fast and tidy program that by default will hardlink duplicates in place. I propose that MA includes a very simple 'cross-reference' checker. A single input line (or input box with a number of lines) that directs to an external, user-specified folder (or folders.)

With that input line or box, add one more button that initiates the command. At this point, how MA handles the CLI info, if at all, is up to author desires. Since the program is reliable enough and 100% non-destructive by default, MA can just say "search started, duplicates will be linked" or something equally throw-away.

Of course, in the meantime the commandline is simple enough and the process merits going through, so give it a shot. If afraid of the big, bad CLI I'd be happy to post a simple batch script.


tl;dr version: use finddupe to automagically hardlink all relevant image files in place between MA and any other Magic resource.
---Qriist

Finddupe found here: sentex . net/~mwandel/finddupe/
^URL malformed due to site spam guards. Sorry.

[LordHelmchen]

One Problem is that Forge looks for flat folders, whereas MA uses zip archives. Unless finddupe handles zips, hardlinks are not the sollution.

Since MA's picture archive is more detailed than Forge's needs, the torrent is wonderful distribution and update method, and the main reason for Forge's image organization is legacy compatibility with MWS, I would say that the best chance lies in modifying Forge to use MA's images...

[LordHelmchen]

This is an interesting idea. Not sure how many people would find themselves in a position to make use of this type of a feature.

A good question, which merrits being asked to MA users here: Is is just me, or are there more people who use Magic album and Forge, but not MWS? Would you like Forge to read your existing MA picture folders, or do you prefer the two to use seperate copies of the image scans ?

Edit:

It seems like MA is the oddball here not us, and we are being more compatible by sticking to the HQ Pics naming convention.

To Goblin Hero: I understand that you are involved with or even responsible for both MA and the HQ Pictures. Why did you drop the ".full" infix for MA's pictures? Even if it is redundant, why create incompatibilites between your two projects?

[Goblin Hero]

Even if it is redundant, why create incompatibilites between your two projects?

Why not? I do not like MWS structure and I've tried to create a new one which can be easily upgraded if WOTC will invent something else.

[LordHelmchen]

Just to clarify (as we're communication through what is a foreign language for both of us): I am not questioning the directory structure you chose for MA (on the contrary, while I have no experience with either the HQpictures or MWS, I can understand MA's structure and find it logical). Using zip archives looks like a smart move, too, as it made torrent distribution possible.
I'm just asking about the change from ".full.jpg" to ".jpg"... while I can see that having the .full infix is not necessary for MA (as it doesn't use any other images, like cropped artwork-only images), it creates a potentially unnecessary divide between the projects.

[LordHelmchen]

The new resource directory handling of forge >= 1.3.11 made this look a lot more doable than before. My changes so far had the downloader report only 476 missing pictures, 233 of which were tokens. Those were downloaded without problems and, though they're technically duplicates as well, are probably best left as duplicates for now.

What I did so far:

forge.profile.properties | Open

Code: Select all

# point this to Magic Album's picture folder, probably populated via Goblin Hero's torrent.
cardPicsDir=D:/Magic - The Gathering/Pictures
cardPicsSubDirs=LEA->Core Sets/1st Limited Edition Alpha/ENG.zip|LEB->Core Sets/1st Limited Edition Beta/ENG.zip|2ED->Core Sets/2nd Unlimited/ENG.zip|ARN->Expansions/Arabian Nights/ENG.zip|ATQ->Expansions/Antiquities/ENG.zip|3ED->Core Sets/3rd Revised/ENG.zip|LEG->Expansions/Legends/ENG.zip|DRK->Expansions/The Dark/ENG.zip|FEM->Expansions/Fallen Empires/ENG.zip|4ED->Core Sets/4th Edition/ENG.zip|ICE->Expansions/Ice Age/ENG.zip|CHR->Special Sets/Chronicles/ENG.zip|HML->Expansions/Homelands/ENG.zip|ALL->Expansions/Alliances/ENG.zip|MIR->Expansions/Mirage/ENG.zip|VIS->Expansions/Visions/ENG.zip|5ED->Core Sets/5th Edition/ENG.zip|POR->Special Sets/Portal/ENG.zip|WTH->Expansions/Weatherlight/ENG.zip|TMP->Expansions/Tempest/ENG.zip|STH->Expansions/Stronghold/ENG.zip|EXO->Expansions/Exodus/ENG.zip|PO2->Special Sets/Portal Second Age/ENG.zip|USG->Expansions/Urza's Saga/ENG.zip|ULG->Expansions/Urza's Legacy/ENG.zip|6ED->Core Sets/6th Classic Edition/ENG.zip|UDS->Expansions/Urza's Destiny/ENG.zip|PTK->Special Sets/Portal Three Kingdoms/ENG.zip|S99->Special Sets/Starter 1999/ENG.zip|MMQ->Expansions/Mercadian Masques/ENG.zip|NMS->Expansions/Nemesis/ENG.zip|S00->Special Sets/Starter 2000/ENG.zip|PCY->Expansions/Prophecy/ENG.zip|INV->Expansions/Invasion/ENG.zip|PLS->Expansions/Planeshift/ENG.zip|7ED->Core Sets/7th Edition/ENG.zip|APC->Expansions/Apocalypse/ENG.zip|ODY->Expansions/Odyssey/ENG.zip|TOR->Expansions/Torment/ENG.zip|JUD->Expansions/Judgment/ENG.zip|ONS->Expansions/Onslaught/ENG.zip|LGN->Expansions/Legions/ENG.zip|SCG->Expansions/Scourge/ENG.zip|8ED->Core Sets/8th Edition/ENG.zip|MRD->Expansions/Mirrodin/ENG.zip|DST->Expansions/Darksteel/ENG.zip|5DN->Expansions/Fifth Dawn/ENG.zip|CHK->Expansions/Champions of Kamigawa/ENG.zip|BOK->Expansions/Betrayers of Kamigawa/ENG.zip|SOK->Expansions/Saviors of Kamigawa/ENG.zip|9ED->Core Sets/9th Edition/ENG.zip|RAV->Expansions/Ravnica City of Guilds/ENG.zip|GPT->Expansions/Guildpact/ENG.zip|DIS->Expansions/Dissension/ENG.zip|CSP->Expansions/Coldsnap/ENG.zip|TSP->Expansions/Time Spiral/ENG.zip|TSB->Expansions/Time Spiral Timeshifted/ENG.zip|PLC->Expansions/Planar Chaos/ENG.zip|FUT->Expansions/Future Sight/ENG.zip|10E->Core Sets/10th Edition/ENG.zip|LRW->Expansions/Lorwyn/ENG.zip|MOR->Expansions/Morningtide/ENG.zip|SHM->Expansions/Shadowmoor/ENG.zip|EVE->Expansions/Eventide/ENG.zip|ALA->Expansions/Shards of Alara/ENG.zip|CFX->Expansions/Conflux/ENG.zip|ARB->Expansions/Alara Reborn/ENG.zip|HOP->Special Sets/Planechase/ENG.zip|M10->Core Sets/Magic 2010/ENG.zip|ZEN->Expansions/Zendikar/ENG.zip|WWK->Expansions/Worldwake/ENG.zip|ROE->Expansions/Rise of the Eldrazi/ENG.zip|M11->Core Sets/Magic 2011/ENG.zip|SOM->Expansions/Scars of Mirrodin/ENG.zip|MBS->Expansions/Mirrodin Besieged/ENG.zip|NPH->Expansions/New Phyrexia/ENG.zip|COM->Special Sets/Commander/ENG.zip|M12->Core Sets/Magic 2012/ENG.zip|ISD->Expansions/Innistrad/ENG.zip|DKA->Expansions/Dark Ascension/ENG.zip|AVR->Expansions/Avacyn Restored/ENG.zip|PC2->Special Sets/Planechase 2012/ENG.zip|M13->Core Sets/Magic 2013/ENG.zip|RTR->Expansions/Return to Ravnica/ENG.zip|GTC->Expansions/Gatecrash/ENG.zip|VAN->Spedial Sets/Vanguard/ENG NTR.zip|ARC->Special Sets/Archenemy/ENG.zip
#problems:
#Tokens         MA:per set TOK.zip vs F:global token dir. We can have Forge create duplicate files with the expected filenames.
#Archenemy      Schemes are in ENG NTR.zip
#Planechase      Planes are in ENG NTR.zip
#PC 2012      Planes are in ENG NTR.zip
#Commander      Oversized Commanders are in ENG OVS FOIL.zip, but traditional sized versions of the commanders are available in ENG.zip anyways.
#Starter 2000   Rhox is only in ENG FOIL.zip
#MBP   Media Insert Promo is split by medium in MA


modify forge/ImageLoader.java and forge/gui/download/GuiDownloadSetPicturesLQ.java:

readMApictures-v2.patch | Open

Code: Select all

Index: src/main/java/forge/ImageLoader.java
===================================================================
--- src/main/java/forge/ImageLoader.java   (revision 20874)
+++ src/main/java/forge/ImageLoader.java   (working copy)
@@ -3,6 +3,11 @@
 import java.awt.image.BufferedImage;
 import java.io.File;
 import java.io.IOException;
+import java.io.InputStream;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.zip.ZipEntry;
+import java.util.zip.ZipFile;
 
 import javax.imageio.ImageIO;
 
@@ -91,6 +96,56 @@
                     break;
                 }
             }
+
+            // attempt to read MA-style named image file from within zipped directory
+            System.out.println(String.format("key: %s ; path: %s ; filename: %s", key, path, filename));
+            String malikePath = (path + filename).replaceAll("/[^/]+$", "");
+            String malikeFilename = filename.replaceFirst(".+/", "");
+            malikeFilename= malikeFilename.replace(".full", "");
+            if (malikeFilename.matches(".+[0-9]+$")) {//adapt art index to MA-style naming
+                Pattern pat = Pattern.compile("([0-9]+)$");
+                Matcher mat = pat.matcher(malikeFilename);
+                malikeFilename = mat.replaceAll(" [$1]");
+//            } else {
+//                System.out.println("'" + malikeFilename + "' did not match");
+            }
+            System.out.println(String.format("malikePath: %s ; malikeFilename: %s ; ext: %s", malikePath, malikeFilename, ext));
+            File zippath = new File(malikePath);
+            if (zippath.exists()) {
+                ZipFile zis;
+                try {
+                    zis = new ZipFile(malikePath);
+                } catch (IOException ex) {
+                    BugReporter.reportException(ex, "Could not open zipped directory " + malikePath + " ");
+                    break;
+                }
+                BufferedImage image = null;
+                ZipEntry zipentry = zis.getEntry(malikeFilename + ext);
+                if (null != zipentry) {
+                    try {
+                        InputStream inputstream = zis.getInputStream(zipentry);
+                        image = ImageIO.read(inputstream);
+                        //image = ImageIO.read(zis.getInputStream(zipentry));
+                    } catch (IOException ex) {
+                        BugReporter.reportException(ex, "Could not open image " + malikeFilename + ext + " from zipped directory " + malikePath + " ");
+                        break;
+                    }
+                } else {
+                    System.out.println(String.format("ZipEntry " + malikeFilename + ext + " not found in zipfile."));
+                }
+                try {
+                    zis.close();
+                } catch (IOException ex) {
+                    BugReporter.reportException(ex, "Could not close zipped directory " + malikePath + " ");
+                    break;
+                }
+                if (null != image) {
+                    return image;
+                }
+            }
+            else {
+                System.out.println(String.format("Zipped directory " + malikePath + " does not exist."));
+            }
         }
         
         return null;
Index: src/main/java/forge/gui/download/GuiDownloadSetPicturesLQ.java
===================================================================
--- src/main/java/forge/gui/download/GuiDownloadSetPicturesLQ.java   (revision 20874)
+++ src/main/java/forge/gui/download/GuiDownloadSetPicturesLQ.java   (working copy)
@@ -17,9 +17,18 @@
  */
 package forge.gui.download;
 
 import java.io.File;
+import java.io.IOException;
 import java.util.Map;
 import java.util.TreeMap;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.zip.ZipEntry;
+import java.util.zip.ZipFile;
 
 import org.apache.commons.lang3.StringUtils;
 
@@ -27,6 +36,7 @@
 
 import forge.ImageCache;
 import forge.card.CardEdition;
+import forge.error.BugReporter;
 import forge.item.CardDb;
 import forge.item.CardPrinted;
 import forge.properties.NewConstants;
@@ -61,10 +71,44 @@
     }
 
     private void addDLObject(String urlPath, String filename, Map<String, String> downloads) {
-        File destFile = new File(NewConstants.CACHE_CARD_PICS_DIR, filename + ".jpg");
-        // System.out.println(filename);
-        if (!destFile.exists()) {
-            downloads.put(destFile.getAbsolutePath(), NewConstants.URL_PIC_DOWNLOAD + urlPath);
+        // check for existing files in MA-style zips first
+        String path = NewConstants.CACHE_CARD_PICS_DIR;
+        String malikePath = (path + filename).replaceAll("/[^/]+$", "");
+        String malikeFilename = filename.replaceFirst(".+/", "");
+        malikeFilename= malikeFilename.replace(".full", "");
+        if (malikeFilename.matches(".+[0-9]+$")) {//adapt art index to MA-style suffixing
+            Pattern pat = Pattern.compile("([0-9]+)$");
+            Matcher mat = pat.matcher(malikeFilename);
+            malikeFilename = mat.replaceAll(" [$1]");
+        }
+        File zippath = new File(malikePath);
+        Boolean filenotinzip = false;
+        if (zippath.exists()) {
+            ZipFile zis = null;
+            try {
+                zis = new ZipFile(malikePath);
+            } catch (IOException ex) {
+                BugReporter.reportException(ex, "Could not open zipped directory " + malikePath + " ");
+            }
+            ZipEntry zipentry = zis.getEntry(malikeFilename + ".jpg");
+            if (null == zipentry) {
+                filenotinzip = true;
+            }
+            try {
+                zis.close();
+            } catch (IOException ex) {
+                BugReporter.reportException(ex, "Could not close zipped directory " + malikePath + " ");
+            }
+        }
+        if (filenotinzip) {
+            // as a quick workaround, one could try to have CACHE_CARD_PICS_DIR ignore cardPicsSubDirs from forge.progile.properties
+            // This would make the downloader look for, download and save the 230-odd conflicting image files in the three-letter-code subdirs.
+            // token handling shows this should work.
+            File destFile = new File(NewConstants.CACHE_CARD_PICS_DIR, filename + ".jpg");
+            // System.out.println(filename);
+            if (!destFile.exists()) {
+                downloads.put(destFile.getAbsolutePath(), NewConstants.URL_PIC_DOWNLOAD + urlPath);
+            }
         }
     }
 }


the following non-token cards are still problematic:

still to be solved | Open

Code: Select all

MA filename
---------------------------------------------------------------------------------------
Schemes are in ENG NTR.zip, currently only traditional cards (from ENG.zip) are loaded
Archenemy      ALL_SCHEMES                  ENG NTR.zip/ALL_SCHEMES
Schemes are in ENG NTR.zip, currently only traditional cards (from ENG.zip) are loaded
Planechase 2012   ALL_PLANES                  ENG NTR.zip/ALL_PLANES
Planechase      ALL_PLANES                  ENG NTR.zip/ALL_PLANES

changed filename in zip, MA seems to have no problem locating the image afterwards.
Alpha         Will-o'-the-Wisp            Will-O'-The-Wisp
M2012         AEther Adept               Aether Adept
Gatecrash      AEtherize                  Aetherize
Legends         Evil Eye of Orms-by-Gore      Evil Eye of Orms-By-Gore
            Torsten Von Ursus            Torsten von Ursus
Mirage         Goblin Elite Infantry         Goblin Elite infantry
Ravnica CoG      Congregation at Dawn         Congregation At Dawn
            Dimir Infiltrator            Dimir infiltrator
            Life from the Loam            Life From the Loam
            Peel from Reality            Peel From Reality
Saviors of K.   Rally the Horde               Rally The Horde
Time Spiral      AEther Web                  Aether Web
            AEtherflame Wall            Aetherflame Wall
Urza's Saga      Tainted AEther               Tainted Aether
Weatherlight   AEther Flash               Aether Flash


Dual cards
Apocylypse      FireIce                     Fire_Ice
            IllusionReality               Illusion_Reality
            LifeDeath                  Life_Death
            NightDay                  Night_Day
            OrderChaos                  Order_Chaos
Dissension      BoundDetermined               Bound_Determined
            CrimePunishment               Crime_Punishment
            HideSeek                  Hide_Seek
            HitRun                     Hit_Run
            OddsEnds                  Odds_Ends
            PureSimple                  Pure_Simple
            RiseFall                  Rise_Fall
            SupplyDemand               Supply_Demand
            TrialError                  Trial_Error
Invasion      AssaultBattery               Assault_Battery
            PainSuffering               Pain_Suffering
            SpiteMalice                  Spite_Malice
            StandDeliver               Stand_Deliver
            WaxWane                     Wax_Wane
Planar Chaos   BoomBust                  Boom_Bust
            DeadGone                  Dead_Gone
            RoughTumble                  Rough_Tumble
T.S.Timeshifted   AssaultBattery               Assault_Battery
Commander      FireIce                     Fire_Ice

Flip cards, looks like Forge needs a seperate image for both flip states
Betrayers of K.   Azamuki, Treachery Incarnate   Cunning Bandit_Azamuki, Treachery Incarnate
            Budoka Pupil               Budoka Pupil_Ichiga, Who Topples Oaks
            Callow Jushi               Callow Jushi_Jaraku the Interloper
            Cunning Bandit               Cunning Bandit_Azamuki, Treachery Incarnate
            Faithful Squire               Faithful Squire_Kaiso, Memory of Loyalty
            Hired Muscle               Hired Muscle_Scarmaker
            Ichiga, Who Topples Oaks      Budoka Pupil_Ichiga, Who Topples Oaks
            Jaraku the Interloper         Callow Jushi_Jaraku the Interloper
            Kaiso, Memory of Loyalty      Faithful Squire_Kaiso, Memory of Loyalty
            Scarmaker                  Hired Muscle_Scarmaker
Champions of K.   Akki Lavarunner               Akki Lavarunner_Tok-Tok, Volcano Born
            Autumn-Tail, Kitsune Sage      Kitsune Mystic_Autumn-Tail, Kitsune Sage.jpg
            Budoka Gardener               Budoka Gardener_Dokai, Weaver of Life
            Bushi Tenderfoot            Bushi Tenderfoot_Kenzo the Hardhearted
            Dokai, Weaver of Life         Budoka Gardener_Dokai, Weaver of Life
            Goka the Unjust               Initiate of Blood_Goka the Unjust
            Initiate of Blood            Initiate of Blood_Goka the Unjust
            Jushi Apprentice            Jushi Apprentice_Tomoya the Revealer
            Kenzo the Hardhearted         Bushi Tenderfoot_Kenzo the Hardhearted
            Kitsune Mystic               Kitsune Mystic_Autumn-Tail, Kitsune Sage.jpg
            Nezumi Graverobber            Nezumi Graverobber_Nighteyes the Desecrator
            Nezumi Shortfang            Nezumi Shortfang_Stabwhisker the Odious
            Nighteyes the Desecrator      Nezumi Graverobber_Nighteyes the Desecrator
            Orochi Eggwatcher            Orochi Eggwatcher_Shidako, Broodmistress
            Shidako, Broodmistress         Orochi Eggwatcher_Shidako, Broodmistress
            Stabwhisker the Odious         Nezumi Shortfang_Stabwhisker the Odious
            Student of Elements            Student of Elements_Tobita, Master of Winds
            Tobita, Master of Winds         Student of Elements_Tobita, Master of Winds
            Tok-Tok, Volcano Born         Akki Lavarunner_Tok-Tok, Volcano Born
            Tomoya the Revealer            Jushi Apprentice_Tomoya the Revealer
Saviors of K.   Erayo's Essence               Erayo, Soratami Ascendant_Erayo's Essence
            Erayo, Soratami Ascendant      Erayo, Soratami Ascendant_Erayo's Essence
            Homura's Essence            Homura, Human Ascendant_Homura's Essence
            Homura, Human Ascendant         Homura, Human Ascendant_Homura's Essence
            Kuon's Essence               Kuon, Ogre Ascendant_Kuon's Essence
            Kuon, Ogre Ascendant         Kuon, Ogre Ascendant_Kuon's Essence
            Rune-Tail's Essence            Rune-Tail, Kitsune Ascendant_Rune-Tail's Essence
            Rune-Tail, Kitsune Ascendant   Rune-Tail, Kitsune Ascendant_Rune-Tail's Essence
            Sasaya's Essence            Sasaya, Orochi Ascendant_Sasaya's Essence
            Sasaya, Orochi Ascendant      Sasaya, Orochi Ascendant_Sasaya's Essence
Commander      Nezumi Graverobber            Nezumi Graverobber_Nighteyes the Desecrator
            Nighteyes the Desecrator      Nezumi Graverobber_Nighteyes the Desecrator

normal (non-full art) lands versions are numbered differently in MA: 1a,2a,3a,4a instead of 5,6,7,8
exact matches still need to be determined, below is just a guess
Zendikar      Forest5                     Forest [1a]
            Forest6                     Forest [2a]
            Forest7                     Forest [3a]
            Forest8                     Forest [4a]
            Island5                     Island [1a]
            Island6                     Island [2a]
            Island7                     Island [3a]
            Island8                     Island [4a]
            Mountain5                  Mountain [1a]
            Mountain6                  Mountain [2a]
            Mountain7                  Mountain [3a]
            Mountain8                  Mountain [4a]
            Plains5                     Plains [1a]
            Plains6                     Plains [2a]
            Plains7                     Plains [3a]
            Plains8                     Plains [4a]
            Swamp5                     Swamp [1a]
            Swamp6                     Swamp [2a]
            Swamp7                     Swamp [3a]
            Swamp8                     Swamp [4a]


Alternate art cards are versioned as "Alt" and found in ENG FOIL.zip
normal cards are not version-suffixed in MA.
Planeshift      Ertai, the Corrupted1         Ertai, the Corrupted
            Ertai, the Corrupted2         ENG FOIL.zip/Ertai, the Corrupted [Alt]
            Skyship Weatherlight1         Skyship Weatherlight
            Skyship Weatherlight2         ENG FOIL.zip/Skyship Weatherlight [Alt]
            Tahngarth, Talruum Hero1      Tahngarth, Talruum Hero
            Tahngarth, Talruum Hero2      ENG FOIL.zip/Tahngarth, Talruum Hero [Alt]

Alternate art cards are versioned as "ST" (Alternate "Two Player Starter Set" version) or "DG" (Alternate "Demogame Booster" version)
normal cards are not version-suffixed in MA.
Portal         Anaconda1                  Anaconda
            Anaconda2                  Anaconda [ST]
            Blaze1                     Blaze
            Blaze2                     Blaze [ST]
            Elite Cat Warrior1            Elite Cat Warrior
            Elite Cat Warrior2            Elite Cat Warrior [ST]
            Hand of Death1               Hand of Death
            Hand of Death2               Hand of Death [ST]
            Monstrous Growth1            Monstrous Growth
            Monstrous Growth2            Monstrous Growth [ST]
            Raging Goblin1               Raging Goblin
            Raging Goblin2               Raging Goblin [ST]
            Warrior's Charge1            Warrior's Charge
            Warrior's Charge2            Warrior's Charge [ST]

Forge and MA seem to disagree on whether those cards exist
Starter 2000                           ENG FOIL.zip/Rhox
            Armored Pegasus
            Bog Imp
            Coercion
            Counterspell
            Disenchant
            Drudge Skeletons
            Flame Spirit
            Flight
            Forest1
            Forest2
            Giant Growth
            Goblin Hero
            Hero's Resolve
            Inspiration
            Island1
            Island2
            Llanowar Elves
            Merfolk of the Pearl Trident
            Mountain1
            Mountain2
            Obsianus Golem
            Orcish Oriflamme
            Plains1
            Plains2
            Prodigal Sorcerer
            Python
            Rod of Ruin
            Samite Healer
            Scathe Zombies
            Shock
            Soul Net
            Spined Wurm
            Stone Rain
            Swamp1
            Swamp2
            Terror
            Venerable Monk
            Wind Drake            


I'm crossposting to the sister-thread again to encourage cooperation between the projects in solving the remaining issues.

[LordHelmchen]

Some small progress on this:

• Include Dragon's Maze in cardPicsSubDirs within forge.profile.properties:
  

  cardPicsSubDirs | Open

  Code: Select all

  cardPicsSubDirs=LEA->Core Sets/1st Limited Edition Alpha/ENG.zip|LEB->Core Sets/1st Limited Edition Beta/ENG.zip|2ED->Core Sets/2nd Unlimited/ENG.zip|ARN->Expansions/Arabian Nights/ENG.zip|ATQ->Expansions/Antiquities/ENG.zip|3ED->Core Sets/3rd Revised Edition/ENG.zip|LEG->Expansions/Legends/ENG.zip|DRK->Expansions/The Dark/ENG.zip|FEM->Expansions/Fallen Empires/ENG.zip|4ED->Core Sets/4th Edition/ENG.zip|ICE->Expansions/Ice Age/ENG.zip|CHR->Special Sets/Chronicles/ENG.zip|HML->Expansions/Homelands/ENG.zip|ALL->Expansions/Alliances/ENG.zip|MIR->Expansions/Mirage/ENG.zip|VIS->Expansions/Visions/ENG.zip|5ED->Core Sets/5th Edition/ENG.zip|POR->Special Sets/Portal/ENG.zip|WTH->Expansions/Weatherlight/ENG.zip|TMP->Expansions/Tempest/ENG.zip|STH->Expansions/Stronghold/ENG.zip|EXO->Expansions/Exodus/ENG.zip|PO2->Special Sets/Portal Second Age/ENG.zip|USG->Expansions/Urza's Saga/ENG.zip|ULG->Expansions/Urza's Legacy/ENG.zip|6ED->Core Sets/6th Classic Edition/ENG.zip|UDS->Expansions/Urza's Destiny/ENG.zip|PTK->Special Sets/Portal Three Kingdoms/ENG.zip|S99->Special Sets/Starter 1999/ENG.zip|MMQ->Expansions/Mercadian Masques/ENG.zip|NMS->Expansions/Nemesis/ENG.zip|S00->Special Sets/Starter 2000/ENG.zip|PCY->Expansions/Prophecy/ENG.zip|INV->Expansions/Invasion/ENG.zip|PLS->Expansions/Planeshift/ENG.zip|7ED->Core Sets/7th Edition/ENG.zip|APC->Expansions/Apocalypse/ENG.zip|ODY->Expansions/Odyssey/ENG.zip|TOR->Expansions/Torment/ENG.zip|JUD->Expansions/Judgment/ENG.zip|ONS->Expansions/Onslaught/ENG.zip|LGN->Expansions/Legions/ENG.zip|SCG->Expansions/Scourge/ENG.zip|8ED->Core Sets/8th Edition/ENG.zip|MRD->Expansions/Mirrodin/ENG.zip|DST->Expansions/Darksteel/ENG.zip|5DN->Expansions/Fifth Dawn/ENG.zip|CHK->Expansions/Champions of Kamigawa/ENG.zip|BOK->Expansions/Betrayers of Kamigawa/ENG.zip|SOK->Expansions/Saviors of Kamigawa/ENG.zip|9ED->Core Sets/9th Edition/ENG.zip|RAV->Expansions/Ravnica City of Guilds/ENG.zip|GPT->Expansions/Guildpact/ENG.zip|DIS->Expansions/Dissension/ENG.zip|CSP->Expansions/Coldsnap/ENG.zip|TSP->Expansions/Time Spiral/ENG.zip|TSB->Expansions/Time Spiral Timeshifted/ENG.zip|PLC->Expansions/Planar Chaos/ENG.zip|FUT->Expansions/Future Sight/ENG.zip|10E->Core Sets/10th Edition/ENG.zip|LRW->Expansions/Lorwyn/ENG.zip|MOR->Expansions/Morningtide/ENG.zip|SHM->Expansions/Shadowmoor/ENG.zip|EVE->Expansions/Eventide/ENG.zip|ALA->Expansions/Shards of Alara/ENG.zip|CFX->Expansions/Conflux/ENG.zip|ARB->Expansions/Alara Reborn/ENG.zip|HOP->Special Sets/Planechase/ENG.zip|M10->Core Sets/Magic 2010/ENG.zip|ZEN->Expansions/Zendikar/ENG.zip|WWK->Expansions/Worldwake/ENG.zip|ROE->Expansions/Rise of the Eldrazi/ENG.zip|M11->Core Sets/Magic 2011/ENG.zip|SOM->Expansions/Scars of Mirrodin/ENG.zip|MBS->Expansions/Mirrodin Besieged/ENG.zip|NPH->Expansions/New Phyrexia/ENG.zip|COM->Special Sets/Commander/ENG.zip|M12->Core Sets/Magic 2012/ENG.zip|ISD->Expansions/Innistrad/ENG.zip|DKA->Expansions/Dark Ascension/ENG.zip|AVR->Expansions/Avacyn Restored/ENG.zip|PC2->Special Sets/Planechase 2012/ENG.zip|M13->Core Sets/Magic 2013/ENG.zip|RTR->Expansions/Return to Ravnica/ENG.zip|GTC->Expansions/Gatecrash/ENG.zip|DGM->Expansions/Dragon's Maze/ENG.zip|VAN->Spedial Sets/Vanguard/ENG NTR.zip|ARC->Special Sets/Archenemy/ENG.zip

• Goblin Hero changed some filenames within the zips, as Magic Album is case agnostic whereas Forge apparently isn't.
  I think there are three more cards (Alpha and Revised: Will-o'-the-Wisp; Dragon's Maze: Bred for the Hunt) with just upper/lower case mismatch.
• Updated the patch to find split cards' images in the MA zips.
  

  readMApictures-v2.1.patch | Open

  Code: Select all

  Index: src/main/java/forge/ImageLoader.java
===================================================================
--- src/main/java/forge/ImageLoader.java   (revision 21955)
+++ src/main/java/forge/ImageLoader.java   (working copy)
@@ -3,6 +3,11 @@
 import java.awt.image.BufferedImage;
 import java.io.File;
 import java.io.IOException;
+import java.io.InputStream;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.zip.ZipEntry;
+import java.util.zip.ZipFile;
 
 import javax.imageio.ImageIO;
 
@@ -54,6 +59,7 @@
         // some S00 cards are really part of 6ED
         if (null == ret ) {
             String s2kAlias = ImageCache.getSetFolder("S00");
+//            System.out.println(String.format("key: %s ; s2kAlias: %s ", key, s2kAlias));
             if ( filename.startsWith(s2kAlias) ) {
                 ret = _findFile(key, path, filename.replace(s2kAlias, ImageCache.getSetFolder("6ED")));
             }
@@ -80,8 +86,9 @@
 
     private static BufferedImage _findFile(String key, String path, String filename) {
         for (String ext : _FILE_EXTENSIONS) {
+//          System.out.println(String.format("key: %s ; path: %s ; filename: %s", key, path, filename));
             File file = new File(path, filename + ext);
-            //System.out.println(String.format("Searching for %s at: %s", key, file.getAbsolutePath()));
+            System.out.println(String.format("Searching for %s at: %s", key, file.getAbsolutePath()));
             if (file.exists()) {
                 //System.out.println(String.format("Found %s at: %s", key, file.getAbsolutePath()));
                 try {
@@ -91,8 +98,66 @@
                     break;
                 }
             }
+
+            // attempt to read MA-style named image file from within zipped directory
+            String malikePath = (path + filename).replaceAll("/[^/]+$", "");
+            String malikeFilename = filename.replaceFirst(".+/", "");
+            malikeFilename= malikeFilename.replace(".full", "");
+            if (malikeFilename.matches("[A-Z][a-z]+[A-Z][a-z]+")) {//add _ to get MA's split cards
+                Pattern pat = Pattern.compile("([A-Z][a-z]+)([A-Z][a-z]+)");
+                Matcher mat = pat.matcher(malikeFilename);
+                malikeFilename = mat.replaceAll("$1_$2");
+            }
+            if (malikeFilename.matches(".+[0-9]+$")) {//adapt art index to MA-style naming
+                Pattern pat = Pattern.compile("([0-9]+)$");
+                Matcher mat = pat.matcher(malikeFilename);
+                malikeFilename = mat.replaceAll(" [$1]");
+            }
+//            System.out.println(String.format("malikePath: %s ; malikeFilename: %s ; ext: %s", malikePath, malikeFilename, ext));
+            File zippath = new File(malikePath);
+            if (zippath.exists()) {
+                if (!zippath.isDirectory()) {
+                    ZipFile zis;
+                    try {
+                        zis = new ZipFile(malikePath);
+                    } catch (IOException ex) {
+                        BugReporter.reportException(ex, "Could not open zipped directory " + malikePath + " ");
+                        break;
+                    }
+                    BufferedImage image = null;
+                    ZipEntry zipentry = zis.getEntry(malikeFilename + ext);
+                    if (null != zipentry) {
+                        try {
+                            InputStream inputstream = zis.getInputStream(zipentry);
+                            image = ImageIO.read(inputstream);
+                            //image = ImageIO.read(zis.getInputStream(zipentry));
+                        } catch (IOException ex) {
+                            BugReporter.reportException(ex, "Could not open image " + malikeFilename + ext + " from zipped directory " + malikePath + " ");
+                            break;
+                        }
+                    } else {
+                        //System.out.println(String.format("ZipEntry " + malikeFilename + ext + " not found in zipfile."));
+                    }
+                    try {
+                        zis.close();
+                    } catch (IOException ex) {
+                        BugReporter.reportException(ex, "Could not close zipped directory " + malikePath + " ");
+                        break;
+                    }
+                    if (null != image) {
+                        return image;
+                    }
+
+                } else {
+                    //System.out.println(String.format("malikePath + " is a directory instead of expected zip."));
+                    //try opening unzipped mapaths?
+                }
+               
+            } else {
+                //System.out.println(String.format("Directory " + malikePath + " does not exist."));
+            }
         }
-       
+
         return null;
     }
 }
Index: src/main/java/forge/gui/download/GuiDownloadSetPicturesLQ.java
===================================================================
--- src/main/java/forge/gui/download/GuiDownloadSetPicturesLQ.java   (revision 21999)
+++ src/main/java/forge/gui/download/GuiDownloadSetPicturesLQ.java   (working copy)
@@ -20,6 +20,11 @@
 import java.io.File;
 import java.util.Map;
 import java.util.TreeMap;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.zip.ZipEntry;
+import java.util.zip.ZipFile;
+import java.io.IOException;
 
 import org.apache.commons.lang3.StringUtils;
 
@@ -30,6 +35,7 @@
 import forge.card.CardEdition;
 import forge.item.PaperCard;
 import forge.properties.NewConstants;
+import forge.error.BugReporter;
 
 @SuppressWarnings("serial")
 public class GuiDownloadSetPicturesLQ extends GuiDownloader {
@@ -61,10 +67,52 @@
     }
 
     private void addDLObject(String urlPath, String filename, Map<String, String> downloads) {
-        File destFile = new File(NewConstants.CACHE_CARD_PICS_DIR, filename + ".jpg");
-        // System.out.println(filename);
-        if (!destFile.exists()) {
-            downloads.put(destFile.getAbsolutePath(), NewConstants.URL_PIC_DOWNLOAD + urlPath);
+        // check for existing files in MA-style zips first
+        String path = NewConstants.CACHE_CARD_PICS_DIR;
+        String malikePath = (path + filename).replaceAll("/[^/]+$", "");
+        String malikeFilename = filename.replaceFirst(".+/", "");
+        malikeFilename= malikeFilename.replace(".full", "");
+        if (malikeFilename.matches("[A-Z][a-z]+[A-Z][a-z]+")) {//add _ to get MA's split cards
+            Pattern pat = Pattern.compile("([A-Z][a-z]+)([A-Z][a-z]+)");
+            Matcher mat = pat.matcher(malikeFilename);
+            malikeFilename = mat.replaceAll("$1_$2");
+        }
+        if (malikeFilename.matches(".+[0-9]+$")) {//adapt art index to MA-style suffixing
+            Pattern pat = Pattern.compile("([0-9]+)$");
+            Matcher mat = pat.matcher(malikeFilename);
+            malikeFilename = mat.replaceAll(" [$1]");
+        }
+        File zippath = new File(malikePath);
+        Boolean filenotinzip = false;
+        if (zippath.exists()) {
+            if (!zippath.isDirectory()) {
+                ZipFile zis = null;
+                try {
+                    zis = new ZipFile(malikePath);
+                } catch (IOException ex) {
+                    BugReporter.reportException(ex, "Could not open zipped directory " + malikePath + " ");
+                }
+                ZipEntry zipentry = zis.getEntry(malikeFilename + ".jpg");
+                if (null == zipentry) {
+                    filenotinzip = true;
+                }
+                try {
+                    zis.close();
+                } catch (IOException ex) {
+                    BugReporter.reportException(ex, "Could not close zipped directory " + malikePath + " ");
+                }
+            }
+        }
+        if (filenotinzip) {
+            //System.out.printline("send state to log here?");
+            // as a quick workaround, one could try to have CACHE_CARD_PICS_DIR ignore cardPicsSubDirs from forge.progile.properties
+            // This would make the downloader look for, download and save the 230-odd conflicting image files in the three-letter-code subdirs.
+            // token handling shows this should work.
+            File destFile = new File(NewConstants.CACHE_CARD_PICS_DIR, filename + ".jpg");
+            // System.out.println(filename);
+            if (!destFile.exists()) {
+                downloads.put(destFile.getAbsolutePath(), NewConstants.URL_PIC_DOWNLOAD + urlPath);
+            }
         }
     }
 }


This leaves 223 pictures not working.