New Folder & File Name Structure for Image Releases &Torrent

[skibulk]

We are considering a revision to the folder and file structure of our releases and torrent. Some discussion has already been held here.

This reply is to Wurm's last comment.

I don't think 4KHQs should be included in the torrent. They are semi-raw (no compression or sharpening) and not intended for distribution. We only keep them for the purpose of correcting release errors.

I'm partial to numbering the sets. Using numbers as a folder sorting mechanism requires leading zeros to maintain the correct order: 01, 02, etc. Without them, alphanumeric sorting is broken: 1, 10, 11, 2, 3, etc. Each time we add a digit, we would also need to retroactively add a leading zero: 001. Also, we are missing some rare sets that would shift the number system: Summer Magic, Alternate 4th Edition.

It would be better to use release dates for sorting: 2015-09 BFZ. Dates keep proper order while providing useful information. The problem with this approach includes redundant or misleading combinations with set names. "2015-XX Commander 2015" is redundant. "2014-XX Magic 2015" is misleading since the years don't match.

Should the torrent be sorted by image variations first or last? For the torrent, I'm think it should be first (It's easier for users to download in bulk). For normal releases, I think it should be last (it's easier to upload on Mega).

XLHQ(300DPI)/Expansion Sets/BFZ

or

Expansion Sets/BFZ/XLHQ(300DPI)

[Huggybaby]

I think 4khqs should be in the torrent, because people love them. People always want the biggest and best---ALWAYS. As far as compression or sharpening, I don't have an opinion.

[charlequin]

The needs people have for integrating with applications and for browsing are pretty much opposed. With a goal of trying to serve both, I'm convinced the simplest possible file naming and arrangement is the way to go. My proposals:

• Name all the files solely by the three-letter code, and include the images in a folder inside with the exact same name, e.g. ARB.zip -> ARB/
• Put every single file in a single base directory, don't even divide by main vs. supplemental or anything. Any program will have a different system so no reason to categorize anything by default.
• Store extras (tokens, tip cards, etc.) as a subfolder of each release. We want to distribute all the extras with every set and lumping 12 ad card fronts or whatever into the main folder is pretty awkward.
• We should only include sizes that we can include for everything in the torrent. There are only two 4KHQs and I don't think we're planning on releasing any more. In fact, I think part of the trouble of keeping a torrent up has been maintaining all these sizes -- we should keep releasing all five current formats, but I think it makes sense to have a primary torrent that's just XLHQs.
• Include a script or batch file to move and rename the directories to a human-readable format. I would propose four categories (Core Sets, Expansions, Box Sets, Random Supplemental Sets) and sort inside each folder by numbers or dates -- i.e. either "001 - LEA - Limited Edition Alpha" or "1993-08 - LEA - Limited Edition Alpha." Then we could also offer any application-specific structure people describe.
• I'm not sure if we want to distribute promos in this torrent. I'm just about finished putting together a set of all released promos in XLHQ, but the folder structure doesn't really match up with what works with the main sets. My current thought is to distribute them in a secondary torrent for people who want a full collection.

[Doyle524]

Hey guys, I'm new here but I really love what you guys do with the card art and maintaining the torrent. I'm already undertaking the process of renaming the cards and sets for use with Forge, and when I finish, I could send whoever is in charge of the project the final Forge-compatible card artwork. Would that be helpful?

[Huggybaby]

Thanks man. Please ask in the Forge forum, I think a prenamed collection might be of interest there.

In this topic we're trying to assemble a central non-game specific collection that can be easily repurposed via a script or simple program.

That said, how you could contribute here is to provide us a spreadsheet with your Forge compatible layout. Forge is very popular and an excellent target for anyone writing the script.

Thanks!

[Huggybaby]

The needs people have for integrating with applications and for browsing are pretty much opposed. With a goal of trying to serve both, I'm convinced the simplest possible file naming and arrangement is the way to go. My proposals:

• Name all the files solely by the three-letter code, and include the images in a folder inside with the exact same name, e.g. ARB.zip -> ARB/
• Put every single file in a single base directory, don't even divide by main vs. supplemental or anything. Any program will have a different system so no reason to categorize anything by default.
• Store extras (tokens, tip cards, etc.) as a subfolder of each release. We want to distribute all the extras with every set and lumping 12 ad card fronts or whatever into the main folder is pretty awkward.
• We should only include sizes that we can include for everything in the torrent. There are only two 4KHQs and I don't think we're planning on releasing any more. In fact, I think part of the trouble of keeping a torrent up has been maintaining all these sizes -- we should keep releasing all five current formats, but I think it makes sense to have a primary torrent that's just XLHQs.
• Include a script or batch file to move and rename the directories to a human-readable format. I would propose four categories (Core Sets, Expansions, Box Sets, Random Supplemental Sets) and sort inside each folder by numbers or dates -- i.e. either "001 - LEA - Limited Edition Alpha" or "1993-08 - LEA - Limited Edition Alpha." Then we could also offer any application-specific structure people describe.
• I'm not sure if we want to distribute promos in this torrent. I'm just about finished putting together a set of all released promos in XLHQ, but the folder structure doesn't really match up with what works with the main sets. My current thought is to distribute them in a secondary torrent for people who want a full collection.

This is an excellent post. I agree with very much of what you said but I do have some responses that I don't have time to prepare right now.

And, some things I don't agree with right now, I might change my mind on, upon further reflection.

[Tolvo]

Hey guys, I'm new here but I really love what you guys do with the card art and maintaining the torrent. I'm already undertaking the process of renaming the cards and sets for use with Forge, and when I finish, I could send whoever is in charge of the project the final Forge-compatible card artwork. Would that be helpful?

I believe that would be helpful. I'd definitely appreciate it. I recently went through the hustle of renaming folders/cards where necessary, but then gave up when I noticed that tokens as used in Forge are named completely differently and kind of non-descript. You'll see you'll have to more or less go through all of them manually to see which is which.

Then again, instead of every user having to spend hours renaming and moving around thousands of files, wouldn't it make more sense for Forge to adapt the name structure as used by the HQ pics team/Gatherer?

[kudit]

The needs people have for integrating with applications and for browsing are pretty much opposed. With a goal of trying to serve both, I'm convinced the simplest possible file naming and arrangement is the way to go.

Since the needs are so opposed, I would suggest prioritizing the needs of applications/publishers over browsing since people can use the apps/sites when they want to browse.

I have a script that creates forge cards and tokens. Sets can be downloaded here:
http://deckmaster.info/images/forge/BFZ.zip
http://deckmaster.info/images/forge/ORI.zip
http://deckmaster.info/images/forge/Standard.zip
http://deckmaster.info/images/forge/Modern.zip (coming soon)
http://deckmaster.info/images/forge/All.zip (coming soon)
PM me or email info AT deckmaster DOT info if you notice any problems or corrections.

This may be a touchy subject, but why don't we simply have the largest quality versions of each image in the torrent and apps/people who need different versions/different crops can use a batch tool to crop/process as needed? This goes back to making the purpose of the torrent very focused and letting people with various needs process the source files how they need them.

Similarly, batch scripts or secondary torrents can be created that have files named and sized/cropped for the particular applications (like Forge or MWS or Cockatrice). This means the "source" torrent contains everything you would need to generate the other torrents but wouldn't contain crops and things that are specific to an application (like crops and low res versions). And it wouldn't be that hard to script the creation of secondary torrents with crops, low res, etc, that are named appropriately for the application and may or may not need to contain promos or marketing cards or tokens.

I do think the "source" torrent should contain promos and tokens and tip cards. I would suggest having one folder per set at the root, then within each set's folder, have the normal cards, and the following folders (if applicable): _Tokens, _Marketing, _Prerelease, _Promos (Prerelease has been separated from Promos since there might be 3 copies of a card like Blight Herder). I have an idea for how to name the marketing cards so that they can be uniquely identified and imported that I can go into if you like.

[skibulk]

I like the idea of only distributing 300DPI images - and leaving the resizing / cropping task to scripts and developers. Ideally the scripts should be cross-platform. Cropping might be more difficult than you think though - crop regions have changed multiple times and some sets have multiple crop regions mixed together (FUT, ZEN Lands, etc). People would complain and repeatedly post threads asking for assistance.

Ultimately it comes down to the Photoshopper who is doing the grunt of the work every set release (currently Charlequin). I did some Photoshopping in the past, and maintaining scripts to handle all these variant images is time consuming. Fixing release errors across 7 image variations is a real PITA, too.

On a side note, It would be nice to create better crops for all sets. Planeswalkers have always had part of the frame showing. Ideally, the frames should be masked out and the artwork placed over a black background. Tokens have never been cropped, but they could be. Again, it comes down to volunteer hours that we just don't have.

[Huggybaby]

Then again, instead of every user having to spend hours renaming and moving around thousands of files, wouldn't it make more sense for Forge to adapt the name structure as used by the HQ pics team/Gatherer?

Absolutely.

[kudit]

skibulk, if you know how the crop areas should be, I can help make a script to do the cropping. I've already scripted making the full cropped versions for forge from the complete card images.

[Doyle524]

Hey guys, I'm new here but I really love what you guys do with the card art and maintaining the torrent. I'm already undertaking the process of renaming the cards and sets for use with Forge, and when I finish, I could send whoever is in charge of the project the final Forge-compatible card artwork. Would that be helpful?

I believe that would be helpful. I'd definitely appreciate it. I recently went through the hustle of renaming folders/cards where necessary, but then gave up when I noticed that tokens as used in Forge are named completely differently and kind of non-descript. You'll see you'll have to more or less go through all of them manually to see which is which.

Then again, instead of every user having to spend hours renaming and moving around thousands of files, wouldn't it make more sense for Forge to adapt the name structure as used by the HQ pics team/Gatherer?

The tokens, using Forge's most recent naming convention, are posted here: viewtopic.php?f=26&t=5298&start=60#p152753

There are several more posted throughout that thread too.

[charlequin]

Do the crops still get a lot of use? I certainly wouldn't complain if we dropped them, but the status quo of generating them for the new releases on MEGA is pretty easy to keep going. I just like the idea of a single-size torrent for simplicity purposes -- every other size we support can be generated from XLHQs, so it might make the most sense to handle them through a script like we're talking about for directory structure.

[Wurm]

Hi people

Thanks to the great charlequin we have the Theros block reprocessed along with the Duel Deck Anthologies. I'm ready to update the torrent but I've been looking at TorrentZip. Seems like a good time to TorrentZip everything as Huggybaby suggested in the other thread.

Thing is, it's made only with Mame in mind. Since we are dealing with JPGs there's no point in wasting time trying to compress these files. As there are no settings it can't be changed. But there's the source code and we only need to change Z_BEST_COMPRESSION to Z_NO_COMPRESSION (or so it seems). We could also set the default date for files to something more "Magic" than the first release of Mame they use. Say, the date when Alpha was released? August 5, 1993. And then we would have our own CCG TorrentZip (it could be linked along with the torrent as an attachment or something).

Changing that C file looks easy. But how do you compile and make an exe? Anyone here knows?

If not, we'll have to settle with the current TorrentZip and have it waste time trying to compress JPGs (in the "best" setting no less )


Btw, I agree in all points with charlequin. Except that I would prefer having tokens [*] with the regular cards as they do serve a purpose for playing. The ads, inserts and other materials do keep them separate without meddling with the cards.

[*] and any other card meaningful for the game, like emblems or the checklist of Innistrad.

The scripts for adapting the structure to other programs should be posted elsewhere, maybe as a link in the forum post containing the torrent. But not in the torrent itself. Let it be only for JPGs.

What are crops used for exactly? Programs seem to use the full cards unless I'm missing something.

[charlequin]

Except that I would prefer having tokens [*] with the regular cards as they do serve a purpose for playing. The ads, inserts and other materials do keep them separate without meddling with the cards.

I just figure it makes sense to be consistent with the extras. Having the tokens in the main folder and the tips/ads in another feels weird to me.

What are crops used for exactly? Programs seem to use the full cards unless I'm missing something.

I don't know about these days, but back in the day that's what one used in Apprentice.