HQ PICS DOWNLOADS and Status List (MTG)

[Huggybaby]

No particular reason. Where DID you find them?
<edit> Well, the Rapidshare link is there, it's not hard to find. Often MF hoses my upload and I don't have time to try again, other times the upload goes fine then later they lose the file.

[extreme]

@lynolf
fiery bombardment is fine. The dot on deglamer - you're right, let's see if we can replace the card easily. If not, i'm sure it doesn't hurt too much
Many sets will be redone, the priorities for me would be 7th, onslaught block and maybe the mirrodin block, but i;m not the scanner.

Coldsnap theme dks are fine, not to be redone.

[Nyth]

Ok... I've had the chance to do more updates to the Excel spreadsheet now and have incorporated Huggybaby's comments. A key is now also included at the bottom of the spreadsheet.

Now that I've itemized all the latest sets out of the collection, the next step should be to create a torrent containing these sets. Within this torrent the following changes need to be made to ensure easy future updates: Rename all old files using the new convention so MWSHQ FUT Fulls.zip becomes CCGHQ MTG FUT (Future Sight) Fulls 080808.zip and FNM.zip becomes CCGHQ MTG FNM (Friday Night Magic Promos) Fulls 080808.zip. I'm suggesting we use the nice round date of 08/08/08 for all previous filenames to ensure that we don't confuse them in the future and to keep everything consistent.

If everyone is ok with this, the next bit open to discussion is regarding folder structure within the torrent, and whether to still include MWS related themes, databases, decks and the like.

Then if everyone is ok with the above, I can start tackling the crops.

Also, I will be seeding the above torrent continuously on a slow but stable connection.

[staggerwingjtstw]

I think a torrent that is independant of the pics for the themes, databases and etc would be preferable. I have a 250g hard drive, so d/ling everything (Fulls, crops, themes, and d/bs) and deleting what I don't want is not a problem. But at one time I was not so lucky and such a d/l would have killed my computer. For those who don't want the pics for this reason or that, something simple like just the themes would be beneficial.

Or if someone like me (but not me because my wife gets mad at me when I don't turn off my computer) was willing, you could host one HUGE torrent with everything, plust 3 smaller torrents of the fulls, the crops, and the misc. That's my two cents.

[Huggybaby]

Thanks again for the spreadsheet Nyth, nice work and very helpful.

Rename all old files using the new convention so MWSHQ FUT Fulls.zip becomes CCGHQ MTG FUT (Future Sight) Fulls 080808.zip and FNM.zip becomes CCGHQ MTG FNM (Friday Night Magic Promos) Fulls 080808.zip. I'm suggesting we use the nice round date of 08/08/08 for all previous filenames to ensure that we don't confuse them in the future and to keep everything consistent.

I guess you're talking about doing this for the new torrent? The torrent doesn't need such long names, FNM.zip is fine. The reason is because the torrent itself is dated, comes from CCGHQ, is for MTG only, and the pics are in Fulls and Crops folders, so all that is self evident. The direct downloads from MF and RS have long names because I don't know where the links will end up, and I want to make sure that wherever they do end up it is clear where they came from.

That does leave two possibilities though, either "FNM.zip" or "FNM (Friday Night Magic).zip". For now I prefer the former.

If everyone is ok with this, the next bit open to discussion is regarding folder structure within the torrent, and whether to still include MWS related themes, databases, decks and the like.

The main torrent only has pics. I only included themes etc. in the update torrent, to pad it out as an experiment. Since bases, decks and themes change every time a new set comes out they shouldn't be included in the main pics torrent.

I think a torrent that is independant of the pics for the themes, databases and etc would be preferable.

Yes, see above.

I have a 250g hard drive, so d/ling everything (Fulls, crops, themes, and d/bs) and deleting what I don't want is not a problem.

There is never a need to download everything and then delete stuff since you can pick and choose from within the torrent.

you could host one HUGE torrent with everything, plust 3 smaller torrents of the fulls, the crops, and the misc.

Again, the pics torrent will never be split, it will always contain both crops and fulls. Because as I said, you can pick only what you want, and splitting the torrent badly decimates the seeding pool.

Huggy

[Teppic]

Is there any reason that the torrent is structured to contain zip files, rather than just directories with the images themselves? The zip files don't really gain you much compression on already compacted jpg images and this approach would mean that people with an existing set of HQ images could just point the torrent at that directory and easily download what they are missing. Also, if an individual image is renamed or improved then just it changes, rather than the zip file for that set. Just some thoughts!

[Snacko]

Torrents operate on chunk not on file basis. If any file in the chunk is invalid(checksum fail on a chunk), you have to download the whole chunk. To make chunk more or less equal to a file it would need to be around 200KiB(for full pictures). Considering the torrent is around 3GiB in size it would mean around 15728 chunks, however for optimal protocol overhead a value of ~1000-2000 chunks is advised. Bigger amount of chunks means bigger overhead thus slowing the whole torrent down, but with bigger chunks you need to download the whole chunk anyway for any file change in that chunk (mostly 2MiB for 2GiB+ torrents).
Also there's file system overhead while reading thousands of small files vs a hundred of bigger ones.
The other issue is that most files you need to download is a whole set anyway as mostly there are no single file changes.
The way you want to do it could easily be achieved by a rsync mirror however it's centralised and the MWSPA does the same.

[emperor799]

I don't know much about archives, but would it e possible to code into a program (say MWS2) the ability to look into archives for images which it then displays onscreen without having to extract any of the individual files into a temp directory or some such?

[Fritti]

Thanks again for the spreadsheet Nyth, nice work and very helpful.

Indeed, very nice work Nyth!

The torrent doesn't need such long names, FNM.zip is fine. The reason is because the torrent itself is dated, comes from CCGHQ, is for MTG only, and the pics are in Fulls and Crops folders, so all that is self evident. The direct downloads from MF and RS have long names because I don't know where the links will end up, and I want to make sure that wherever they do end up it is clear where they came from.

I think that the date also needs to be in the title of the .torrent zip files, because then it will be way easier to see what files are updated from the .torrent. Not all of the .zip files will have the same date as the torrent, and besides I might not have the exact date of the torrent in mind, or even the .torrent file lying around to remind me. Having the date as part of the .zip file name makes that way easier. The CCGHQ MTG part is unnecessary I think, although I wouldn't mind it either way.

That does leave two possibilities though, either "FNM.zip" or "FNM (Friday Night Magic).zip". For now I prefer the former.

Hmm, for a newbie such as me the descriptions are pretty attractive. Then again I'm fine with just the abbreviations as well since there's a nice README.txt that should explain all.

The main torrent only has pics. I only included themes etc. in the update torrent, to pad it out as an experiment. Since bases, decks and themes change every time a new set comes out they shouldn't be included in the main pics torrent.

Agreed.

Nyth, I think that the Collectors Edition zip should also be in yellow - the zip file and folder are still called "B" instead of the yet-to-be-determined abbreviation for the Collector's Edition. Also, could you run torrentzip on your collection, and then post the md5sums so we can compare our downloads? Thanks!

[Fritti]

I'm visually comparing the two MED sets right now (the one from the Llorwyn update torrent with the one from Mediafire) and I'm not able to spot which one is the "right one". The quality seems to be about the same, however the card text differs (along with the card type and rarity). For example, compare the Argivian Archeologist, or Animate Dead.

I guess someone with MTGO needs to check which of the two sets is correct...

[Fritti]

About the new Eventide Tips & Tricks cards (thanks!!): wouldn't it be better to create a top-level directory 'TIP' for all tips & tricks, and then name the files "eventide1.full.jpg" etc? That way all tips & tricks are compiled, just like the TK, PRE and REL sets. Just a thought.

[emperor799]

Nyth, I think that the Collectors Edition zip should also be in yellow - the zip file and folder are still called "B" instead of the yet-to-be-determined abbreviation for the Collector's Edition.

I've always just shortened it to BCE myself.

[Huggybaby]

Teppic said: "Is there any reason that the torrent is structured to contain zip files, rather than just directories with the images themselves?"
Already answered here: http://slightlymagic.net/forum/viewtopic.php?p=5292#p5292
Again, torrents cannot handle 30,000 files! If you don't believe me, then try it.

Snacko, I adjust piece size to get 10,000-15,000 pieces, I don't worry about overhead. Smaller pieces allow folks to join the swarm quicker and make updates more efficient.

emperor799 said: "I don't know much about archives, but would it be possible to code into a program (say MWS2) the ability to look into archives for images which it then displays onscreen without having to extract any of the individual files into a temp directory or some such?" Yes, it's possible, and I think the next MWS will be able to do that. I've mentioned it to Det in the past anyway.

Fritti, if the torrent is released on a certain date, then you already know that every file in the torrent is current as of that date (barring mistakes of course). When the set is actually released is immaterial as long as the user knows he has the latest available. If we were archiving every set released then it would be a different matter.
If you want to know which MED set is more current, check them against the MWSPA update list.

It's clear that the tips and tricks cards will have to be organized somehow and your idea has as much merit as any.

[Fritti]

Fritti, if the torrent is released on a certain date, then you already know that every file in the torrent is current as of that date (barring mistakes of course). When the set is actually released is immaterial as long as the user knows he has the latest available.

Of course, I agree with this. My point was, that once I forget the torrent date, I have no way of knowing which version I have, and therefore no way of knowing what updates are released after the torrent has been compiled. And I think it's pretty easy to forget the torrent date...

If you want to know which MED set is more current, check them against the MWSPA update list.

Ah, of course, doh. I'll do that. Still seems strange that the card text, type and rarity seems to be the only thing that differed though. I sense a story here

[Nyth]

Ok, so I've now gone back through the Crops and have added them back into the Excel spreadsheet.

Not as many changes were evident as with the fulls. Things that need checking for the Crops are:

Which is the correct Beta crops file? Is one of them the BCE (Beta Collector's Edition)?

Assumption is that the LOR Torrent 8E Crop set is newer than the MediaFire link.

Assumption is that the LOR Torrent RAV Crop set is newer than the MediaFire link.

Suggest that crops of EVG and I2P be removed from Fulls zip and new Fulls and Crops zip be created.

Have updated 6E and AP links and have added EVE link.

After that whole exercise, the next step has to do with fixing the inconsistent names for zips and/or folders within zips. Are we now following the list for set names as posted in http://slightlymagic.net/forum/viewtopic.php?f=15&t=522? Or does that also need updating?

Also, I haven't checked any of these sets against MWSPA 1.16's latest update list. Is it safe to do so or will the sets in the Excel sheet turn out to be newer than those downloadable on MWSPA? I only mention this because I've tested this with 8E Fulls and it seems to think that the torrent and MF files are older... Hmmm...

Fritti, I will look at including the MD5 checksums for the files on this list for my next update.

Love the idea for the Tips and Tricks. That would work perfectly.

As for the tokens and promos, is there a reason a new zip hasn't been released with the latest block zips included? We currently have TK, Shadowmoor Tokens, Eventide Tokens, Tokens by MahamotiD and so on. You can't just update the full token zip (or variants by different people) because then some images will be replaced (eg: Elemental from Shadowmoor needs to be numbered to not override Kamigawa's Elemental token). The promos just needs the LRW, MOR, SHM, EVE ones added.