Upscale low res card images to high res using Deep Learning
Discuss Card Scans and Other Artwork Here
Moderator: CCGHQ Admins
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 26 Jul 2017, 16:40
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by jenjia » 26 Jul 2017, 17:23
Just my 2 cents,
There are programs/websites where you can make "fake" cards, just use those for the text/borders/etc and use your algorithm to generate only the image.
In the end, if will be much better
There are programs/websites where you can make "fake" cards, just use those for the text/borders/etc and use your algorithm to generate only the image.
In the end, if will be much better
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 26 Jul 2017, 17:28
True, but that would take ages. The computer does all the work here.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by nesci » 26 Jul 2017, 20:26
Really good job and interesting posts.
I think @jenjia has a point and I also think there's a way to make the computer do "almost" all the work.
1) Create all the cards without the artwork with @Ahkren's cards generator.
2) Use ImageMagick to extract the artwork of your rendered cards
3) Use ImageMagick to put the artwork on the cards from the generator
What do you think?
I think @jenjia has a point and I also think there's a way to make the computer do "almost" all the work.
1) Create all the cards without the artwork with @Ahkren's cards generator.
2) Use ImageMagick to extract the artwork of your rendered cards
3) Use ImageMagick to put the artwork on the cards from the generator
What do you think?
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 26 Jul 2017, 21:51
Thanks - I'll take a look at Ahkren's generator - looks good. I guess the only fiddly bit would be scripting imagemagick differently for each card type (full art lands, aftermath, planeswalkers etc).
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 26 Jul 2017, 21:55
Also - the upsampling effect on the art will never be as good as the borders as each image is different - so I'm not sure how much it would improve the generator results to use these upsampled images - it does seem to learn a good sharpening model though so perhaps it would be better than nothing.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 27 Jul 2017, 13:13
Got the generator running - just need to create some masks for imagemagick and I'll be ready to try out your suggestions. Still not sure quite how the planeswalker art is going to work though.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 27 Jul 2017, 13:36
I may be able to get high res art for all the planeswalkers online now I suppose
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 30 Jul 2017, 14:28
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 30 Jul 2017, 15:21
Here is how the HQ card generator border looks in comparison:
Clearly this is much better in terms of the text, however I would argue that the upscaled image from the network is closer to the real look of the card. I may continue to use the upscaled images (as I only use 150dpi on Android forge - and at that resolution the upscaled images are pretty much as good as downsampled scans. It depends what you want from the cards though - I still may try to script up some imagemagick to see how the whole set looks with the the HQ borders and text. I can also try increasing the amount of training data and the size of the network to see if I can get the super resolution any better.
Clearly this is much better in terms of the text, however I would argue that the upscaled image from the network is closer to the real look of the card. I may continue to use the upscaled images (as I only use 150dpi on Android forge - and at that resolution the upscaled images are pretty much as good as downsampled scans. It depends what you want from the cards though - I still may try to script up some imagemagick to see how the whole set looks with the the HQ borders and text. I can also try increasing the amount of training data and the size of the network to see if I can get the super resolution any better.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by jenjia » 31 Jul 2017, 11:13
You should try to train your deep learn network with only the artwork (low & high res), when using the HQ Card Generator.
I think the borders are just noise for the network. Just a guess.
(Disclaimer: I also work with Computer Vision, if you want you can check my CCG Scanners for iOS and Android, named: MTG Manager, Card Scanner for YuGiOh, Card Scanner for PokeTCG, Card Scanner for Vanguard )
I think the borders are just noise for the network. Just a guess.
(Disclaimer: I also work with Computer Vision, if you want you can check my CCG Scanners for iOS and Android, named: MTG Manager, Card Scanner for YuGiOh, Card Scanner for PokeTCG, Card Scanner for Vanguard )
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 31 Jul 2017, 11:54
Yes, I would have to retrain on just the art to get the best results.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by nesci » 01 Aug 2017, 12:57
How do you train your deep learning network?
You got me interested in your project and I have a few very important pictures that are low quality that I would be more than happy to enlarge their resolution.
You got me interested in your project and I have a few very important pictures that are low quality that I would be more than happy to enlarge their resolution.
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 02 Aug 2017, 07:38
I used the code provided here as a starting point - just tweaking some parameters and a few changes to the model to better work for this problem (I removed the batch normalization layers from the generative model to avoid different parts of the final card having different intensity profiles):
https://github.com/david-gpu/srez
That network is designed to upsample a 16x16 square to 64x64 by training on high res image squares (which it downsamples 4x to use as input). I wrote some python tensorflow code to use the trained model to upsample new 16x16 inputs (as the provided code did not include this part) then wrote some python to loop over a low res spoiler HOU card image, chopping it into 16x16 blocks every 4 pixels (so the blocks overlap). These 16x16 blocks are then upsampled by the network in batches (to avoid running out of memory on the graphics card - all this works with tensorflow using CUDA for best performance). The 64x64 blocks are then patched together into the high res card averaging the overlapping blocks for best results.
The training data I used in the latest version is the AKH high res survey images with a bit of de-noising applied in Photoshop to remove JPEG noise and artifacts. These images are chopped up into 64x64 blocks (100,000+) by looping over the card in increments of 16 pixels.
If you want to use these kinds of networks to upsample images you need to have high res examples of similar images to train the network on - if your training images don't contain similar features to the images you want to upsample you will not get good results.
Training takes around 6 hours (before my graphics card runs out of memory - I'd like to train it longer) - then each card takes 3 minutes to upsample.
https://github.com/david-gpu/srez
That network is designed to upsample a 16x16 square to 64x64 by training on high res image squares (which it downsamples 4x to use as input). I wrote some python tensorflow code to use the trained model to upsample new 16x16 inputs (as the provided code did not include this part) then wrote some python to loop over a low res spoiler HOU card image, chopping it into 16x16 blocks every 4 pixels (so the blocks overlap). These 16x16 blocks are then upsampled by the network in batches (to avoid running out of memory on the graphics card - all this works with tensorflow using CUDA for best performance). The 64x64 blocks are then patched together into the high res card averaging the overlapping blocks for best results.
The training data I used in the latest version is the AKH high res survey images with a bit of de-noising applied in Photoshop to remove JPEG noise and artifacts. These images are chopped up into 64x64 blocks (100,000+) by looping over the card in increments of 16 pixels.
If you want to use these kinds of networks to upsample images you need to have high res examples of similar images to train the network on - if your training images don't contain similar features to the images you want to upsample you will not get good results.
Training takes around 6 hours (before my graphics card runs out of memory - I'd like to train it longer) - then each card takes 3 minutes to upsample.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 02 Aug 2017, 07:39
I have increased the size of the network layers slightly in the hope of more detailed upsampling - I am processing the full set again as I think this is a (very) small improvement over the previous versions:
The artist info text is clearer, the text is very slightly sharper and better representative of the original font and the edition icon and mana icons are better.
I may process this version on the full set, then I will look to increase the amount of training data and try to find a way to get the network to train for longer without running out of memory.
The artist info text is clearer, the text is very slightly sharper and better representative of the original font and the edition icon and mana icons are better.
I may process this version on the full set, then I will look to increase the amount of training data and try to find a way to get the network to train for longer without running out of memory.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Who is online
Users browsing this forum: No registered users and 26 guests