It is currently 21 Sep 2017, 05:08
   
Text Size

Upscale low res card images to high res using Deep Learning

Discuss Card Scans and Other Artwork Here

Moderator: CCGHQ Admins

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 26 Jul 2017, 16:40

All done.

Neheb, the Eternal.full.png
Drivendespair.full.png


PM me if you are interested
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby jenjia » 26 Jul 2017, 17:23

Just my 2 cents,

There are programs/websites where you can make "fake" cards, just use those for the text/borders/etc and use your algorithm to generate only the image.

In the end, if will be much better
jenjia
 
Posts: 10
Joined: 19 Jun 2015, 13:05
Has thanked: 2 times
Been thanked: 2 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 26 Jul 2017, 17:28

True, but that would take ages. The computer does all the work here.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby nesci » 26 Jul 2017, 20:26

Really good job and interesting posts.

I think @jenjia has a point and I also think there's a way to make the computer do "almost" all the work.

1) Create all the cards without the artwork with @Ahkren's cards generator.
2) Use ImageMagick to extract the artwork of your rendered cards
3) Use ImageMagick to put the artwork on the cards from the generator

What do you think?
nesci
 
Posts: 5
Joined: 18 Jul 2017, 02:41
Has thanked: 1 time
Been thanked: 0 time

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 26 Jul 2017, 21:51

Thanks - I'll take a look at Ahkren's generator - looks good. I guess the only fiddly bit would be scripting imagemagick differently for each card type (full art lands, aftermath, planeswalkers etc).
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 26 Jul 2017, 21:55

Also - the upsampling effect on the art will never be as good as the borders as each image is different - so I'm not sure how much it would improve the generator results to use these upsampled images - it does seem to learn a good sharpening model though so perhaps it would be better than nothing.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 27 Jul 2017, 13:13

Got the generator running - just need to create some masks for imagemagick and I'll be ready to try out your suggestions. Still not sure quite how the planeswalker art is going to work though.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 27 Jul 2017, 13:36

I may be able to get high res art for all the planeswalkers online now I suppose
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 30 Jul 2017, 14:28

Another example of the final superresolution effect - I think it does a good enough job on the art to make it worth using in combination with the HQ card generator:

These images show first the low res image, then bicubic upsampling - then the neural network output:

Sifter-Wurm_crop.gif


Sifter-Wurm.gif
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 30 Jul 2017, 15:21

Here is how the HQ card generator border looks in comparison:

Sifter-Wurm-HQborder.gif


Clearly this is much better in terms of the text, however I would argue that the upscaled image from the network is closer to the real look of the card. I may continue to use the upscaled images (as I only use 150dpi on Android forge - and at that resolution the upscaled images are pretty much as good as downsampled scans. It depends what you want from the cards though - I still may try to script up some imagemagick to see how the whole set looks with the the HQ borders and text. I can also try increasing the amount of training data and the size of the network to see if I can get the super resolution any better.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby jenjia » 31 Jul 2017, 11:13

You should try to train your deep learn network with only the artwork (low & high res), when using the HQ Card Generator.

I think the borders are just noise for the network. Just a guess.

(Disclaimer: I also work with Computer Vision, if you want you can check my CCG Scanners for iOS and Android, named: MTG Manager, Card Scanner for YuGiOh, Card Scanner for PokeTCG, Card Scanner for Vanguard )
jenjia
 
Posts: 10
Joined: 19 Jun 2015, 13:05
Has thanked: 2 times
Been thanked: 2 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 31 Jul 2017, 11:54

Yes, I would have to retrain on just the art to get the best results.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby nesci » 01 Aug 2017, 12:57

How do you train your deep learning network?
You got me interested in your project and I have a few very important pictures that are low quality that I would be more than happy to enlarge their resolution.
nesci
 
Posts: 5
Joined: 18 Jul 2017, 02:41
Has thanked: 1 time
Been thanked: 0 time

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 02 Aug 2017, 07:38

I used the code provided here as a starting point - just tweaking some parameters and a few changes to the model to better work for this problem (I removed the batch normalization layers from the generative model to avoid different parts of the final card having different intensity profiles):

https://github.com/david-gpu/srez

That network is designed to upsample a 16x16 square to 64x64 by training on high res image squares (which it downsamples 4x to use as input). I wrote some python tensorflow code to use the trained model to upsample new 16x16 inputs (as the provided code did not include this part) then wrote some python to loop over a low res spoiler HOU card image, chopping it into 16x16 blocks every 4 pixels (so the blocks overlap). These 16x16 blocks are then upsampled by the network in batches (to avoid running out of memory on the graphics card - all this works with tensorflow using CUDA for best performance). The 64x64 blocks are then patched together into the high res card averaging the overlapping blocks for best results.

The training data I used in the latest version is the AKH high res survey images with a bit of de-noising applied in Photoshop to remove JPEG noise and artifacts. These images are chopped up into 64x64 blocks (100,000+) by looping over the card in increments of 16 pixels.

If you want to use these kinds of networks to upsample images you need to have high res examples of similar images to train the network on - if your training images don't contain similar features to the images you want to upsample you will not get good results.

Training takes around 6 hours (before my graphics card runs out of memory - I'd like to train it longer) - then each card takes 3 minutes to upsample.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 02 Aug 2017, 07:39

I have increased the size of the network layers slightly in the hope of more detailed upsampling - I am processing the full set again as I think this is a (very) small improvement over the previous versions:

Abrade.full.png.xhlq (5).png


The artist info text is clearer, the text is very slightly sharper and better representative of the original font and the edition icon and mana icons are better.

I may process this version on the full set, then I will look to increase the amount of training data and try to find a way to get the network to train for longer without running out of memory.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

PreviousNext

Return to Pictures

Who is online

Users browsing this forum: No registered users and 9 guests


Who is online

In total there are 9 users online :: 0 registered, 0 hidden and 9 guests (based on users active over the past 10 minutes)
Most users ever online was 279 on 11 Jul 2013, 22:03

Users browsing this forum: No registered users and 9 guests

Login Form