Page 1 of 19

Upscale low res card images to high res using Deep Learning

PostPosted: 13 Jul 2017, 23:15
by austinio7116
I have been working on a project to train a neural network (Deep Convolutional Generative Adversarial Network) to upsample images from 265x370 to 1060x1480 resolution. There is plenty of training data as we have HQ scans of all but the most recent set and, once trained, the network can be used to upsample the new sets where we only have low-res spoilers. The network is only trained on parts of the card other than the art, so the art does get a bit modified with the current version. Here is a sample of my first attempt - more examples to follow if people are interested:

low res:
card.png
Low Res spoiler input


high res:
Ammit Eternal.xlhq.png
High Res Render Output



EDIT:

In case you are reading this thread for the first time - here's how the upscaling looks now (this image shows the low res, the upscaled output - then as we have them now the high res scans too):

compare1.jpg


And here is a video showing the effect up close: https://www.youtube.com/watch?v=qpFsaIc8BgU

And now with the new ESRGAN model added as the third image:

comparison.jpg

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 14 Jul 2017, 04:47
by kudit
Very interested. Please keep us posted!

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 14 Jul 2017, 09:55
by austinio7116
Improved reconstruction using just the centre of each patch - side by side comparison with nearest neighbour. I've got this running on the full set now, so will get a better idea of where I need to improve the training.

compare.jpg

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 16 Jul 2017, 07:51
by austinio7116
Here is another attempt this time training on the art too:

Aerial Guide.full.png.xhlq.png.jpg
Abrade.full.png.xhlq.png.jpg
Desert of the Glorified.full.png.xhlq.png.jpg
Nicol Bolas, God-Pharaoh.full.png.xhlq.png.jpg
Appealauthority.full.png.xhlq.png.jpg

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 16 Jul 2017, 09:20
by austinio7116
I'm going to try two things now. First tweaking the training parameters, then testing with a smoothed training set i.e. remove the dot matrix effect of the high Res scans.

I am also trying to work out why the borders seem to be increasing in contrast too much.

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 16 Jul 2017, 13:52
by kudit
We have hyper high resolution images from the last few sets from the surveys that would be better end results for training. My primary use would be to have the OCR area cleared up from the image gallery versions as sometimes it's hard to read the artist name. What are you using to code this in?

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 16 Jul 2017, 14:02
by austinio7116
Tensorflow python. Good idea to use the survey images - I'll try that. I have them already somewhere I think.

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 16 Jul 2017, 14:07
by austinio7116
Based on this approach: https://github.com/david-gpu/srez

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 16 Jul 2017, 22:15
by austinio7116
Here are a few examples using smart blurred (photoshop) versions of the high res training scans. Some improvement to the noise in the art, but a slight reduction in detail elsewhere. Still no improvement in the high contrast in the borders:

Abrade.full.png.xhlq.png
Aerial Guide.full.png.xhlq.png
Ammit Eternal.full.png.xhlq.png
Angel of the God-Pharaoh.full.png.xhlq.png


Training with the high res survey images now.

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 17 Jul 2017, 00:02
by Bog Wraith
Can someone please post the link to the survey that has the HQ images? I had this for Amonkhet and I'd like to get the images for use until our own CCGHQ imaging team release our in house version.

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 17 Jul 2017, 05:25
by austinio7116
I am training on the AKH survey images and using the model to upscale HOU. There are no HOU HQ survey images that I know of - if there were this project would not be necessary.

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 17 Jul 2017, 18:26
by austinio7116
Using the high res survey images for training I've seen improved results in the art, but the border (especially the artist/collection no area) is not so good:

Abrade.full.png.xhlq.png


It could be the training parameters that differ though so I need to try a few more things out. Even if this is as good as I can get it however I could combine the outputs of two of the networks we have so far to give this:

combinedresult.jpg


This uses the border and text from the network trained on normal HQ scans and the image from the network trained on the survey images. Ideally I'd like to get a single network producing good results though.

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 18 Jul 2017, 07:43
by austinio7116
I've decided to process the whole set (this will take around 48 hours) with what I think is the best results I have so far (posted above at 16 Jul 2017, 07:51). That will provide us with a full set of upscaled images albeit with some artifacts on the art. Once I have that I can try to find a better set of training parameters to produce good results from the survey image trained model and either combine the two or, if I can get good results across the whole card with that model, replace them.

I will then give the invocations a go too.

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 18 Jul 2017, 07:59
by Agetian
Very nice! :) This is an awesome project!

- Agetian

Re: Upscale low res card images to high res using Deep Learn

PostPosted: 18 Jul 2017, 19:34
by Bog Wraith
austinio7116 wrote:I am training on the AKH survey images and using the model to upscale HOU. There are no HOU HQ survey images that I know of - if there were this project would not be necessary.
It hasn't been necessary up till now either. The survey images have been available for the last few sets so naturally I thought that this set would be done as well.

Glad you're doing this but I was under the impression you were doing this for future sets to get these HQ images faster than even with the survey method.

Your results look very promising and I look forward to seeing it when done, not only for this set, but hopefully for others in the future too. 8)