It is currently 21 Sep 2017, 05:05
   
Text Size

Upscale low res card images to high res using Deep Learning

Discuss Card Scans and Other Artwork Here

Moderator: CCGHQ Admins

Re: Upscale low res card images to high res using Deep Learn

Postby skibulk » 18 Jul 2017, 19:54

This is an interesting project. Have you dabbled with OCR at all? I think you might get better results if you use machine learning to re-construct the card without the text and then layer the text back in using a real font. You can use OCR to detect positioning and line breaks, to calculate leading. Microsoft's OCR service is pretty awesome: https://azure.microsoft.com/en-us/servi ... er-vision/

I'd suggest using the HQ Card Generator instead, but you'll always encounter a delay in the release of new frames and symbols. It does have all the fonts though. You can find that software here: viewtopic.php?f=30&t=17795
Last edited by skibulk on 18 Jul 2017, 19:58, edited 1 time in total.
User avatar
skibulk
HQ Team Member
 
Posts: 989
Joined: 19 Jul 2010, 20:34
Location: Northeast USA
Has thanked: 378 times
Been thanked: 302 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 18 Jul 2017, 19:58

Yes, I had thought of doing that, but I'm pretty new to neural networks and the network I have is for super resolution. If I had the time to learn how to build the network you suggest (which I presume would need some custom layers) that would be the way to get very good results at least on the text. I think this super resolution method should get pretty good results if I can optimize the training process a bit more. If anyone else has the skills and time to try integrating OCR too that would be great to see though.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 18 Jul 2017, 20:39

Some examples of what version 1 will look like - processed 14 hours so far:

Dagger of the Worthy.full.jpg
Appealauthority.full.jpg
Ammit Eternal.full.jpg
Abrade.full.jpg
Forest2.full.jpg
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 19 Jul 2017, 10:35

skibulk wrote:This is an interesting project. Have you dabbled with OCR at all? I think you might get better results if you use machine learning to re-construct the card without the text and then layer the text back in using a real font. You can use OCR to detect positioning and line breaks, to calculate leading. Microsoft's OCR service is pretty awesome: https://azure.microsoft.com/en-us/servi ... er-vision/

I'd suggest using the HQ Card Generator instead, but you'll always encounter a delay in the release of new frames and symbols. It does have all the fonts though. You can find that software here: viewtopic.php?f=30&t=17795
One example where the pure superresolution approach may actually be better would be something like the "Hour of" series - where the text is overlayed on a different textured background - of course OCR would still work if you had enough background templates to cover each scenario but the generic approach may be more flexible to future changes in templates:

Hour of Revelation.full.png
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby Agetian » 19 Jul 2017, 14:23

Looking very good so far! :) Keep up the great work!

- Agetian
Agetian
Programmer
 
Posts: 3060
Joined: 14 Mar 2011, 05:58
Has thanked: 596 times
Been thanked: 391 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 19 Jul 2017, 15:06

Quite pleased to see that in some cases the art is upscaled quite unexpectedly well:

Screenshot_20170719-155959.png
Low res


Screenshot_20170719-160025.png
Upscaled
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 20 Jul 2017, 12:01

Finished the first pass of processing the full HOU set. There are a few missing cards that I need to add and a bit of post processing left to do before I have something ready to share.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 20 Jul 2017, 19:31

First full set of HOU at upsampled 200dpi is complete and post processed - should be available somewhere soon:

Hour of Devastation.full.jpg
Obelisk Spider.full.jpg
Wildfire Eternal.full.jpg
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 20 Jul 2017, 19:35

I overlaid random foils from AKH on the foils :-)
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 20 Jul 2017, 22:14

Some animated GIFs showing the upsampling effect of the network:

Wildfire-Eternalzoom.gif


Wildfire-Eternal.gif
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 21 Jul 2017, 06:25

I think I've worked out why the survey images training set didn't work as well. If you zoom in on the high res survey jpgs there is a lot of noise around the artist/collection no text at the bottom of the card (and the rest of the text too) - I suspect that the network is learning to replicate that noise. I will try pre-filtering the training set to remove the noise and the JPEG artifacts and re-training on that.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 23 Jul 2017, 15:56

I think I've a good theory as to why the left and right borders are overexposed. The network works on small patches, so to keep the calculation within memory on my graphics card I am batching the upscaling in columns. The network contains batch normalisation layers which are therefore batch normalizing the columns containing mostly black border at the edges which is different from the random batches it sees during training. I can try removing these layers from the model, or failing that restructure the batching process sonthe black border patches are distributed evenly.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 24 Jul 2017, 13:44

austinio7116 wrote:I think I've a good theory as to why the left and right borders are overexposed. The network works on small patches, so to keep the calculation within memory on my graphics card I am batching the upscaling in columns. The network contains batch normalisation layers which are therefore batch normalizing the columns containing mostly black border at the edges which is different from the random batches it sees during training. I can try removing these layers from the model, or failing that restructure the batching process sonthe black border patches are distributed evenly.
Removing the normalization layers from the network has fixed all of the intensity/colour issues that were present in the first batch - especially the over-exposure of the borders. I just need to find some ideal parameters for training on the survey data and I will be able to generate a much prettier second batch of HOU images.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 24 Jul 2017, 14:00

Here's an example of a raw output of the new network with the normalization issues fixed. I hope to find a better set of training parameters to improve the resolution further before running again on the full set:

Abrade.full.png.xhlq (1).png
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

Re: Upscale low res card images to high res using Deep Learn

Postby austinio7116 » 26 Jul 2017, 11:02

I have now found some parameters that I am happy with and am processing the full set. Here are a few raw examples:

Abrade.full.png.xhlq.png
Aerial Guide.full.png.xhlq.png


It only takes 3 minutes per card now so the full set should not take long.
austinio7116
 
Posts: 163
Joined: 10 Mar 2017, 11:59
Has thanked: 15 times
Been thanked: 46 times

PreviousNext

Return to Pictures

Who is online

Users browsing this forum: Aswan jaguar and 11 guests


Who is online

In total there are 12 users online :: 1 registered, 0 hidden and 11 guests (based on users active over the past 10 minutes)
Most users ever online was 279 on 11 Jul 2013, 22:03

Users browsing this forum: Aswan jaguar and 11 guests

Login Form