Upscale low res card images to high res using Deep Learning
Discuss Card Scans and Other Artwork Here
Moderator: CCGHQ Admins
Re: Upscale low res card images to high res using Deep Learn
by skibulk » 18 Jul 2017, 19:54
This is an interesting project. Have you dabbled with OCR at all? I think you might get better results if you use machine learning to re-construct the card without the text and then layer the text back in using a real font. You can use OCR to detect positioning and line breaks, to calculate leading. Microsoft's OCR service is pretty awesome: https://azure.microsoft.com/en-us/servi ... er-vision/
I'd suggest using the HQ Card Generator instead, but you'll always encounter a delay in the release of new frames and symbols. It does have all the fonts though. You can find that software here: viewtopic.php?f=30&t=17795
I'd suggest using the HQ Card Generator instead, but you'll always encounter a delay in the release of new frames and symbols. It does have all the fonts though. You can find that software here: viewtopic.php?f=30&t=17795
Last edited by skibulk on 18 Jul 2017, 19:58, edited 1 time in total.
-
skibulk - HQ Team Member
- Posts: 995
- Joined: 19 Jul 2010, 20:34
- Location: Northeast USA
- Has thanked: 379 times
- Been thanked: 335 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 18 Jul 2017, 19:58
Yes, I had thought of doing that, but I'm pretty new to neural networks and the network I have is for super resolution. If I had the time to learn how to build the network you suggest (which I presume would need some custom layers) that would be the way to get very good results at least on the text. I think this super resolution method should get pretty good results if I can optimize the training process a bit more. If anyone else has the skills and time to try integrating OCR too that would be great to see though.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 18 Jul 2017, 20:39
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 19 Jul 2017, 10:35
One example where the pure superresolution approach may actually be better would be something like the "Hour of" series - where the text is overlayed on a different textured background - of course OCR would still work if you had enough background templates to cover each scenario but the generic approach may be more flexible to future changes in templates:skibulk wrote:This is an interesting project. Have you dabbled with OCR at all? I think you might get better results if you use machine learning to re-construct the card without the text and then layer the text back in using a real font. You can use OCR to detect positioning and line breaks, to calculate leading. Microsoft's OCR service is pretty awesome: https://azure.microsoft.com/en-us/servi ... er-vision/
I'd suggest using the HQ Card Generator instead, but you'll always encounter a delay in the release of new frames and symbols. It does have all the fonts though. You can find that software here: viewtopic.php?f=30&t=17795
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by Agetian » 19 Jul 2017, 14:23
Looking very good so far! Keep up the great work!
- Agetian
- Agetian
- Agetian
- Programmer
- Posts: 3471
- Joined: 14 Mar 2011, 05:58
- Has thanked: 676 times
- Been thanked: 561 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 19 Jul 2017, 15:06
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 20 Jul 2017, 12:01
Finished the first pass of processing the full HOU set. There are a few missing cards that I need to add and a bit of post processing left to do before I have something ready to share.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 20 Jul 2017, 19:31
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 20 Jul 2017, 19:35
I overlaid random foils from AKH on the foils
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 20 Jul 2017, 22:14
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 21 Jul 2017, 06:25
I think I've worked out why the survey images training set didn't work as well. If you zoom in on the high res survey jpgs there is a lot of noise around the artist/collection no text at the bottom of the card (and the rest of the text too) - I suspect that the network is learning to replicate that noise. I will try pre-filtering the training set to remove the noise and the JPEG artifacts and re-training on that.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 23 Jul 2017, 15:56
I think I've a good theory as to why the left and right borders are overexposed. The network works on small patches, so to keep the calculation within memory on my graphics card I am batching the upscaling in columns. The network contains batch normalisation layers which are therefore batch normalizing the columns containing mostly black border at the edges which is different from the random batches it sees during training. I can try removing these layers from the model, or failing that restructure the batching process sonthe black border patches are distributed evenly.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 24 Jul 2017, 13:44
Removing the normalization layers from the network has fixed all of the intensity/colour issues that were present in the first batch - especially the over-exposure of the borders. I just need to find some ideal parameters for training on the survey data and I will be able to generate a much prettier second batch of HOU images.austinio7116 wrote:I think I've a good theory as to why the left and right borders are overexposed. The network works on small patches, so to keep the calculation within memory on my graphics card I am batching the upscaling in columns. The network contains batch normalisation layers which are therefore batch normalizing the columns containing mostly black border at the edges which is different from the random batches it sees during training. I can try removing these layers from the model, or failing that restructure the batching process sonthe black border patches are distributed evenly.
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 24 Jul 2017, 14:00
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Re: Upscale low res card images to high res using Deep Learn
by austinio7116 » 26 Jul 2017, 11:02
-
austinio7116 - Posts: 451
- Joined: 10 Mar 2017, 11:59
- Has thanked: 47 times
- Been thanked: 169 times
Who is online
Users browsing this forum: No registered users and 78 guests