Upscale low res card images to high res using Deep Learning

[nesci]

ok thanks for you answer, I guess i won't be able to make it work for my pictures.

But out of curiosity, how could you train your network for only the artwork?
Taking the survey pic of AKH, cropping to only get the artwork, dividing them into 64x64 blocks and then do the same with low quality pic and 16x16 blocks and then train it with all the blocks?

[austinio7116]

Basically yes - except there is no need to generate the low quality data as the training process downsamples the high res for you.

[austinio7116]

I have processed the full set with this 3rd version. If anyone wants to host this set (from CCGHQ or elsewhere) please let me know.

[Zarthbenn]

Have you looked into Waifu2x? They have been working on a similar but different feat as far as deep learning art upscaling. While their goal is primarily art focused there might be something there to help.

Here is the same inital test you ran with their program.

Before:

After:

Your results look very promising and a huge step up from the inital run.

[austinio7116]

I love that result, especially on the art. I'll definitely have a look when I get back from my holiday.

[austinio7116]

Waifu2x looks extremely well suited to upsampling the low res cards - despite not being specifically trained for magic cards. It does a great job on the art and a reasonable job on all the icons. The text is OK too as long as you are not expecting perfection. If anyone fancies installing Waifu2x locally and batch upsampling cards I think the results would be as good as or better than what I am getting at the moment. I'll continue to refine my training process to see if I can beat Waifu and may even get round to installing it myself at some point so I can use it myself. It would be good to try training it with the high res renders too.

Thanks for pointing me to it.

[austinio7116]

I've made some more improvements to the pipeline and will be processing the XLN cards as they are spoiled - here's a mini preview of the raw output - I'll post some more fully processed images once I have them:

Spoiled low res:

Super res:

[austinio7116]

As it is now a deeper network trained for longer on more data it is performing much better on the art. It also performs very well on unseen card frames, like the new double sided cards - as well as handling the token frames despite not having trained it on tokens.

[kevlahnota]

DO you have a complete collection of this? from Alpha to the Latest Release? Where to download them? thanks.

[Bog Wraith]

As it is now a deeper network trained for longer on more data it is performing much better on the art. It also performs very well on unseen card frames, like the new double sided cards - as well as handling the token frames despite not having trained it on tokens.

The results are remarkable. I am most impressed with the software your using and its ability to learn from each successive run. It strikes me that this method can/will be used for additional graphics processing and indeed, I wonder if it could be applied to video, especially analog video when it is being converted and imported digitally.

I look forward to seeing where this might lead to.

[austinio7116]

DO you have a complete collection of this? from Alpha to the Latest Release? Where to download them? thanks.

I have only used the network to upsample the latest sets (where we only have the 265x370 spoiler images) so that I have sufficient quality images until the CCGHQ scans become available - you are much better off looking up those collections for all sets up to HOU.

[austinio7116]

I wonder if it could be applied to video, especially analog video when it is being converted and imported digitally.

Neural networks of this type are being used by the Facebooks and Twitters of the world to use super resolution for video compression.

[Bog Wraith]

Neural networks of this type are being used by the Facebooks and Twitters of the world to use super resolution for video compression.

That is correct, but that is being done with, for the most part, digital video. What I'm referring to is importing video off of VCR tape, converting it to digital, then upscaling it while it's being processed. There is so much analog video that needs to be digitally converted for both delivery, editing and in some ways most importantly for archiving purposes.

I've worked with & have converted thousands upon thousands of hours of video for over 2 decades in my field of video editing in which it was a small part of what I did in creating multi media content for delivery on different mediums, now mostly delivered via streaming as opposed to the old ways of DVD/Blue-ray platforms.

Any upscaling that is used is done so on a frame by frame basis, which is an extremely time extensive process which thus cost a lot of money to do, and the computing power necessary to do this pushes the hardware to its limits to accomplish this. It's much better today then say 10 years ago because the power on top of the line desktop workstations is much more now then it was then.
The same comparison to this is how animation or special effects rendering is done in the same frame by frame method which also takes a lot of both time & processing power to achieve.

What I see of the results you are getting is to me very inspiring and I, being a retired multi media content creator find, as a certain Vulcan science officer would say, "Fascinating"!

[Xitax]

The full spoiler is out! Can't wait to see what this looks like on the full set

[austinio7116]

The full spoiler is out! Can't wait to see what this looks like on the full set

Working on the final lot