It is currently 24 Jul 2019, 02:29
   
Text Size

Working on loading cards lazily

Post MTG Forge Related Programming Questions Here

Moderators: timmermac, friarsol, Blacksmith, KrazyTheFox, Agetian, CCGHQ Admins

Working on loading cards lazily

Postby Myrd » 30 Oct 2016, 15:34

Hey all,

One of the things that annoys me the most about Forge is that it takes forever to load (e.g. several minutes) - which is especially painful on mobile when I just want to get a quick game or two in while commuting.

So I started working on fixing this. The idea is that instead of loading all the cards at startup, we only load cards as they're needed. For example, when you select a deck to play, it will load the cards needed by that deck only - and the opponents deck.

Some screens will probably still require loading the full DB - such as the deck construction UI that lets you search all cards.

So far, I have a hacky prototype that seems to mostly work. I load no cards at startup (making it blazing fast), but when a deck is loaded (currently, when entering the deck manager for example - but can probably be done even lazier than that), it will attempt to load all the cards in it.

When a card is not found in the DB, it tries to compute the filename for that card and load it from there - adding it to the relevant editions.

So far, seems promising. I'm able to load most cards lazily, though one thing that I ran into is that double-faced cards have both names in the filename - but decks only have the front face - so not sure how to best get the filename for those - maybe just prefix matching. I also haven't figured out how this interacts with e.g. random deck generation or selection by the AI - whether those need access to all cards or not.

Anyway, this is just a heads up that I'm working on this so that there's no effort duplication if someone else was about to try the same thing.
Myrd
 
Posts: 87
Joined: 24 Nov 2014, 05:58
Has thanked: 4 times
Been thanked: 31 times

Re: Working on loading cards lazily

Postby Fizanko » 30 Oct 2016, 15:49

This sounds very interesting, good luck.
Innistrad world for Forge (updated 17/11/2014)
Duel Decks for Forge - Forge custom decks (updated 25/10/2014)
User avatar
Fizanko
Tester
 
Posts: 768
Joined: 07 Feb 2014, 11:24
Has thanked: 154 times
Been thanked: 87 times

Re: Working on loading cards lazily

Postby Hanmac » 30 Oct 2016, 16:50

hm you talk about the card files, not the images or something like that?

yeah i think it might be able to make the read of them more lazy but i don't think thats what make the startup of Forge that slow. (didn't have mobile version)

imo loading the Card scripts should not slow them down.
(thats only reading the text files, it isn't even creating the cards?)
i think Forge does create "One" card for each parsed card?

for your lazy loading, you might do it in different steps.

  • like when the DeckView is opened (not the Deck Editor)
    it only needs to load the Cards for this given Deck if they are not loaded yet.
  • using DeckEditor all card files need to be loaded, and i don't think that can be changed.
  • making a Duell:
    • against AI with a fixed Deck: only need to load this one.
    • against aI with Random Deck: need to load all cards.

and about the DFC, we can change the file names to only have the name of the Front Face. The file name is only for the developers and not used anywhere.

PS: i think if you implement that, make it configurable, like it might be disabled on the desktop-version as default.
Hanmac
 
Posts: 942
Joined: 06 May 2013, 18:44
Has thanked: 229 times
Been thanked: 153 times

Re: Working on loading cards lazily

Postby Myrd » 30 Oct 2016, 17:11

Hanmac wrote:hm you talk about the card files, not the images or something like that?

yeah i think it might be able to make the read of them more lazy but i don't think thats what make the startup of Forge that slow. (didn't have mobile version)

imo loading the Card scripts should not slow them down.
(thats only reading the text files, it isn't even creating the cards?)
i think Forge does create "One" card for each parsed card?

for your lazy loading, you might do it in different steps.

  • like when the DeckView is opened (not the Deck Editor)
    it only needs to load the Cards for this given Deck if they are not loaded yet.
  • using DeckEditor all card files need to be loaded, and i don't think that can be changed.
  • making a Duell:
    • against AI with a fixed Deck: only need to load this one.
    • against aI with Random Deck: need to load all cards.

and about the DFC, we can change the file names to only have the name of the Front Face. The file name is only for the developers and not used anywhere.

PS: i think if you implement that, make it configurable, like it might be disabled on the desktop-version as default.
Making it configurable to start off sounds good. This way, I can start implementing it and we leave it off by default until all the kinks are worked out.

I'll have to get more exact numbers later, but from my experience (from profiling it on desktop) most of the startup time is spent script loading - it's like 20k files to read and parse.

For now, I've worked around the DFC problem via prefix matching but I agree that it might make sense to just change their naming convention in the future. (i.e. once I land the new loading method and we agree to make sense to make it the default - since at that point changing the DFC naming will be a pure speed improvement since it will avoid needing to do prefix matching)
Myrd
 
Posts: 87
Joined: 24 Nov 2014, 05:58
Has thanked: 4 times
Been thanked: 31 times

Re: Working on loading cards lazily

Postby Hanmac » 04 Nov 2016, 06:39

@Mryd:
its cool that you begin, but if i try to get the Preconstructed Decks i get a StackError.

Code: Select all
at forge.deck.Deck.getMain(Deck.java:114)
at forge.deck.Deck.getAllCardsInASinglePool(Deck.java:283)
at forge.deck.Deck.getAllCardsInASinglePool(Deck.java:279)
at forge.deck.Deck.convertByXitaxMethod(Deck.java:221)
at forge.deck.Deck.loadDeferredSections(Deck.java:215)
at forge.deck.Deck.getMain(Deck.java:114)
your loadDeferredSections function might cause this.
Hanmac
 
Posts: 942
Joined: 06 May 2013, 18:44
Has thanked: 229 times
Been thanked: 153 times

Re: Working on loading cards lazily

Postby Myrd » 04 Nov 2016, 22:56

Hanmac wrote:@Mryd:
its cool that you begin, but if i try to get the Preconstructed Decks i get a StackError.

Code: Select all
at forge.deck.Deck.getMain(Deck.java:114)
at forge.deck.Deck.getAllCardsInASinglePool(Deck.java:283)
at forge.deck.Deck.getAllCardsInASinglePool(Deck.java:279)
at forge.deck.Deck.convertByXitaxMethod(Deck.java:221)
at forge.deck.Deck.loadDeferredSections(Deck.java:215)
at forge.deck.Deck.getMain(Deck.java:114)
your loadDeferredSections function might cause this.
Thanks for the heads up. Working on a fix.

EDIT: Fixed.
Myrd
 
Posts: 87
Joined: 24 Nov 2014, 05:58
Has thanked: 4 times
Been thanked: 31 times

Re: Working on loading cards lazily

Postby Myrd » 06 Nov 2016, 18:25

I've started landing code for it, but it's not really ready for prime time yet. For example, switching the list of decks to play against in Constructed UI currently results in all those cards being loaded synchronously - which is less efficient than loading them on start up. The right way to fix this is to not load all the cards for the decks when just showing them in a list UI.

I'll continue working to improve this in the future.
Myrd
 
Posts: 87
Joined: 24 Nov 2014, 05:58
Has thanked: 4 times
Been thanked: 31 times


Return to Developer's Corner

Who is online

Users browsing this forum: No registered users and 4 guests


Who is online

In total there are 4 users online :: 0 registered, 0 hidden and 4 guests (based on users active over the past 10 minutes)
Most users ever online was 287 on 31 Mar 2019, 04:11

Users browsing this forum: No registered users and 4 guests

Login Form