Working on loading cards lazily

[Myrd]

Hey all,

One of the things that annoys me the most about Forge is that it takes forever to load (e.g. several minutes) - which is especially painful on mobile when I just want to get a quick game or two in while commuting.

So I started working on fixing this. The idea is that instead of loading all the cards at startup, we only load cards as they're needed. For example, when you select a deck to play, it will load the cards needed by that deck only - and the opponents deck.

Some screens will probably still require loading the full DB - such as the deck construction UI that lets you search all cards.

So far, I have a hacky prototype that seems to mostly work. I load no cards at startup (making it blazing fast), but when a deck is loaded (currently, when entering the deck manager for example - but can probably be done even lazier than that), it will attempt to load all the cards in it.

When a card is not found in the DB, it tries to compute the filename for that card and load it from there - adding it to the relevant editions.

So far, seems promising. I'm able to load most cards lazily, though one thing that I ran into is that double-faced cards have both names in the filename - but decks only have the front face - so not sure how to best get the filename for those - maybe just prefix matching. I also haven't figured out how this interacts with e.g. random deck generation or selection by the AI - whether those need access to all cards or not.

Anyway, this is just a heads up that I'm working on this so that there's no effort duplication if someone else was about to try the same thing.

[Fizanko]

This sounds very interesting, good luck.

[Hanmac]

hm you talk about the card files, not the images or something like that?

yeah i think it might be able to make the read of them more lazy but i don't think thats what make the startup of Forge that slow. (didn't have mobile version)

imo loading the Card scripts should not slow them down.
(thats only reading the text files, it isn't even creating the cards?)
i think Forge does create "One" card for each parsed card?

for your lazy loading, you might do it in different steps.

• like when the DeckView is opened (not the Deck Editor)
  it only needs to load the Cards for this given Deck if they are not loaded yet.
• using DeckEditor all card files need to be loaded, and i don't think that can be changed.
• making a Duell:
  
  • against AI with a fixed Deck: only need to load this one.
  • against aI with Random Deck: need to load all cards.

and about the DFC, we can change the file names to only have the name of the Front Face. The file name is only for the developers and not used anywhere.

PS: i think if you implement that, make it configurable, like it might be disabled on the desktop-version as default.

[Myrd]

hm you talk about the card files, not the images or something like that?

yeah i think it might be able to make the read of them more lazy but i don't think thats what make the startup of Forge that slow. (didn't have mobile version)

imo loading the Card scripts should not slow them down.
(thats only reading the text files, it isn't even creating the cards?)
i think Forge does create "One" card for each parsed card?

for your lazy loading, you might do it in different steps.

• like when the DeckView is opened (not the Deck Editor)
  it only needs to load the Cards for this given Deck if they are not loaded yet.
• using DeckEditor all card files need to be loaded, and i don't think that can be changed.
• making a Duell:
  
  • against AI with a fixed Deck: only need to load this one.
  • against aI with Random Deck: need to load all cards.

and about the DFC, we can change the file names to only have the name of the Front Face. The file name is only for the developers and not used anywhere.

PS: i think if you implement that, make it configurable, like it might be disabled on the desktop-version as default.

Making it configurable to start off sounds good. This way, I can start implementing it and we leave it off by default until all the kinks are worked out.

I'll have to get more exact numbers later, but from my experience (from profiling it on desktop) most of the startup time is spent script loading - it's like 20k files to read and parse.

For now, I've worked around the DFC problem via prefix matching but I agree that it might make sense to just change their naming convention in the future. (i.e. once I land the new loading method and we agree to make sense to make it the default - since at that point changing the DFC naming will be a pure speed improvement since it will avoid needing to do prefix matching)

[Hanmac]

@Mryd:
its cool that you begin, but if i try to get the Preconstructed Decks i get a StackError.

Code: Select all

at forge.deck.Deck.getMain(Deck.java:114)
at forge.deck.Deck.getAllCardsInASinglePool(Deck.java:283)
at forge.deck.Deck.getAllCardsInASinglePool(Deck.java:279)
at forge.deck.Deck.convertByXitaxMethod(Deck.java:221)
at forge.deck.Deck.loadDeferredSections(Deck.java:215)
at forge.deck.Deck.getMain(Deck.java:114)


your loadDeferredSections function might cause this.

[Myrd]

@Mryd:
its cool that you begin, but if i try to get the Preconstructed Decks i get a StackError.

Code: Select all

at forge.deck.Deck.getMain(Deck.java:114)
at forge.deck.Deck.getAllCardsInASinglePool(Deck.java:283)
at forge.deck.Deck.getAllCardsInASinglePool(Deck.java:279)
at forge.deck.Deck.convertByXitaxMethod(Deck.java:221)
at forge.deck.Deck.loadDeferredSections(Deck.java:215)
at forge.deck.Deck.getMain(Deck.java:114)


your loadDeferredSections function might cause this.

Thanks for the heads up. Working on a fix.

EDIT: Fixed.

[Myrd]

I've started landing code for it, but it's not really ready for prime time yet. For example, switching the list of decks to play against in Constructed UI currently results in all those cards being loaded synchronously - which is less efficient than loading them on start up. The right way to fix this is to not load all the cards for the decks when just showing them in a list UI.

I'll continue working to improve this in the future.