What about a new start ?

[digitalcollector2]

Reposted from a wrong forum. This is more appropriate

I was wondering ...
Why Java ?

Yes, it runs independently on many platforms. But, it's slow. Also can be buggy with memory issues. And slow to develop a really big project.

I want to propose building Forge using Python.

Yes, it can be slow as Java, no, it won't be platform independent (but can easily be adjusted). But it will be robust, easy and faster to develop and won't have memory quirks. The implementation doesn't seem to be dependant to the latest version of Python (mentioned because of the latest necessary Java 8 installation for the Forge 1.6.0 to ran). People won't have to download Python, slightlymagic could provide the game with its own special distro.
Porting to Python should be easy since the whole organization/data structure/encoding is already built; it will just need to be conformed to a new coding style.

I understand that most of the developers on this project may not be familiar with Python, so may I recommend Jython ?

For some reason the online game features are still ornamental. I don't know if this is a language dependant issue, if it is, I think Python will resolve it. And Forge is getting a bit slower to launce with every new version, isn't it a bit much for a 2D simple interfaced application to occupy 450MB in memory ?

The Android version will have to continue with Java.

I am not a professional programmer.

[Xitax]

Transferring the code base? At some point I guess it would be more trouble than benefit since you potentially lose so much.

As for getting slower each launch - this is due to parsing deck files every time (as far as I know). The more decks I have the slower it's gotten over time - I have over 10K decks now. But it's possible also that something else is causing it. Anyway, I'm not a programmer either, but it seems that it should be possible to only parse the new files rather than the whole thing. But I'd be interested in what the developers have to say.

[KrazyTheFox]

Java itself is not slow. In fact, Java is very, very fast. The thing is: you need to know how to program Java for performance. In fact, Python can't just be as slow as Java—it is actually quite a bit slower than Java. Python is good for ease of use, but is less than ideal if you need good performance. If you go the Jython route, then you're getting close to the same performance as Java... but why not just use Java then? At that point you're adding an additional layer of complexity for no gain.

Porting something this large over to another language is never "easy". Each language has its own semantics, caveats, "gotchas" and every language has its own style. You shouldn't try to copy code structure directly from Java to Python—each language has strengths and weaknesses that you should code around/for.

The online features not working is not a Java issue. It's that the rules engine (what actually runs the Magic games) was never designed to be run in a networked environment. When you don't design for this from the start, it becomes extremely difficult to add it in after the fact. The developer that was working on this hasn't been around in a couple of years and that project has stalled until someone picks it up again.

Forge is an old application, going on over 10 years. With developers coming and going with varying levels of skill, the application will generally increase in memory footprint over time. The fact that it runs as well as it does on some fairly old hardware is nothing short of impressive. Are there things that could be done better? Absolutely. Those things generally require a large reworking of the existing code or starting from scratch.

Separating the project into two languages, one for Android and one for everything else, will harm the project. Right now, with the shared code base, bug fixes and new features are generally available in both versions with very minimal (often no) effort. Splitting it up would double the cost of new features and bug fixes (and add cost where there was none previously).

I think overall we have *much* more to lose by switching to something like Python than if we don't. One thing we could look at is using a different JVM language that is completely Android compatible. However, I'm not sure how well this would go over with all the developers. We don't want to alienate anyone with the changes we make.

[digitalcollector2]

Where can I find Forges' general data structures and organization map or something relevant ? Plain simple natural language or little use of Java where an overall look is documented (including card structures) so a new developer can start working on it without actually reading the code from begining to end. Thinking of making my own version using Python and see for my self how quickly it can scale up.

[KrazyTheFox]

We... don't have anything like that. You'll find that pretty much doesn't exist for any project, let alone open source ones. Nobody really likes doing documentation, so the solution is to try to keep the code clean and easy to read. I don't think there's any straightforward way to describe something as complex as a card game rules engine. There's so many pieces that are all intertwined in very specific and unique ways. Even a rules engine with the most beautiful code in the world would be challenging to describe at best, especially if you're new to software development.

What you want to do is a truly enormous undertaking.

[Seravy]

While I know nothing about Phython, considering the size and number of Forge sources, you might be faster if you write new software that can convert java sources into Phyton instead doing it manually. In fact I wouldn't consider converting anything over a few thousand lines without such a tool, from any language to any other.

Speed, well, the startup times are horrible...having to browse through some 29 thousand individual image files takes a while (and I suspect is not very HDD friendly). Decks I have no problems with but I imagine having 10k of them will do the same.
I do notice lag and heavy swapping at times though - even when I'm still actively playing the game, not just when changing to another task and returning. Usually takes anywhere from 3-10 seconds of complete lack of response from the game.

[digitalcollector2]

I always thought that building something from the beginning whould invoke a feeling of satisfaction knowing that now you are going to make it better, faster, using less code by implementing more optimized and less complicated structures. 50% of the code might actualy stay the same. As KrazyTheFox pointed out, the online feature was not in the original plans that's why it's so difficult to implement now.

[Marek14]

I always thought that building something from the beginning whould invoke a feeling of satisfaction knowing that now you are going to make it better, faster, using less code by implementing more optimized and less complicated structures.

Do you have data to corroborate that hypothesis?

[Fizanko]

Speed, well, the startup times are horrible...having to browse through some 29 thousand individual image files takes a while (and I suspect is not very HDD friendly). Decks I have no problems with but I imagine having 10k of them will do the same.

Well, as an user i find this a bit concerning.

I have some questions regarding the amount of files to parse and browse the program is doing.

As there are a lot of MTG sets i don't play and am not interested to play, so i've been thinking in doing a "lighter" Forge for my use (and harddrive lifespan i guess), but i wonder if in the end it would work or if the time spent doing it would fill the logs with errors or break the whole program if i removed :

- the corresponding edition files from the edition folder
- the entries related to those sets in the blockdata files
- the corresponding cards from the cardsfolder.zip from the cardsfolder .. folder
- the corresponding decks contained within the quest worlds opponents or the precontructed ones
- the corresponding cards pictures

Would Forge still work or would it still require the presence of the removed files ?

[digitalcollector2]

I always thought that building something from the beginning whould invoke a feeling of satisfaction knowing that now you are going to make it better, faster, using less code by implementing more optimized and less complicated structures.

Do you have data to corroborate that hypothesis?

I've felt like that many times before. I guess i'm a perfectionist (in a good way). When i say start from the beginning I don't mean literary from scratch ... but like building something better from the rubbles of the old one. I hope some day soon I acquire the adequate knowledge to rebuild Forge with Python (the GUI mostly).

[Agetian]

Speed, well, the startup times are horrible...having to browse through some 29 thousand individual image files takes a while (and I suspect is not very HDD friendly). Decks I have no problems with but I imagine having 10k of them will do the same.

Well, as an user i find this a bit concerning.
I have some questions regarding the amount of files to parse and browse the program is doing.

As there are a lot of MTG sets i don't play and am not interested to play, so i've been thinking in doing a "lighter" Forge for my use (and harddrive lifespan i guess), but i wonder if in the end it would work or if the time spent doing it would fill the logs with errors or break the whole program if i removed :

I assume Forge would still work if you did that, but all things considered, I'm fairly sure that your time and energy is better spent elsewhere. Yes, Forge does load a significantly big amount of files, but it's not really that bigger compared to the amount of files located in the assets of pretty much any modern big enough videogame (it's just that most videogames pack those assets inside bigger archives so you're not necessarily seeing them all at the same time; we're not doing it for pictures and for decks; also, many videogames don't load all of them at the same time, but they still load a good amount of data at startup, at least comparable and usually much bigger in size compared to what Forge is trying to load, and complex games still require lots of assets to be loaded for each of their maps and whatnot).

Also, Forge doesn't really move those files anywhere, so it doesn't increase fragmentation of the hard drive just by loading those assets every time. It also doesn't fully load all the pictures into memory every time you start the game, only all the card scripts (and it processes the decks), so pictures are not really a concern. Because your decks are sort of up to you (Forge doesn't package any Constructed decks other than quest opponents and stuff, which it doesn't load immediately upon startup as far as I know), and scripts are loaded very quickly on any modern enough rig (and not really moved anywhere on the HDD), the potential harm from such an operation is really blown out of proportion.

To cut it short, yes, your HDD is working actively while Forge is loading, but I wouldn't call it catastrophically HDD unfriendly in some odd way compared to how other games are doing it (and how they are loading their thousands of textures, sounds, maps, entity definitions, whatever, while you're playing the game). Unless you're paranoid enough to start cutting your other videogames down to size because they're doing it as well, there is really no need to start doing it with Forge either. So, basically, I don't think you should waste your time on such a thing, but that's just my opinion.

Case in point: I have not only played Forge every day, but also developed Forge and sorted hundreds of thousands of decks from MTGDecks.net at least several dozen times, checked out the Forge SVN repository multiple times, etc., thus performing hundreds, if not thousands, of move operations on millions of Forge-related files as a developer. All of it was done on the same hard drive, at least in the last four or five years. My HDD is still alive and well, and it doesn't really feel like it got damaged because of Forge or anything. ^^ This is a *much* heavier HDD workload than any ordinary (non-dev) Forge user would ever experience out of Forge alone.

P.S. If you still decide to proceed with your idea, please understand that we as developers can't provide any support for such a customized installation, which will definitely be difficult for you to maintain when it comes to updating it to newer versions of Forge, and will have the potential of producing all sorts of crashes and glitches that the stock version of Forge does not have (after each merge iteration).

- Agetian

[Agetian]

I always thought that building something from the beginning whould invoke a feeling of satisfaction knowing that now you are going to make it better, faster, using less code by implementing more optimized and less complicated structures.

Do you have data to corroborate that hypothesis?

I've felt like that many times before. I guess i'm a perfectionist (in a good way). When i say start from the beginning I don't mean literary from scratch ... but like building something better from the rubbles of the old one. I hope some day soon I acquire the adequate knowledge to rebuild Forge with Python (the GUI mostly).

You are indeed welcome to try if you'd like. As a rather long-time Forge developer, I witnessed at least a couple "let's rewrite Forge from scratch because <insert your reason here>" effort announcements before. Rather serious announcements, mind you, sometimes coming from people with good programming experience. Guess what, those people never came back and Forge still hasn't been rewritten from scratch because it's a massive, tremendous undertaking even for a big team of people, let alone as a solitary effort. But that being said, not going to stop you from trying. Please do let us know if anything ever comes out of your ambitious plans.

By the way, there are some plans to eventually attempt to decouple the client-side and the server-side code completely, thus making Forge a fully client-server architecture-based application. If that works out, your idea of writing a custom GUI (in Python or in basically any other language of your choice) would be much easier than it would be right now. Networking will also become possible then (despite the fact that we'll still use Java ). No promises on when it'll be done though or even if it'll come to life (once again, this is easier said than done).

- Agetian

[Marek14]

I always thought that building something from the beginning whould invoke a feeling of satisfaction knowing that now you are going to make it better, faster, using less code by implementing more optimized and less complicated structures.

Do you have data to corroborate that hypothesis?

I've felt like that many times before. I guess i'm a perfectionist (in a good way). When i say start from the beginning I don't mean literary from scratch ... but like building something better from the rubbles of the old one. I hope some day soon I acquire the adequate knowledge to rebuild Forge with Python (the GUI mostly).

If I read this right, you assumed that your feelings are the default shared by the rest of humans. I remember when I used to think that, but, sadly, it's a common mistake.

I think that for humans in general, it's more interesting and more exciting to do something new than to retread what has already been done. I'm not saying that this is a good thing -- I'm only saying that this is how it looks to me. Modifying is easier than redoing.

[Fizanko]

[...]

Thanks for the explanations about how it works.
I'm still interested by making my "light" version of Forge (especially a Ravnica-only one as it has always been my prefered block to play) and see how it can work, along my main Forge.

Good to know that it's possible in theory, i was wondering if out of the obvious files (cards, editions, quest opponents, decks) there was some hardcoded stuff that would throw errors if i removed non-Ravnica stuff from a Forge install.