Why? Reducing waste. Making game state smaller.

[Braids]

"Why?"
first, i think i owe an explanation for my recent meddlings with the svn trunk, despite the fact that most of my work (on minimax) is supposed to be on a branch. summary: i have been working on the trunk to make the braids-minimax branch easier to merge.

• to keep the braids-minimax branch up to date, i have to merge the changes from the trunk to the branch. merging in svn has been nightmarish. i sometimes get strange error messages when i run svn merge. for instance, it's common for svn merge to fail in the res folder because of an error returned from a PROPFIND request. to work around this, merging becomes a multiple step process. first i merge with `--depth immediates' on the parent and then with `--depth infinity' on the subdirectories. if a subdirectory fails, i apply these two steps recursively to it and its subdirectories.
• resolving conflicts from merging is time consuming and somewhat unnerving. i am sometimes unsure of whether i am semantically breaking something. after my prior merge from r9969, i had reverted some of my cosmetic changes to reduce the number of conflicts. i thought the next merge would go much more smoothly. well, it didn't. it resulted in 30 conflicts in source code that i must resolve manually.
• merging is often a long process, and i find myself with extra time while i babysit it, so my natural tendency is to work on the trunk along with everyone else.

in the near future, i may be asking to merge some of my smaller changes currently on the braids-minimax branch toward the trunk. i'll try to describe them in a post first. i can also post an svn diff before committing the changes, if people want to review them.

but before that there are a few things i'd like to change about how forge uses heap memory.

"Reducing waste."

for starters, i know i can make forge faster and more memory efficient by having CardFactory.getCards (and perhaps CardFactory.getAllCards) return an immutable Iterable instead of a list. if i am able to convert both, we will no longer need the allCards field in CardFactory; instead, those two methods will safely iterate over the values in CardFactory.map. (i can wrap the map's value iterator with an immutable iterator that does not support the remove method.) that list has several thousand elements, and i'm pretty sure i can just make it disappear.

this affects a lot of files, especially the ones that i foretold in r10084.

"Making game state smaller."
second, for minimax, we will need to make the game state more memory efficient. one way to do this is to use trees {TreeMap, TreeSet} instead of hashes {HashMap, Hashtable, HashSet} in fields that comprise the game state. hashes, while potentially very fast, usually have large amounts of null pointers inside them, depending on the load factor. this is overhead. each pointer is using up to 64 bits. why trees? they are more memory efficient. trees' get and put methods take a little longer than a hash , but they only have null pointers at their leaves. java keeps its trees balanced, and a balanced tree has at most (2 * ceiling(ln(n))) (2 * ceiling(log2(n))) null pointers, i think. n is the number of items in the tree. if my math is wrong, please correct me.

from looking at the code, this also means i may need to change some methods' signatures to use Map<X,Y> instead of Hashtable or HashMap. replacing these specific class types with interface types decouples them from implementation, which is a good design practice.

Edit: if the hash's keys have evenly distributed hash codes. in the worst case, all of a hash's contents go into the same bucket, so their worst case time for a get or put is proportional to n, the number of keys. but a balanced tree's worst case time is always proportional to ceiling(log2(n)). so, trees have a second advantage in unusual situations.

[Snacko]

There are many places where toArray() is used and all those collections are being passed as such, it would reduce memory footprint to just pass the collection, however this means lots of refactoring.

For merging you might want to look at git svn, or we could switch the project from svn to mercurial (supported via googlecode) as it's so much better for branching, merging and many more useful operations.

[Braids]

There are many places where toArray() is used and all those collections are being passed as such, it would reduce memory footprint to just pass the collection, however this means lots of refactoring.

i'm not ready to tackle those just yet, but it sounds absolutely tempting.

For merging you might want to look at git svn, or we could switch the project from svn to mercurial (supported via googlecode) as it's so much better for branching, merging and many more useful operations.

it's interesting you should mention this now, because i am ready to throw in the towel with the minimax branch. it has gone too far. by that i mean it has changed too many lines of code in existing files. i have been spending all of my time on that branch merging and no time improving minimax since its creation. :frustrated:

here is one example: recently there were a lot of good indentation changes and javadoc additions to the trunk. while i understand and support their utility, trying to merge them has crushed my spirit and has mortally wounded the branch.

unless the tools you mentioned have a much smarter diff3 algorithm, i doubt they would be of much use when it comes to conflicts. i can get around the PROPFIND errors in svn ok. i cannot script or automate my way out of merge conflicts. :choke:

at this point, i am thinking of requesting that the braids-minimax branch be retired from active status. i want to leave it there, just not use it directly. i could use it as reference material. slowly add minimax code to the trunk. i could start with code that is vestigal at first. not called by anything except maybe some TestNG tests. that reduces the likelihood that the new code would break builds. then slowly weave more and more into production code in a steady manner, keeping everyone apprised of my activities, and accepting constructive criticism.

i've already proven that i can rollback a bad change, so i think this is safe as long as i pay attention to Chris H.'s requests for reduced activity before betas.

i would really appreciate more comments on this matter. thank you.

[Sloth]

I think this is the better way. As long as you inform us before big changes and help us keeping up with your speed, I don't mind you messing with the main branch Braids.

In fact I wanted to propose you to do exactly this before, seeing you struggle with both branches at the same time. I'm not really the one to speak though, not being one of the real architects of forge's structure, I only fill the spaces and build the decorations on top.

[jeffwadsworth]

Sometimes Rob is right...

viewtopic.php?f=52&t=4885&p=63169#p63169

[Chris H.]

Just to give people a heads up, if possible, I would like to release the next beta on this upcoming Tuesday. This gives everyone a couple of days to add in a few finishing touches.

I am still getting used to the new Maven system and there may be a few wrinkles to iron out but I think that it is fairly stable at this time. We should be careful about adding additional dependancies, I guess that we should give everyone a few days notice so that we will be better prepared for any issues that come up.

Braids, no one wants to see you burn out and loose interest. Your idea of moving small pieces of the other branch into the main branch should help to reduce the frustration.

And we all have various war stories about our experiences with googlecode's SVN. Some people had to jump through a number of hoops just to get their system set up to the point where they had any hope of participating.

I was considering having a topic devoted to these war stories but was afraid that it might scare some people off. At times we all feel like the walking wounded.

[Snacko]

@Braids
I ran some sample merges with your branch and as expected mercurial and git handle all the formatting just fine and there are very little to no conflicts.

Would be nice to switch from svn to mercurial for the repo. For eclipse it's just a switch subclipse > mercurial eclipse. To enable mercurial for our project either Rob or Dennis would have to switch the settings at google code.

http://code.google.com/p/support/wiki/ConvertingSvnToHg

[Braids]

Sometimes Rob is right...

viewtopic.php?f=52&t=4885&p=63169#p63169

i . . . don't understand the relevance. can you explain?

[Chris H.]

I see that there is a version of mercurial for Mac OS.

[Braids]

@Braids
I ran some sample merges with your branch and as expected mercurial and git handle all the formatting just fine and there are very little to no conflicts.

that is impressive. i would expect lots of issues with certain files, especially where i rewrote most of proliferate as a prototype.

Would be nice to switch from svn to mercurial for the repo. For eclipse it's just a switch subclipse > mercurial eclipse. To enable mercurial for our project either Rob or Dennis would have to switch the settings at google code.

. . . i was just getting used to svn and all its quirks.

Edit 1: i chose proliferate because it was interesting. it posed a number of choices that the AI could make, yet these choices were controllable by setting up which permanents and players had counters on them before the proliferation.

[slapshot5]

I see that there is a version of mercurial for Mac OS.

good to know.

[slapshot5]

There are many places where toArray() is used and all those collections are being passed as such, it would reduce memory footprint to just pass the collection, however this means lots of refactoring.

i'm not ready to tackle those just yet, but it sounds absolutely tempting.

I tried to do this once, and broke everything on a local branch. Definitely worthwhile though.

-slapshot5

[Rob Cashwalker]

I don't think Mercurial Eclipse is a good idea. You have to signup with JavaForge.com just to download it.

When I worked on the draft mode, I went weeks without updating my local code, until I had everything working. Then I went ahead with an update, fixed the major differences and then posted my update.

Dennis is working on his Shandalar quest mode in much the same way, but he keeps his local copy fairly up to date. I don't think he's doing nearly as much architectural modifications, so maybe it doesn't apply.

[Braids]

I don't think Mercurial Eclipse is a good idea. You have to signup with JavaForge.com just to download it.

i'll have to side with Rob on this one. i can cope with svn for a while yet.

When I worked on the draft mode, I went weeks without updating my local code, until I had everything working. Then I went ahead with an update, fixed the major differences and then posted my update.

Dennis is working on his Shandalar quest mode in much the same way, but he keeps his local copy fairly up to date. I don't think he's doing nearly as much architectural modifications, so maybe it doesn't apply.

perhaps that is the answer. . . to focus on adding code, and instead of changing existing code, making a note to mark it deprecated in the future.

does anyone have other ideas or agreements?

[friarsol]

I've done my fair share of large changes since I started on the team a year or so ago. I've found it easiest to explain exactly where a huge change is coming and exactly what it's goal is. This is important, make sure you give people a day or two (since not everyone can update everyday) to check in fixes so you can merge things with your code, since you know what you changed and why. As an added bonus, someone has been in that code fairly recently and can give some pointers about why something works a certain way, or give warning for some consideration that might not be obvious.

Make sure your passion doesn't get you too wrapped in getting the change committed, and think that 20+ people play in the source code, and hundreds play with the released product. This past cycle has been the most unstable I've seen the codebase since probably when I committed the changes for the Phase structure (in November or so).

Definitely as Rob suggested on a different thread to split up larger changes if you can. Especially for cleanup related things that affect a lot of cards, it's easier to see where a potential problem occurred and get an appropriate fix in when related changes go in separately.

There has been plenty of contractions in Forge in the last 6 months (The original coder of Wagic wrote a great blog a few months back about it http://wololo.net/wagic/2011/04/25/freewar/ ) and most come from building a new structure and then converting cards to use it.

From your enthusiasm I can tell you want to fix a lot, but think about things like this:

• When I first started on the project there was a single cards file for all of the cards. People had to merge each change into a single file. You wouldn't believe how many merging issues there were until we finally split the single file into individual files per card.
  
• Any time you wanted to use a "custom" (read: Non-Mana) cost you had to write it yourself or copy/paste it from somewhere. Cost.java didn't exist. It's not perfect in how it works. But it works pretty damn well.
  
• Ditto for Targeting.
  
• The AbilityFactory/Trigger system is along the same lines. There was already this idea of Factories in CardFactory. But we broke off to try to create a much more legible system where the Script would actually mean something.