Branch discussions

[Chris H.]

Several people have mentioned that we might want to consider creating branches in our Forge project at the SVN. Our discussions so far have taken place on various topics and I felt that it might make sense to create a topic devoted to this issue.

I myself and other people have little experience in this area. It might be fairly easy to create a branch and to then merge the changes made to the new branch into the main trunk at some point. I can not volunteer to be the team leader and to then guide everyone's efforts in this area.

We recently had a branch with just the cardsfolder for use with adding in the new DKA cards before release. This worked and did not create any problems with my being able to release daily snapshot builds. A couple of links for anyone interested:

http://www.slightlymagic.net/forum/viewtopic.php?f=52&t=5804&start=15#p78965

http://www.slightlymagic.net/forum/viewtopic.php?f=52&t=5804&start=15#p78989

[friarsol]

This is definitely something we should try to start doing sometime soon.

We can do things like this:

1. As we approach a release with all the expected features, we spawn a branch for that release.

2. No new features are added into this branch, only bugfixes. Normal development continues on the main branch. Bugfixes would be merged into the release branch (and the trunk if appropriate).

3. During this "release time", release candidates are released in some regular fashion (maybe not daily, mostly dependent on how many bugfixes are coming in). These would be the same theoretically as a nightly build.

4. Once the Beta is released, the branch should be locked from future check-ins.

[Hellfish]

I was gonna ask about branching for the Archetype recognition work (It's coming, I swear! >_< ). The thing is, this branch (as with the release branches) will cover the entire project, essentially doubling the disk space required,right? Won't this escalate quickly?Might want to talk to our gracious host about that, if so.

Correct me if I'm wrong, I'm not familiar with SVN's inner workings.

Also, a step-by-step on branching with subclipse on the wiki would be good.

[friarsol]

I was gonna ask about branching for the Archetype recognition work (It's coming, I swear! >_< ). The thing is, this branch (as with the release branches) will cover the entire project, essentially doubling the disk space required,right? Won't this escalate quickly?Might want to talk to our gracious host about that, if so.

Correct me if I'm wrong, I'm not familiar with SVN's inner workings.

In short, no.


From http://svnbook.red-bean.com/en/1.0/ch04s02.html

SVN Structure | Open

Cheap Copies

Subversion's repository has a special design. When you copy a directory, you don't need to worry about the repository growing huge—Subversion doesn't actually duplicate any data. Instead, it creates a new directory entry that points to an existing tree. If you're a Unix user, this is the same concept as a hard-link. From there, the copy is said to be “lazy”. That is, if you commit a change to one file within the copied directory, then only that file changes—the rest of the files continue to exist as links to the original files in the original directory.

This is why you'll often hear Subversion users talk about “cheap copies”. It doesn't matter how large the directory is—it takes a very tiny, constant amount of time to make a copy of it. In fact, this feature is the basis of how commits work in Subversion: each revision is a “cheap copy” of the previous revision, with a few items lazily changed within. (To read more about this, visit Subversion's website and read about the “bubble up” method in Subversion's design documents.)

Of course, these internal mechanics of copying and sharing data are hidden from the user, who simply sees copies of trees. The main point here is that copies are cheap, both in time and space. Make branches as often as you want.

[Chris H.]

Copied from message:

Re: Forge version 1.2.7

Are we still aiming for a release on the 28th (AVR Pre-release day)?

Then we should consider making Snapshot builds with AVR cards to have some more testers before the beta (merging the branch).

 
Maybe we can have a Snapshot build available on the 28th for anyone who wants to play with AVR that day. And when release shortly after mtgdata is ready for Oracle text. This way we can merge the AVR cards some point this week (maybe Wed) and give the svn folk sometime before that first Snapshot happens.

 
I wish I had seen this before Sloth did the merge. There is really no reason to merge the cards until the complete data is ready. You can make a snap shot build by switching the cardsfolder the appropriate branch before the build is kicked off. As for the SVN folk, they already have access to the cards if the know how to switch branches.

Typically for a "feature" branch like the AVR cards, you only want to merge it into the trunk once. After the merge, no further work should be done in that branch and the branch could even be deleted. Otherwise you end up in a situation like we have now where cards got checked into the branch after the merge.

Also the feature branch should be updated with changes from the trunk before a merge. This gets any updates to older cards so you don't get conflicts when you merge the branch. You would also want to do the update from the trunk right before a snap shot build with the feature branch.

[friarsol]

I wish I had seen this before Sloth did the merge. There is really no reason to merge the cards until the complete data is ready. You can make a snap shot build by switching the cardsfolder the appropriate branch before the build is kicked off. As for the SVN folk, they already have access to the cards if the know how to switch branches.

Typically for a "feature" branch like the AVR cards, you only want to merge it into the trunk once. After the merge, no further work should be done in that branch and the branch could even be deleted. Otherwise you end up in a situation like we have now where cards got checked into the branch after the merge.

Also the feature branch should be updated with changes from the trunk before a merge. This gets any updates to older cards so you don't get conflicts when you merge the branch. You would also want to do the update from the trunk right before a snap shot build with the feature branch.

We should definitely build a more strict timeline of when everything is occurring now that we're using Branches for new sets. I didn't have an issue with merging the branch on Wednesday as magiccards.info usually updates on Thursday allowing for any Set related code additions to go in, and then the SetInfoScript to be run once it could grab the right data.

I definitely wasn't expecting a Snapshot build to happen until all of that was performed (and really I wasn't expecting it until Saturday or maybe late Friday). It would probably simplify things long-term to not merge until we can do all of the above things, as we could just handle it all in a row instead of having to step gingerly for a few days near Set Releases.

[ArsenalNut]

We should definitely build a more strict timeline of when everything is occurring now that we're using Branches for new sets. I didn't have an issue with merging the branch on Wednesday as magiccards.info usually updates on Thursday allowing for any Set related code additions to go in, and then the SetInfoScript to be run once it could grab the right data.

I definitely wasn't expecting a Snapshot build to happen until all of that was performed (and really I wasn't expecting it until Saturday or maybe late Friday). It would probably simplify things long-term to not merge until we can do all of the above things, as we could just handle it all in a row instead of having to step gingerly for a few days near Set Releases.

I definitely agree about having a more strict timeline. I would suggest waiting till both magiccards.info is updated and the picture URLs are valid before merging back into the main trunk. I think this would help minimize potential problems. I haven't tested to see what happens when you try to download card images when the URL is not valid. I have also had the deck editor crash on me a couple time when working on the AVR cards. It looked like it was having issues with sorts when there wasn't valid set info. I'll try to reproduce the crash and post to the bug report message.

[ArsenalNut]

Here's a quote from Apache's Subversion Best Practices webpage from the Know when to create branches section

The Branch-When-Needed system

(This is the system used by the Subversion project.)

Users commit their day-to-day work on /trunk.
Rule #1: /trunk must compile and pass regression tests at all times. Committers who violate this rule are publically humiliated.
Rule #2: a single commit (changeset) must not be so large so as to discourage peer-review.
Rule #3: if rules #1 and #2 come into conflict (i.e. it's impossible to make a series of small commits without disrupting the trunk), then the user should create a branch and commit a series of smaller changesets there. This allows peer-review without disrupting the stability of /trunk.

Pros: /trunk is guaranteed to be stable at all times. The hassle of branching/merging is somewhat rare.

Cons: Adds a bit of burden to users' daily work: they must compile and test before every commit.

I think this would the best branching model for the Forge project to adopt.

[Chris H.]

Here's a quote from Apache's Subversion Best Practices webpage from the Know when to create branches section

The Branch-When-Needed system

(This is the system used by the Subversion project.)

Users commit their day-to-day work on /trunk.
Rule #1: /trunk must compile and pass regression tests at all times. Committers who violate this rule are publically humiliated.
Rule #2: a single commit (changeset) must not be so large so as to discourage peer-review.
Rule #3: if rules #1 and #2 come into conflict (i.e. it's impossible to make a series of small commits without disrupting the trunk), then the user should create a branch and commit a series of smaller changesets there. This allows peer-review without disrupting the stability of /trunk.

Pros: /trunk is guaranteed to be stable at all times. The hassle of branching/merging is somewhat rare.

Cons: Adds a bit of burden to users' daily work: they must compile and test before every commit.

I think this would the best branching model for the Forge project to adopt.

 
Looks good to me, but rule #1 may be somewhat extreme.

In the past there have been several occasions where someone meant to submit all of the files that they had changed/created but...

sometimes we accidentally leave out one of the pieces of our submission. This can result in a project that fails to build and run.

When this happens the team has been capable of noticing that there is a problem and we tend to work together as we try to rectify this situation.

Publicly humiliating the responsible person could make a bad situation worse by convincing them to leave the project. We should consider rewording rule #1.

[ArsenalNut]

Publicly humiliating the responsible person could make a bad situation worse by convincing them to leave the project. We should consider rewording rule #1.

How about we just add a after the sentence? I didn't take the statement literally. I assumed it was the author trying to be funny sort of like "Management will continue the beatings until morale improves". I think the intent is to remind people to be careful not to break the trunk since the majority of the work is done there.

[Chris H.]

Publicly humiliating the responsible person could make a bad situation worse by convincing them to leave the project. We should consider rewording rule #1.

How about we just add a after the sentence? I didn't take the statement literally. I assumed it was the author trying to be funny sort of like "Management will continue the beatings until morale improves". I think the intent is to remind people to be careful not to break the trunk since the majority of the work is done there.

 
Yeah, I suspected that this was just a Dilbert like joke.

Your point that we do not want to break the trunk is a good one.

[Chris H.]

How about we just add a after the sentence?

 
I have a first draft for a re-write for rule #1.

Rule #1: /trunk must compile and pass regression tests at all times. We have had several commits to the SVN trunk in the past which broke the trunk and it would not successfully compile and run.

Your local copy may compile and run but a commit to the SVN trunk may fail to push all of your new material to the SVN. This will leave the truck in a state where some of your new material is missing and this can break the trunk.

We also have had several instances where a commit will compile and run on some computers but not others. Your code may not be compatible with some versions of java and code that runs correctly on the Windows OS may sometimes fail to work correctly when run under the Mac OS.

[ArsenalNut]

I added a step-by-step for updating a branch from the trunk to the Wiki. Someone suggested that the branch information doesn't really belong under the installation section. I think it makes sense to move this information under "Working with the project". The new section organization could be

3.X Branches

3.X.1 Forge Branching Model
3.X.2 Creating Branches
3.X.3 Switching Between Branches
3.X.4 Updating a Branch
3.X.5 Reintegrating Branch into the Trunk

I still need to write the step-by-step for branch reintegration.

[Chris H.]

We have a section named "3.5 Committing changes to the repository". There is a brief warning to not commit code with errors.

The new branches section appears to be similar in purpose and could possibly become a new 3.7 section.