Configuration Management: Bugs and Branches

Most software when the time comes to make a release will establish a branch. As the branch is tested bugs will be found that must be fixed. A very common dilemma is do we fix it on the main branch then merge the fix to the release branch, or do we fix it in the release branch and merge it back to the main branch. Either way, the changes must be in both locations.

This is a very old debate, and developers will usually be adamant about which side they support. I have been involved in projects that have practiced both methods. I would simply like to share my experiences from both sides.

Branch to Main:
Pros

• One of the main rationals for this method is that the release is on a time line to get out. The fixes need to be put into the branch first to be able to maintain schedule and insure that the problems are fixed to support the customer.
  
• Mainline code will continue to be enhanced for future releases so it will diverge from the branch code and the fix for the main branch may not be valid for the branch, so you should fix the branch first.
  

Cons

• The biggest risk is that the fix never makes it back to the main branch so the next release you risk breaking the customer. Or even worse, an enhancement that was requested at the last minute by the customer, never makes it back to the main line.
  

Main to Branch:
Pros

• It guarantees that the main branch always has all the fixes and enhancements, so that future releases are not at risk of missing critical bug fixes or minor customer enhancements.
  

Cons

• It takes longer to update the branch with fixes, potentially causing a delay in the software release.
• Risk that the fixes from the main branch don't get merged to the release branch.
  
• The longer the branch lives the more difficult the merges to the branch become. You only want the bug fixes, not the new development.

Experiences:
My experiences have shown me that the testing involved on the release branch is much more rigorous than the testing on the main branch. No one wants the software to fail for the customer so must of the testing is focused on the release. Due to new development, the main branch is "broken" more frequently and while that is not desired, it is not entirely unexpected. It is more difficult to test reliably the main branch because the development my not be finished, and bugs would be written against code that wasn't ready to be tested. The release branch is usually much more stable with only critical defects being fixed, which makes it easier to test.

I have been involved in releases of software that went out to the customer only to have bugs reported back, that were fixed in a previous version and are broken again in the version they just received. Customers are not very understanding of this situation, especially on government contracts. They will call into question the processes you use, the reliability of your release process, even your ability to write bug free code. Some very uncomfortable meetings to have to go through.

One project I was on, the branch lived for a year and a half. The mainline code had diverged so much that bug fixes were completely different in both places. In fact we branched 3 times from the original branch without ever going back to the main. It was truly a nightmare to maintain. Which ever method you choose, the longer the branch lives, the more difficult the merges become, either direction.

There are pros and cons to both sides. From my experience, release branches should be short lived, and bug fixes should be merged in as quickly as possible after you branch. This negates a lot of the code divergence cons used as arguments for both sides. Due to the heavy testing on the release branch, it is not very likely that a required bug fix will be missed on the branch.

For the reasons stated above I am a proponent of fixing in the main branch then merging to the release branch. That said both methods can work. Going from the branch to the main requires a well defined process that everyone must follow to guarantee that changes are merged into main once they are verified on the branch.