To succeed with distributed rapid development, a branch-merge strategy is key. A good strategy facilitates processes among multiple developers or teams and is the basis for any well-functioning DevOps pipeline that uses continuous integration (CI).
While there are many ways to implement CI and DevOps without a branch-merge strategy, most mature organizations use them to shift code defects left and reduce the negative impacts of a defect for those working on the particular change.
Let’s explore branching strategies, merging strategies, and how you can put them together in a way that’s right for your team in order to bring quality features to production faster.
Branching Strategies
Irrespective of your version control tool, a good branching strategy helps teams of developers to easily collaborate, while not allowing disruptive or code-breaking changes by one developer to impact other code. Changes to the branch don’t affect other developers until the developer or team has tested the changes and decides to merge the code. Developers can still pull down changes from other developers to collaborate on features and ensure their private branch doesn’t diverge too far from the main code line.
Branching models may differ between organizations, but there are four strategies that are most commonly implemented. Choosing the right strategy is paramount to a successful implementation.
Trunk-based Development (No Branching)
Trunk-based development means all developers work on the same branch, and when changes are tested and ready, a developer pushes their code to the central repository. Small organizations or those with strong internal testing practices find this strategy useful because it reduces complexity and encourages the development organization to swarm on the problem.
If your program is interested in utilizing feature toggles in your implementation, this strategy could be very effective. Unfortunately, it does come with risks. Large multi-team development can struggle with this strategy, as one defect can halt all forward progress until the trunk is fixed, causing unnecessary churn or delays—which are compounded if your compnay culture exhibits a “blame and deflect” tendency to defect resolution.
Release Branching
Release branching refers to the idea that a release is contained within a branch. When a team starts working on a new release, a branch is created (e.g., “1.1 development branch” or “Release 2.1”), and all work done until the next release is stored in this branch.
This strategy is most commonly used in waterfall and Scrummerfall development processes. Release branching can be unwieldy and hard to manage if you have many people working on the same branch. Unless you have very small release cycles of less than two weeks, release branches nearly always ensure late cycle development, release delays, long testing cycles, and challenges integrating multiple features.
Feature Branching
Feature branches, which are often coupled with feature flags or toggles that enable and disable a feature within the product, are used to collect a series of user stories that can be merged into a master and deployed as one complete feature. This makes it easier to move toward a continuous delivery process, and, if used in conjunction with toggles, it’s also easy to decide when to expose end-users to new features.
This type of approach reduces the time of delivery and testing cycles. A mature software development lifecycle is required to implement feature branching due to the need for small, rapid releases, so to use this strategy effectively, your organization must have minimal viable feature sets. Without the discipline and experience of small releases, the tendency will be to build large, complex features, similar to release branching.
Story or Task Branching
Story or task branching directly connects a user story to changes in source code. It’s the lowest level of branching, and each issue implemented has its own branch, which is typically associated with some user story ID.
Because agile centers around user stories, this type of model is ideal for organizations with a mature agile development process that clearly breaks down stories into small, releasable sets of functionality. This gives a release manager complete flexibility relating to what stories can go with what release, puts no restrictions on how frequently releases can go to production, and limits exposure of defects as far left in the development process as feasible.
Merging Strategies
Branches are intended to be short-lived, making them easy to merge. The ability to frequently (and automatically) merge code is critical to avoiding long, costly merge conflicts. There are three common strategies to merging code from your branches.
Manual Code Review and Merge
The simplest of merging strategies is each branch being manually code reviewed and tested prior to being merged into the master branch. This may work as an initial stage to building out your DevOps capabilities, but it can be plagued with human error and delays. Organizations that adopt this strategy liekly will find that far more defects make their way to the master on average than the other two strategies, but it will still be far less than no branch-merge strategy at all.
Minimal Continuous Integration
Used most commonly in small development projects, this strategy involves using a build orchestration tool, like Jenkins, to compile and test the source code. Often this includes implementing a series of quality gates, or mechanisms that require no human intervention to quantitatively enforce quality, to prevent code that does not pass your tests from being merged into master. Code that passes all quality gates is automatically merged to the master branch.
If your organization has complicated or long-running test suites, this strategy may not be for you. Large feedback cycles can cause bottlenecks and introduce a higher frequency of merge conflicts.
Continuous Integration Pipeline with Quality Gates
This strategy leverages integration branches—or branches that map to stages in your DevOps pipeline—quality gates, and automated merges from your build orchestration tool to ensure bugs and defects are easily identified at particular stages in your pipeline and don’t get merged into the master.
This strategy is helpful for organizations that use a variety of testing types to ensure quality, such as unit tests, functional tests, security tests, regression tests, and load or performance tests. The simplest and easiest to run are incorporated into early stages, while more complex and time-consuming tests are reserved for quality gates (and branches later in your pipeline). This has the benefit of weeding out easy-to-find issues early, making the application of manual testing more efficient and effective.
In practice, we may use branches for development, test, and staging, corresponding to pipeline stages. As code passes the required tests and moves to the next stage in the pipeline, a corresponding merge is made to each branch. This allows us to identify race conditions in integration between multiple features and isolate them before they are found in production. Releases may stop at production, but development can continue until a resolution is made to the defect stage or branch.
Putting It All Together
While there is not one approach that will effectively apply to every organization, the leading best practice is to utilize story branching with automated merges from your quality gates within your continuous integration pipeline.
The graphic above shows a sample DevOps pipeline using Gitflow with recommended testing practices, isolated into three stages: build and integration, quality assurance, and staging and preproduction. Each stage has a series of quality gates to allow a quantitative pass/fail decision. For example, the QA stage requires feature tests, system tests, integration tests, and smoke tests to all pass in order to move to the next stage.
Utilizing the best-practice branch-merge strategy, we would like to have a series of branches that one commit would traverse through to reach production, as depicted in the table below.
| Branch | Description | Quality gates | On success, merges to |
| Story-132 | This represents the developers’ initial story branch where changes are made in isolation. | Changes are compiled, and unit tests and static code analysis is performed on the branch. If a fast-forward merge is possible without conflict, the code passes all quality gates. | Integration |
| Integration | This represents code that has passed development’s “sniff test.” This branch is typically locked down and commits can only be made by the build tool, to ensure developers haven’t gamed the system. | Code is again compiled and a deployable package is built for the QA stage. After a deployment, the pipeline would perform smoke tests, feature tests, and system and integration tests. If all were successful, it would pass all quality gates. | QA |
| QA | This represents code that has passed the QA stage and is ready for testing in a staging environment. | The deployable package is deployed to a staging environment and the pipeline performs security, performance and load, regression, and accessibility testing. If all tests pass successfully, the code passes the quality gates. | Staging |
| Staging | This represents code that is ready to be deployed to production. | After a manual inspection by a release manager, the code is deployed to production. If the deploy was successful, it passes the quality gate for this stage. | Master |
There are two major benefits to creating a branch-merge policy for a development organization:
- You have a policy that allows the integration of rapid changes while still enforcing mechanisms to ensure quality, which removes the confusion of moving fast and provides direction to your development team
- Separating changes into small, discrete units can help encourage testing changes in isolation, increasing the odds of identifying bugs and defects earlier in your software development lifecycle
Teams should adopt the best-fitting branch-merge strategy and rely on existing resources and plugins from their build orchestration tools. Over time, the team can iteratively add quality gates and adopt smaller-scoped branches, reducing release sizes and cycle time to bring features to production faster.
The first step is deciding which branch-merge strategy is a fit for your organization, both culturally and technically.
This is a repost of my article on Agile Connection.
