In small projects, there are typically a few developers and a few branches underway. As those branches reach readiness, depending on the flow of the project each developer may merge their own branches, or a tech lead may merge on request. Or sometimes they pile up (in a queue, conceptually) for review and merge.
But in large teams working (either a large single project or a monorepo) there can be dozens/hundreds/thousands of developers creating branches all the time. Many teams have the practice of doing a modest or moderate amount of work per branch, resulting in a large number of branches that must be merged to move the work forward.
To be merged, a change (branch) needs to be at least conceptually on top of the current mainline (regardless of whether a merge-commit or rebase strategy is used), and tests/lint/build/etc must pass. As each merge happens, the baseline moves forward – so the next candidate sometimes can’t be accurately tested until that point. At scale, this becomes the limiting factor in system progress: If the tests take 1 hour, the maximum number of merges you can do per day can be as low as 24.
Long before that limit though, having humans perform each of these operations is impractical, so teams typically adopt an automated system to queue up proposed changes/branches for merge. The function is roughly:
Developers submit a branch as ready for merge, into a queue; often mediated by a human review and approval process.
Automated merge system keeps picking the top thing in the queue.
Rebase it on the current mainline.
Test, Lint, Build, etc.
When a change fails to re-baseline / merge / test, notify the developers working on that branch, they will need to resolve either a syntactic or semantic merge conflict and resubmit.
Of course there are strategies to increase parallelism; the process remains conceptually serialized but practically parallelized.
Related or unrelated branches can be combined and tested + merged as a group.
Use a build system that has a great understanding of what subset must be re-tested for a given set of changes.
Pre-build each candidate on a recent baseline, to pre-populate a build cache; a well-crafted build process (with Bazel or other tools) can safely optimize away some of the rebuild-everything process.
Here are some of the available merge queue tools – but many organizations create an internal merge queue management system.
Google’s Go language (“golang”) was first released in early form in 2009, and reached 1.0 in 2012. Go has matured quite quickly compared to many other new languages. We find several aspects of Go appealing:
Go is a pragmatic language, with features chosen to ease and speed development of substantial projects.
Go uniquely combines a rapid cycle “scripting” language feel with a robust static compilation process.
Go’s compiler generates statically linked binaries; this is very “old-school” in 2015, but static compilation trades off disk space (which is cheap) for simplicity and reliability in production (which is valuable).
It is suitable for both small and large teams, although our work so far as been small.
Go has good support for concurrency, well-suited to software that scales up to run on many core machines.
Go has proven relatively easy to learn, for developers coming from a wide variety of backgrounds.
Go has a good standard library.
Go can target multiple platforms, yet does not use a separately installed runtime.
Go Development at Oasis Digital
Oasis Digital has made use of Go for various small systems, including: