What’s a monorepo? Imagine if every time you needed to make a new source control repository, you just made a new top-level folder in your current repository instead. You’re doing monorepo! All the software written by everyone in your company stored together, versioned together.
Releasing a shared library
Owning a shared library is hard. You establish a public API and try not to change it, because breaking changes are really hard to push into users codebases. Your users put off the cost of upgrades, even easy ones. If you make a breaking change, users will delay upgrading, so how do you roll it out? If a user finds a bug, will they ask you to back-port the fix into a patch release on an older version? Then you discover that users don’t actually respect the public API, instead as the number of dependents grows they bake in unintended assumptions that any behavior of your software will remain the same (it’s been dubbed Hyrum’s Law: http://hyrumslaw.com ). Changes you thought were non-breaking can cause the same problems. You can easily find your job taken over with release engineering tasks.
Release engineering is all this careful work of tagging versions of various systems such that they work together. It’s part of the umbrella term “integration” — mixing streams of changes across several systems and seeing what happens. Many enterprises have processes where a QA or staging environment mixes all the parts together, where they bake for a week or longer. You find lots of bugs here, and they’re expensive to fix. That’s due to several factors: the bad change was made long ago, the QA environment is unavailable for other purposes until all the integration is complete, and you are interacting with parts of the org where your communication mechanisms are less effective at a distance.
A key consequence of a monorepo is that release engineering doesn’t happen anymore, at least for dependencies between two systems in the same repo. Because all our code is in one repo, Google has no shared QA/Staging environment! The monorepo changes the way you interact with other teams such that everything is always integrated. And hey, our industry has a name for that: continuous integration. If you don’t have a monorepo, you’re not really doing continuous integration, you’re doing frequent integration at best. In a monorepo, even pre-commit testing is already integrated.
The difference is profound. When we make any change to Angular, we need to sync this into Google’s monorepo. Doing so means that every Angular user immediately gets the change. Every commit is a release! Read that sentence again. Did you really read it again? Every commit is a release! That requires that we get really good at doing Continuous Integration: we run the user’s tests to make sure it’s safe to release Angular many times a day. This also keeps us honest about breaking changes. We pay the cost of upgrading users at the moment we land the change. Doing that requires we either use the great tooling we’ve built at scale, or narrow down the scope of the breakage (or both).
Continuous Integration
If you make a change, what should you build and test? Ideally, anything that depends on the code you just changed.
A naive build system does the same work every time you run it. If you ask it to build and test, it will build all the code in the repository and run all the tests. And it does this each time you make a change.
However, the daily resource demand for naive CI is theoretically quadratic O(C × T):
C is the number of changes committed by all engineers per day
T is the cumulative resource requirement of all the tests
T is the cumulative resource requirement of all the tests
A monorepo increases these factors a lot.
"Google chose the monolithic-source-management strategy in 1999 when the existing Google codebase was migrated from CVS to Perforce. Early Google engineers maintained that a single repository was strictly better than splitting up the codebase, though at the time they did not anticipate the future scale of the codebase and all the supporting tooling that would be built to make the scaling feasible.
Over the years, as the investment required to continue scaling the centralized repository grew, Google leadership occasionally considered whether it would make sense to move from the monolithic model. Despite the effort required, Google repeatedly chose to stick with the central repository due to its advantages.
The monolithic model of source code management is not for everyone. It is best suited to organizations like Google, with an open and collaborative culture. It would not work well for organizations where large parts of the codebase are private or hidden between groups.
At Google, we have found, with some investment, the monolithic model of source management can scale successfully to a codebase with more than one billion files, 35 million commits, and thousands of users around the globe. As the scale and complexity of projects both inside and outside Google continue to grow, we hope the analysis and workflow described in this article can benefit others weighing decisions on the long-term structure for their codebases."