Scaling Codebases Without Platform Bloat
The Hidden Tax on Developer Velocity
Redundant compilation lags, bloated CI pipelines, and version drift for internal modules quietly drain developer velocity. And normally, it's too late when leadership gets wind of it.
According to recent studies, the hidden tax on your team looks exactly like this:
- The Direct Time Drain: Your engineers waste an average of 25 to 50 minutes due to compilation lags.
- The Redundant Build: An engineer wastes valuable time rebuilding something that already exists and is being used in another team.
- The Pipeline Blocker: Your Frontend Team spends an entire sprint cycle completely idle, waiting for the Core Platform Team to version-tag and publish the latest approved button changes.
All these, as bad as they sound, are common occurrences in the majority of software engineering teams trying their best to ship fast and iterate faster. The problems listed are not inevitable, and can be solved if we carefully design our engineering culture and the codebase that it revolves around from the beginning.
In today's edition, let's tackle them one-by-one and continue where we left off last time in our adventure of building the ideal monorepo setup for startups. Assuming that you followed my advice of having a monorepo for all your codebase needs for your startup, let's hit the road.
The Redundant Build: A Guide to Module Re-Use
While there's no sure-shot way of knowing what an engineer thinks before embarking on the painstaking journey of rebuilding the wheel, we can mitigate the risk through some primary cautions:
- Build a collection of re-usable modules.
- Business logic must never be allowed to be repeated.
- Engineers should learn about the existing system for their use-case.
Though I know, forcing software engineers to do something they don't approve of is never a good idea. But companies and team leads should enforce these elementary coding principles rigorously via code-reviews and documentation.
Building a Collection of Reusable Modules
Startups that successfully keep growing also tend to constantly rebuild many parts of their system again and again. To give you a few examples:
UI Components: The same kind of button, the same navigation bar, the same sidebar nav, etc. are rebuilt across different product lines to keep the look and feel of the brand consistent. The website may be using the same color scheme as the product dashboard, and hence the components get duplicated.
Backend Modules: If you have more than one backend, chances are that you are duplicating the authentication, the logging, the middlewares, the utility modules, and many others depending upon your particular use-case.
Due to this, some parts of your website might look and feel different than other parts - worst-case, it might feel they are built by different companies altogether. And it's not just about how the UI looks. If you have duplicated backend code, your team sometimes might end up debugging an edge-case over a weekend that an engineer didn't think of, while rebuilding the authentication pipeline all over again.
To best avoid the above duplication, use the packages directory in your monorepo root judiciously - that is, if you followed the monorepo structure that I advocated in the previous edition. And if you are already doing that, it's better to keep the reusable modules inside the packages directory - loosely coupled, highly parameterised, and customizable through environment variables and feature flags.
Consistent Business Logic
If there's one thing that you should absolutely avoid, then it has got to be business logic duplication. Given enough time, codes change, bugs appear and get fixed, engineers might come and go, but hardcore business logic seldom changes.
To give you an example, if you are building an accounting system, then your tax calculation logic must never be repeated. If it does get repeated across your backends and your frontends, then each update to the tax calculation logic, debugging sessions, and eventual hot-fixing would need to be made across the board and be forced to be compatible in each of the environments, causing huge business consequences.
I would recommend keeping a separate privileged module inside your shared-modules for such sensitive business logic, and have a CODEOWNERS entry such that unauthorised engineers (most likely the interns) can never make any changes to the same. An example CODEOWNERS entry might look something like this:
# .github/CODEOWNERS
# Guarding core business logic from unauthorized modifications
# Global fallback auditors
* @esence-io/platform-leads
# Strict guardrails for sensitive domain algorithms
/packages/domains/billing/tax-calculator.ts @esence-io/finance-architects
/packages/domains/appointments/compliance.go @esence-io/medical-directors
Documentation for Reusable Modules
This might be the only thing that separates an average organization from a great organization that has some degree of foresight. However, in an early-to-mid-stage startup, telling engineers to spend hours writing long-form documentation on a separate Confluence or Notion workspace is a fantasy. It creates out-of-date documentation platforms that nobody trusts, adding to your organizational bloat.
The frugal, high-velocity alternative is Git-Adjacent Documentation:
- Workspace READMEs: Every package inside your
/packages/*directory must contain a minimalistREADME.mdexplaining its API footprint, schema rules, and runtime assumptions. If an engineer alters code logic, they update the text in the exact same git commit string. - Strict Typing: Let your compiler act as your documentation hub. By enforcing explicit input/output interface boundaries on shared components, your IDE automatically tells the frontend engineer exactly what parameters a component expects without them ever leaving their workspace terminal window.
- Better Code Comments: Come on, no one writes good comments. If code isn't discoverable from inside the repo editor, it doesn't exist. Keep it in git and the engineers will follow.
Trunk-Based Development: Unblocking the Pipeline
What if every engineer works on the latest changes produced by every other team? And what if every team starts taking ownership of the changes they make? To answer both questions with a strong "Yes," your engineering team has to follow the paradigm of Trunk-Based Development.
Trunk-Based Development, as opposed to module-versioning and importing foreign modules from registries, means that a monorepo contains everything it needs and every module it has is on the latest version. Cross-module dependencies just become a question of either importing those modules or their DLLs or binaries, completely bypassing the network overhead of fetching latest modules.
There are no version numbers to increment, no private registries to maintain, and no upstream breaking changes hidden behind a semver tag. And no wasteful standup meetings discussing whether a change is breaking enough to bump up the version entirely.
Once your company starts following trunk-based development:
- If the Core Platform UI Team changes the button style or its parameters, it's going to be the Core UI Team's responsibility to change it across the codebase - thereby creating implicit ownership of all the changes that break integration tests or that make other project's compilers scream.
- The cost and complexity of infrastructure behind private module/package registries instantly become zero.
- Setting up the local environment for a new joinee just becomes executing the build commands.
- If a shared or inter-team module breaks, no one has to get blocked for getting fixes and updates.
- All teams, to some extent, become software/module testers for all the other teams by becoming direct consumers with a zero-lag feedback loop.
One easy way to do it for TypeScript workspaces is by simply including workspace dependencies like this:
"dependencies": {
"@packages/ui": "workspace:*"
}
It isn't complicated, unless you start making it.
Compilation Lags and Caching Build Artifacts: With Nx
You pay money-tax to the government for earning, and pay time-tax to the compiler for coding. Startups hire accountants for reducing tax paid to the government, but seldom pay any attention to the tax paid to the compiler that slowly leaks their pockets in terms of wasted productive hours of a developer on a payroll.
You can read the full article on my blog - Engineering Insights at Esence.io.
Comments
No comments yet. Start the discussion.