Engineering Principles

tl;dr: Betterment’s collaboratively-maintained engineering principles explain our practices and establish interlocking chains of reasoning that help us build better software than what came before. Cut to our principles.

What We Mean by Principles

The first definition of principle in the Oxford English Dictionary is:

A fundamental truth or proposition that serves as the foundation for a system of belief or behaviour or for a chain of reasoning.

Without getting too lofty about the meaning of truth, whether a corporation should be trafficking in belief systems, or what constitutes good behavior, we can trim that down to a tidy core definition for our purposes:

A principle is a proposition that serves as the foundation for a chain of reasoning.

This is important because the chain of reasoning is where the leverage is. All organizations have goals, and they must communicate effectively to achieve them. As an organization scales, the number of potential communication channels grows combinatorially. The resulting communication overhead is one of the biggest threats to individual impact and team productivity. With a large shared context of principles and practices connected by interlocking chains of reasoning, orgs can vastly lower the communication overhead to achieve those goals.

So what, concretely, is a principle? Principles explain practices. If a practice is of the form “we do X,” a principle is of the form “[Given context,] we do X because rationale.”

Applying Principles to Engineering

An engineering principle at Betterment might describe anything from how we choose a design pattern to how we shape and staff teams.

As an engineering team, when we have shared context feeding our collective intuition, we can skip a lot of preamble with each new design or initiative. We can leverage the energy that we as a team and the broader software engineering community have already invested, and avoid reinventing the wheel.

But healthy shared principles don’t stifle innovation, and Betterment’s engineering principles mustn’t. When principles are right, they can help us identify areas where invention is vital, and where it can have outsized impact.

At Betterment, we continuously integrate our latest thinking into our base of shared principles. But when a previously accepted principle doesn’t hold up to scrutiny, or isn’t compatible with a principle we’d like to adopt, we must act. We can supersede it with updated thinking reflecting our learnings. We can remove it altogether. Or we can add qualifying context until it lives up to its promise: to serve as the foundation for an incisive chain of reasoning. But we can’t leave it alone, knowing that it could set up others on the team for confusion or failure.

When we get the grooming process right, insights that have eluded us tend to snap into focus across the organization with each new iteration.

Standing on the Shoulders of Giants

We have the opportunity to build on the work of others - ranging from our own teammates and leaders of the young field of software development to thinkers, experimenters, builders, and practitioners throughout history.

But in software, the generally accepted principles don’t go far enough. The industry is still nascent, and it doesn’t have a comprehensive, consistent set of solutions to problems that teams face every day. Sometimes this is because a short-sighted practice hasn’t been recognized by the industry and eradicated yet, or a promising practice is still taking root. Sometimes different communities take vastly different approaches that work on their own but don’t integrate well, or exhibit tradeoffs that make their solutions unsuitable for us. Sometimes the seeming wisdom that “it depends” prevents the broader community from coalescing around a high-leverage insight.

We have a unique combination of ingredients at Betterment: a collaborative team stacked with horsepower, creativity, thoughtfulness, and experience both within and beyond the engineering org. We have an executive team who understand that in order to succeed as the world’s first and most enduring digital-native financial institution, we must improve continuously, indefinitely. We have the track record and opportunity to change how software gets developed in the modern world. We can do it faster, smarter, safer, and better than those who came before us.

Our current set of principles and practices isn’t going to get us there on its own, though. To succeed we must build and maintain our principles, together, with the intent to create a whole greater than the sum of its parts. The absence of a principle shouldn’t be interpreted as a barrier, but an opportunity to adopt and contextualize new high-leverage practices and technologies into new principles that align with or amend our current ones.

These principles aren’t all grand and sweeping. Some of them are extremely granular and may even seem trite, but all of them are applicable to the kinds of work we commonly do at Betterment, and we benefit from sharing them. Teams within Betterment maintain their own compatible local principles, too. Some of them will prove useful to other folks and bubble up to this list, and others will remain within their teams, helping them communicate and solve difficult challenges specific to their problem domains.

To paraphrase the rifleman’s creed: These are our principles. There are many like them, but these are ours.

Our Principles

Glossary

Business Domain

A meaningful business function that could be understood by a customer as a necessary component of our business. Business domains are eligible to become application domains if the need arises (e.g. trading, retail, custody, b4b), but we don’t have to build an application for each business domain. For example, CRM is a business domain, but we buy CRM software as a SaaS product. Finance is a business domain, but we haven’t had to build any apps for them as a combination of NetSuite, spreadsheets, and other products satisfy their technology needs.

Application Domain

The largest unit of shared ownership in the domain oriented services model. In a healthy ecosystem, this would comprise one repository containing one or more applications written in the same language with a single core business logic library depending on a single database and single file store at runtime.

Domain Module

A single cohesive feature or capability of a piece of software. A typical application or library will be made up of tens of domain modules.

Domain Oriented Services

Our term for apps that center on customer-driven business domains, and which, in order to leverage local transactionality over multi-object operations, replicate needed facts in their own databases. This approach is like what you would do if each app were its own little startup and wanted to run independently of its service provider apps and customer apps. In order to reap the benefits of domain oriented services without taking on unwieldy per-feature data synchronization concerns, we lean on a small set of distributed system consistency tools and patterns.

Entity Oriented Services

Our term for a common pattern in industry where apps center on entities (business objects) rather than functional domains. An example entity oriented service would be an account service, which provides all persistence of a single entity (accounts) or a set of account-related entities to a variety of collaborator applications. Entity oriented services have the advantage of providing a single org-wide source of truth for the facts they hold. But to do so, they must either integrate tightly with their many collaborator services, or provide a database-like API surface area a la GraphQL. Additionally, they must either absorb transactional business logic belonging to consumer business domains, far from the business logic’s ownership team, or eliminate transactionality from dependent apps’ toolboxes for those entities. Organizationally, these characteristics lead to either a low-morale, stable ownership team without a business mandate, with split allegiances to its collaborators and limited visibility into future architectural demands, or a shared-maintenance model where contributing teams must fight to rationalize their needs into a common model that satisfies them both, resulting in an unlovable and under-maintained design by committee.

Flyweight Events

Our term for a pattern where the event producer service sends a notification containing only the identifier for the impacted entity (regardless of whether it was a creation, update, or destruction event). Because no state is included in the message, the receiver can retrieve the entity from the producer service eventually and be guaranteed to have received the latest state at the time, regardless of delivery order of the notification events. Note that this does not guarantee that the receiver will see all interim states of the entity. If that is required, using immutable entities may be needed, and other patterns may be appropriate.