Code

Microservices Reify Compute

The complexity is real, suffering temporary

December 10, 2024

Twenty five years ago the most popular solution for web apps was known as “LAMP” (Linux, Apache, MySQL, PHP). A typical web app today, deployed with Kubernetes might comprise of 5x as many technologies. That we don’t use such acronyms anymore tells us something about how software engineering has changed.

This has mostly been a journey from implicit to explicit state. For example, Kubernetes manifests declare how much memory an application will need. This is necessary to make deployments' resources calculable. But this requires more work from the software engineer, to estimate and declare the memory needed by the app.

Under a monolithic deployment, no memory declaration is required. Co-deployed apps share the machine’s memory. This simpler arrangement is fine for smaller scales. However once scale drives aggregate demand to exceed available memory, the problem becomes intractable. Some explicit resource allocation is needed - at this stage it could be accomplished by segmenting the apps and deploying each segment to their own machine for example. Microservices are the end state of this journey.

The evolution from the implicit to the explicit is necessary for progress. Why? Well, fundamentally, all technological progress depends on increasing order:¹

The more ordered and improbable our world becomes, the richer we become, and, as a consequence, the more disordered the universe becomes overall.

— John Constable, Energy, Entropy and the Theory of Wealth

As mentioned, explicit declarations make calculation possible. More than that though, their tools introduce a layer of indirection between expectation and outcome. Docker, Terraform, Kubernetes, etc. all reify aspects of computation. And the more we can reason about, the more we can strategize, plan and act.

Complexity

A popular way to think about complexity in software is to distinguish between the “essential” complexity of the problem and the “accidental” complexity introduced by attempting to solve it.² But this distinction tends to fixate our thinking; “Complexity” becomes a boogeyman, terrifying engineers, blowing budgets and ruining plans.³

Instead of treating complexity like a symptom of bad design, we should embrace it.⁴ The first step is getting away from the metaphysical concept of complexity-qua-complexity. We must make it real.

All complexity is perceived - so we should ask: “who is perceiving it?”. Not everything has to be solved with technology. Are new members of the team taking too long to start contributing? Maybe they should be paired with a team member for their first contribution.

Commonalities provide a base for new abstractions. If your annual survey says most engineers are struggling with Terraform, can infrastructure management be abstracted? Usually a handful of archetypes describes most apps' infrastructure. Addressing commonalities to improve engineer velocity is the start of creating an Internal Developer Platform.

Most engineers are familiar with the pattern of identifying commonalities and abstracting them.⁵ Far fewer are familiar with the idea of bringing intelligence to solve important problems. For example, typically we only know if Kubernetes will horizontally scale a pod by observing it in action. But we could parse autoscaler expressions and simulate scaling conditions, so that we could prevent deployment of apps that don’t scale as expected. Even better, an intelligent tool could help engineers write correct scaling rules in the first place.

And organizations aren’t on their own. There is a growing market of tools and services to help. Manage Kubernetes applications from the terminal with K9. Visualize deployments with Argo CD. Cut your logging spend with Grafana Loki. Kubecon and other events are great ways to learn about new tools and services that can deliver more value from microservices.

To deliver scale, microservices made software engineering more complex. But it’s not the end of the story. Our primitives will get better.⁶ Reifying compute has created a new logical substrate to build on, and it is full of possibilities.

Notes

  1. Energy, Entropy and the Theory of Wealth, John Constable, 2016.
  2. No Silver Bullet, Fred Brooks, 1986.
  3. E.G. Complexity is killing software developers.
  4. Ceteris paribus, simpler is better. But does a simpler solution exist? When a solution is described as “too complex” I find it helpful to ask myself “as compared to what?”.
  5. Don’t Repeat Yourself.
  6. E.G. Infrastructure as Code - Can We Do Better? is a fascinating review of the unsatisfactory state of IaC and potential improvements.

Tags: distributed-systems complexity software-engineering reification entropy