In his book The Nature of Technology, W. Brian Arthur describes technology as combinations of other technologies.
Take a software program like Git. To figure out what technologies Git is made of, perhaps all we have to do is inspect its dependencies? Surprisingly, Git has very few real dependencies.¹
Arthur’s description of technology is recursive, so where does it end? Arthur’s answer is in elemental technologies. These are technologies which deliver an effect based on a natural phenomenon:
That certain objects—pendulums or quartz crystals—oscillate at a steady given frequency is a phenomenon. Using this phenomenon for time keeping constitutes a principle, and yields from this a clock.— W. Brian Arthur, The Nature of Technology
If we re-examine Git not by its software dependencies, but by the algorithms and data structures in its source², we get closer. For example, consider merging:
- Recursive three way merge is built from Diff3 and a DAG of commits;
- Commit objects associate metadata with a top-level tree;
- Tree objects are directory-like structures which associate filenames with sub-trees and hashes;
- Hashes are SHA-1 hashes of blobs;
- Blobs consist of a header string and DEFLATE-compressed binary data.
Diff3 operates on the phenomena of structural similarity to reconcile two edits with a common ancestor. SHA-1 takes advantage of the uniform distribution of modulo. Lossless compression algorithms like DEFLATE operate on the phenomenon of statistical redundancy in data.
A technology is a programming of phenomena to our purposes.— W. Brian Arthur, The Nature of Technology
A Computing Ontology
We can model Git’s merging ability as an ontology of technologies:
Note the hierarchy: the lines denote “using” relationships. Git’s recursive three-way merge uses a version of Diff3 that uses Myers’ difference algorithm.³
Moreover, the importance of a member can be shown by its descendants. The broader the applicability and larger the effect, the more descendants it will have. This is why the dictionary (hashed binary tree of pairs) is an important data structure, but the config struct for any particular app is not.
“Importance” here is different from economic value. Schumpeter explained that most economic benefits are not delivered by invention, but by innovation and the subsequent diffusion of new technology throughout an economy.⁴
Innovation’s frontier
Zooming out from Git, imagine that we have modeled all computing technologies as an ontology. We could use the Computing Ontology to frame some key terms:
- Invention is the discovery of a new phenomenon and an algorithm and/or data structure to harness it. A new node is added.
- Innovation is a novel combination of existing nodes to accomplish a new effect; new child nodes are added.
- Configuration is the software engineering we do every day. Creating apps, fixing bugs etc. These create or modify child nodes, but they’re unlikely to have descendants.
In terms of the number of descendants: invention > innovation > configuration.
The frontier of computing innovation then, is defined as the set of all unique combinations of all nodes in the Computing Ontology.
Git’s speed, correctness guarantees and efficient use of space were essential to its success.⁵ Without the availability of suitable elemental technologies, Git could not exist.
Disruption
What would it take to displace Git? Discovering a better algorithm that uses the same data structures wouldn’t be enough as Git could adopt it. For example, Git now defaults to using SHA-256 instead of SHA-1.
It seems like disruption would require an innovation that created a fundamentally better way of version control. Such as using a schema-aware difference algorithm. Maybe no such technologies exist yet, in which case they would need to be invented first.
But that’s not all:
I am struck that innovation emerges when people are faced by problems—particular, well-specified problems.— W. Brian Arthur, The Nature of Technology
What makes disruption possible is not just that the elemental technologies must have been invented, but that the problem/opportunity has been identified.
So whilst Git seems secure for now, the Infrastructure-as-Code paradigm does not. Brian Grant’s excellent series on IaC identifies fundamental issues with how tools like Terraform work.
We can model the effects of disruption using the Computing Ontology; the disrupted technology and its descendant nodes will be discarded in favor of a new hierarchy. Arthur calls this destructive cascade an “avalanche”. Unrelated technologies, or those higher up the mountain will be spared.
Notes
- E.G. in Git 2.47.1 many dependencies are optional.
- Git Internals - Git Objects describes some of these.
- Git uses the linear space variant, which occasionally makes for some weird diffs.
- The Theory of Economic Development, Joseph, A. Schumpeter.
- Linus explained this in an early talk at Google. Git was so much faster than the rest that it changed how users actually did version control.