Technology as a Search Problem

Comments

One of my ongoing research interests is understanding the nature of technological progress: How does new technology come into the world and improve over time? What sort of factors affect it? How, in general, should we think about it?

I've come at this question from a few different angles: posts looking at the operations and legacy of Bell Labs, posts looking at how specific technologies improved over time, posts looking at how frequent multiple invention is and how long we wait for new inventions, and so on.

My current line of thinking is that creating some new technology is best thought of as a search process. A given technology can be thought of as a point in some high-dimensional "design space" or "technology space," which contains every possible technology: every conceivable arrangement of parts, materials, natural phenomena, and so on.

The job of the would-be inventor is to navigate this space, sifting through the arrangements of elements that don't work to find the ones that will.

This search process is made difficult by the fact that, as with any situation where you're enumerating the possible combinations of a large number of elements, technology space is incomprehensibly vast.

In "Are Technologies Inevitable?" economist Matt Clancy notes that Wikipedia listed 272 "named equations" that describe various regularities in the world, ranging from the Navier-Stokes equations (which describe the motion of fluids) to the Arrhenius equation (which describes chemical reaction rates as a function of temperature) to the Exner equation (which describes how sediment gets deposited in a river).

If we use this as a (very rough) estimate as to the number of different phenomena that a technology can leverage, it implies an enormous number of possible technologies that could be created. As Clancy writes:

If we've got 272 regularities to work with, for example, then the number of possible combinations of these regularities is two, raised to the power of 272 (since every regularity can be used or not used: two options). But 2^272 is more than some estimates of the number of atoms in the observable universe.

And that's got to be an undercount of the number of possible technologies, since it counts all technologies that use the same principles together as the same thing, even if they orchestrate them in vastly different ways or use them in vastly different magnitudes.

By comparison, the most common estimate for the number of atoms in the universe is around 10^80, or ~2^265.

And it's likely there are many, many more regularities that some technology could harness than 272. (Some in-progress work on this that I haven't yet completed suggests that there are at least thousands.)

And as Clancy notes, this outrageously huge number dramatically undercounts the number of possible technologies, because it doesn't take into account that the same phenomena can be used in lots of different ways.

In Brian Arthur and Wolfgang Polak's paper "The Evolution of Technology within a Simple Computer Model" (which we looked at earlier this year), Arthur and Polak note that even within the universe of logic circuits made up of three basic elements (the NAND gate and the values of 1 and 0), the number of different circuits you can make for even a small number of inputs and outputs is staggeringly large:

The emergence of circuits such as 8-bit adders seems not difficult. But consider the combinatorics. If a component has n inputs and m outputs there are (2^m)^(2^n) possible phenotypes, each of which could be realized in a practical way by a large number of different circuits. For example, an 8-bit adder is one of over 10^177,554 phenotypes with 16 inputs and 9 outputs.

Search Space and Information

So the search space for some new technology is vast. And the way that we tackle vast search spaces is with information. Each bit of information cuts a search space in half, so even a relatively small amount of information can quickly narrow down what seems like a huge search space.

When we create some new technology, what we're doing is gathering information: figuring out what will work and what won't work, what's promising and what isn't, narrowing down the huge space of technological possibilities and finding something within it that accomplishes some particular goal.

In the 19th century, for instance, many people were interested in building a mechanical flying machine, but it wasn't clear the exact arrangement of elements that would yield a successful one.

Thanks to knowledge of the laws of physics, and an existence proof of birds, would-be inventors were already working within a considerably narrowed search space: they knew roughly what elements a flying machine would need to have - lifting surfaces, a propulsion system, and a means of control - and had a general idea of how they should be arranged.

But there were a huge number of ways that you could build a flying machine from these basic elements, and people spent a great deal of time and effort building different sorts of machines that did not in fact work.

The Wright brothers' eventual success can in large part be attributed to the fact that they were exceptionally talented at gathering information, enabling them to converge on a working airplane design far faster than their competitors.

I previously explored the nature of technological search in that previous piece looking at Arthur and Polak technological evolution paper, which uses a simple simulation to evolve different logic circuits, finding workable technologies amidst an enormous sea of possibilities.

I really liked the basic simulation method Arthur used in this paper - randomly generating technologies to see how easy or hard it was to find working ones - and wanted to use simulation myself to further explore the dynamics of technological search.

But while Arthur uses digital circuits - different arrangements of logic gates like NAND, OR, AND, etc. that can perform functions like addition - I wanted a more general model that could capture the nature of many different types of technology, while still being simple enough to simulate easily.

The PTOP Model

The model I came up with is inspired by another of Brian Arthur's examinations of the nature of technology: his book The Nature of Technology. In this book Arthur describes technology as fundamentally "a set of phenomena captured and put to use." He writes that:

... a practical technology consists of many phenomena working together. A radio receiver is not just a collection of parts. Nor is it merely a miniature signal processing factory. It is the orchestration of a collection of phenomena - induction, electron attraction and repulsion, electron emission, voltage drop across resistance, frequency resonance - all convened and set up to work in concert for a specific purpose.

Edison's lightbulb, for instance, took advantage of several different phenomena:

• Joule heating, the phenomenon that electric current running through a material will raise its temperature.
• Incandescence, the phenomenon that a hot enough object will emit light.
• Transparency, the phenomenon that certain materials will allow the passage of light.
• And so on.

Phenomena like this are typically triggered automatically when the right conditions are created, which you can think about as being when the right objects, with the right properties, are placed in some particular arrangement.

In the lightbulb, Joule heating occurs automatically when electric current runs through the filament, and incandescence occurs automatically when the filament gets hot enough. And the bulb had a vacuum atmosphere to prevent another, unwanted phenomenon - carbon reacting with oxygen - automatically occurring and destroying the carbon filament.

These three elements - Phenomena, Objects, and Properties - are the basic components of the technological model I came up with, which I call PTOP, for Phenomena Triggered by Objects with Properties.

In this model you have some collection of objects, which can have some specified set of properties. Whenever the right objects with the right properties are placed in the proper arrangement, a phenomenon will be automatically triggered, which can have various effects: modifying the properties of existing objects, creating new objects, and so on.

By chaining the correct objects and phenomena together in the correct order, you can create your desired outcome, which is some particular object or objects with some specific set of desirable properties.

A simple, conceptual PTOP model for the lightbulb, for instance, might look like this:

By arranging the right sort of filament, an atmosphere, a bulb container, and electric current, you can trigger a series of phenomena that will ultimately result in the creation of light with desirable properties.

And here's a sketch of a PTOP model for a more complicated technology, a Xerox machine, which uses various phenomena such as "photoconductivity" and "corona discharge" to generate a copy of a document:

I like this model because it captures the basic elements - natural phenomena and the conditions that trigger them - inherent to all types of technology, such that we can study the 'space' of possible technologies.

And you can implement the basic idea in many different ways, from very simple representations of technology to very complex ones. In some ways this is bad (it took me a super long time, for instance, to come up with a simulable implementation that I was happy with) but it's useful for studying different aspects of technological search.

Let's look at a few different implementations of a PTOP model and see what we find.

A Level 1 PTOP Model

Let's start with an extremely simple version of PTOP. In this version, we have 10 possible objects, each with two possible properties.

The first property, P0, can take on any value from 1 to 10 and is inherent and unique to that object. You can think of this as the model having 10 different chemical elements, each taking on a different value of P0.

The second property is size, which can also take on any value from 1 to 10, but is adjustable. So you can have a large object with P0 equal to 1 and size equal to 10, or a small object with P0 equal to 1 and size equal to 1.

Overall there are 100 possible sorts of objects: 10 different object types, each of which can be 10 different sizes.

This version of the model has four phenomena in total. Each phenomenon is triggered when the right two objects are connected, and the first of those objects (what we'll call the input object) has the correct size.

When the phenomenon is triggered, it adds a specific property to the second object (what we'll call the output object). So in the image below, when an object with a P0 value of 1 and size 5 is connected to an object with a P0 value of 2, it triggers a phenomenon, which adds a new property, P1, to the second object.

The phenomena in this model are arranged in a hierarchy: triggering each phenomenon requires an object with the property added by the previous phenomenon. So to trigger the phenomenon that adds P2, you need an object with the P1 property. P3, in turn, requires an object with P2, and so on.

You can think of this hierarchy of phenomena as representing something like "to trigger incandescence, I need an object with a high temperature. To achieve high temperature, I need to use the phenomenon of Joule heating, which I can trigger with the right sort of electric current. To create electric current, I need ..." and so on.

In this version of the model, the goal is to create an object with the "final" property of P4. You can think of this as representing something like "light with the proper brightness" or "lead shot with the proper size and sphericity."

This can be done by creating the exact right arrangement of objects: five objects arranged in a line, each with the correct P0 property and size. This is a very simple implementation of PTOP, which we'll call a "level 1 model."

But it does capture a few things about the technological search process. First, the search space in even this very simple model is very large: 10 object types, 10 sizes, and 5 objects total means (10x10)^5 = 10 billion possible arrangements to produce a P5-level phenomenon.1 If you increased that to 100 objects, 100 sizes, and 10 different phenomena, you'd have 10^40 possible options.

If we added more property types (maybe objects can have the properties of "temperature" and "density" in addition to size) the numbers would quickly become beyond astronomical.

But this simple version also shows the power that information can bring to bear on a large search space. When we looked at Arthur's circuit search, we noted that what made the search tractable was modularity - finding a working design for a subcomponent, then locking that modular design into the system and using it as a repeatable 'primitive' element for further combinations.

By preventing you from having to get every single part of a design correct in one shot, modularity effectively screens off many possibilities and reduces the size of the search space that must be traversed. As we found in that simulation:

A useful way of thinking about our 68 NAND gate search is that it's like a huge, branching tree. At every step - every time we add a gate - there are thousands of branches, each one representing one possible way to wire up the gate. Each branch then splits into thousands more (representing all the possible ways of wiring up the next gate), which split into thousands more, which split into thousands more, until at the end we have 2^852 possible "leaves," each one representing a unique way of wiring up all 68 gates. Trying to get all 68 gates right at once, and then checking to see whether or not you did, is like examining one single leaf, one path from the base of the tree all the way to the tip of a branch. Not only are you overwhelmingly likely to guess wrong, but you haven't narrowed down your possibilities at all: all you know is that one single leaf