The History of AI, Part 3: Neurons on Paper, McCulloch, Pitts, and the Birth of Cybernetics

Part 2 to Part 3: Is the Brain a Computer?

The History of AI, Part 3 of 24. In Part 2, Alan Turing defined computation and asked whether machines could think.

Now we meet the people who asked a different question: what if the brain itself is a kind of computer? This is where neural networks, feedback, and information theory all begin.

By the early 1940s, two big ideas were sitting side by side, almost touching.

On one side: logic and computation. Boole had shown that reasoning can be written as algebra, and Turing had defined exactly what a computing machine is.

On the other side: neuroscience. Biologists had established that the brain is made of billions of individual cells called neurons, connected in vast networks, passing tiny electrical signals along their branches. A neuron receives signals, and if enough arrive, it "fires" and sends its own signal onward. That's about as much as anyone knew for sure.

Someone was bound to put them together and say: a neuron is a kind of logic gate, and the brain is a computing machine built from them. That someone turned out to be an odd partnership of a middle-aged psychiatrist and a homeless teenage prodigy.

This chapter is about the birth of the neural network idea, about a sprawling science called cybernetics that briefly promised to explain minds, machines, and societies in one framework, and about the quiet groundwork that made later AI possible. It's also, unusually for a history of technology, a story with a genuinely sad ending.

The Psychiatrist and the Runaway

Warren McCulloch (1898-1969) was a restless, poetic, charismatic American neurophysiologist and psychiatrist. He had studied philosophy and psychology and medicine, and he carried around a question that he said had obsessed him since youth: "What is a number, that a man may know it, and a man, that he may know a number?" In other words, how can a physical, biological thing, a brain, come to know abstract truths?

Walter Pitts (1923-1969) was a very different figure. Born into a rough, poor family in Detroit, he was a self-taught prodigy who, as the often-told story goes, escaped bullies as a boy by hiding in a library, where he read Bertrand Russell and Alfred North Whitehead's Principia Mathematica. He is said to have written to Russell with corrections, and Russell invited him to study in England. He couldn't go.

Instead, at around fifteen, he left home for good and wandered into the University of Chicago, where he sat in on lectures by the philosopher Rudolf Carnap and impressed everyone he met, despite having no official student status and often nowhere to live.

Through a fellow student, Jerome Lettvin, Pitts met McCulloch. McCulloch, who had a large house and a large heart, essentially took the young man in. The two began working together on a way to use logic to describe the brain.

"A Logical Calculus of the Ideas Immanent in Nervous Activity"

In 1943 they published a paper with that mouthful of a title in the Bulletin of Mathematical Biophysics. It is one of the most important and least-read papers in the history of AI.

Here is their central move. They took the real, messy biological neuron and simplified it brutally into an abstract unit with these properties:

• It has several inputs, each either on (1) or off (0).
• Some inputs are excitatory (they push the neuron toward firing), and some are inhibitory (they veto it).
• The neuron has a threshold. If enough excitatory inputs are on, and no inhibitory input is on, the neuron fires (outputs 1). Otherwise it stays silent (outputs 0).

That's all. And from this tiny building block, McCulloch and Pitts proved something remarkable: networks of these units can compute any logical function. With the right wiring you can build AND, OR, and NOT, and since you can build any logic circuit from those three, you can build any logical computation. They also argued that, given a tape, such networks could in principle compute anything a Turing machine can.

Let's see how little code this takes. Here is the McCulloch-Pitts neuron:

def mp_neuron(inputs, weights, threshold):
    """Fire (1) if the weighted sum of inputs reaches the threshold."""
    total = sum(i * w for i, w in zip(inputs, weights))
    return 1 if total >= threshold else 0

# AND: both inputs must be on
AND = lambda a, b: mp_neuron([a, b], [1, 1], threshold=2)

# OR: at least one input must be on
OR = lambda a, b: mp_neuron([a, b], [1, 1], threshold=1)

# NOT: an inhibitory (negative) weight flips the signal
NOT = lambda a: mp_neuron([a], [-1], threshold=0)

for a in (0, 1):
    for b in (0, 1):
        print(f"{a} {b} -> AND={AND(a, b)} OR={OR(a, b)} ")
print("NOT 0 = ", NOT(0), " | NOT 1 = ", NOT(1))

Run it and you'll see the truth tables Boole would recognize.

Real neurons are enormously more complicated than this, and McCulloch and Pitts knew it. But the paper's power isn't biological accuracy. It's the bridge. It said: whatever the brain does, if it can be described logically, then a network of simple threshold units can do it. Thinking might not be mystical. It might be wiring.

Wires and Weights

Notice the two ingredients that appear in the code: weights and a threshold. In the 1943 model, the weights were fixed by hand. Later researchers would let the weights change, and that single modification, learning, would become the central theme of modern AI.

A modern deep learning model has billions of weights, and nearly all of its "knowledge" is stored in them. But the core operation, "multiply inputs by weights, add them up, compare to a threshold," is still the McCulloch-Pitts neuron, with the hard threshold usually replaced by a smooth function.

There's one more thing worth noticing: a single McCulloch-Pitts unit can't compute everything. It can do AND and OR, but a single unit cannot compute XOR (exclusive or: "one or the other, but not both"). You need at least two layers of units to do it. That small fact will come back with a vengeance in Part 6, when it becomes the center of a famous and devastating critique of early neural networks.

Why It Mattered

The paper's impact was less about what researchers did with it immediately and more about the frame it created:

• It made the brain a legitimate object of mathematical study.
• It linked neuroscience to computation, suggesting the mind is something a machine might replicate.
• It inspired John von Neumann, who cited the paper in his 1945 report describing the stored-program computer architecture and used its neuron-like notation to describe how computer components might work.
• It also contained an idea that would take years to mature: networks with loops, where signals feed back into earlier units, could have memory. That insight prefigured recurrent neural networks.

Meeting Norbert Wiener

In 1943, Pitts, only 20 years old, moved to Cambridge, Massachusetts, drawn to MIT by one of the most formidable intellects in America: Norbert Wiener.

Norbert Wiener (right) examines the chess-playing automaton El Ajedrecista in 1951. Public domain, via Wikimedia Commons.

Wiener (1894-1964) was a former child prodigy who earned his PhD from Harvard at 18. A mathematician with wide-ranging curiosity and a reputation for absentmindedness (there are many stories of him wandering the halls of MIT lost in thought), he had spent the early war years on a very practical problem: anti-aircraft fire.

The Prediction Problem

To shoot down a fast-moving airplane, you can't aim at where it is. By the time your shell arrives, the plane will have moved. You must aim at where it will be. That requires predicting the pilot's path from recent observations, which is hard because the pilot is a human being trying to evade you.

Wiener, working with the engineer Julian Bigelow, developed mathematical methods for predicting such trajectories, treating the pilot's motion as a noisy signal and finding the best statistical guess about the future.

But they noticed something deeper: the whole system, a gun, a radar, a human operator, and an airplane, behaved as a loop. The gun makes an observation, adjusts its aim, observes the result, and corrects. This is feedback.

In a famous 1943 paper, "Behavior, Purpose and Teleology," Wiener, Bigelow, and the physiologist Arturo Rosenblueth argued that goal-directed behavior, in both animals and machines, could be explained through feedback. A thermostat has a "goal" (a target temperature). A guided missile has a "goal." A person reaching for a cup has a "goal." In each case, the system compares its current state with the desired state and acts to reduce the difference.

That sounds obvious today, but in 1943 it was heretical. Scientists had avoided talk of "purpose" as unscientific, a leftover from Aristotle. Wiener showed that purposeful behavior could be explained mechanistically.

Cybernetics

In 1948, Wiener published a book titled Cybernetics: Or Control and Communication in the Animal and the Machine. He had coined the word from the Greek kybernetes, "steersman", the person who steers a ship, constantly correcting course in response to wind and waves. (The same Greek root gives us "govern" and "governor.")

The book is a strange mixture of dense mathematics and sweeping speculation, and, to the astonishment of its publisher, it became a bestseller. Its central claim was that control and communication are the same kind of thing across biology, machines, and even society. Organisms, thermostats, computers, and economies are all systems that process information and use feedback to regulate themselves.

For a few years, cybernetics was the hot interdisciplinary idea. It influenced psychology, engineering, biology, anthropology, and management theory, and it supplied a vocabulary (feedback, input/output, signal, noise, control) that we still use constantly.

The Macy Conferences

The movement's social hub was a series of ten invitation-only meetings, the Macy Conferences, held between 1946 and 1953 in New York. They were the sort of gathering that seems almost impossible today: mathematicians, neurophysiologists, psychologists, anthropologists (including Margaret Mead and Gregory Bateson), engineers, and philosophers all in one room, arguing late into the night, with McCulloch chairing.

They debated how brains compute, how language works, how communities regulate themselves, and whether machines could think. Many of the questions that structured the next 70 years of cognitive science and AI were first sharpened in those rooms.

Claude Shannon: A Bit of Information

One regular in these circles quietly changed the world twice before he turned 33.

Claude Shannon in a 1958 MIT portrait. Public domain, via Wikimedia Commons.

Claude Shannon (1916-2001) was a playful, inventive engineer who loved unicycles, juggling, and building odd gadgets. In his 1937 master's thesis at MIT, "A Symbolic Analysis of Relay and Switching Circuits," he noticed that the on/off behavior of electrical switches could be described using Boolean algebra, the same algebra Boole had invented in 1854 to describe logic. Series circuits correspond to AND, parallel circuits to OR. This turned circuit design from an art into a mathematical discipline, and laid the foundation for digital computers. It is often called the most important master's thesis of the century.

Then, in 1948, the same year as Wiener's book, Shannon published "A Mathematical Theory of Communication," founding information theory. He defined the bit as the unit of information and showed how to measure the information content of a message, how much a noisy channel can carry, and how to encode messages so errors can be corrected. Every text message, streamed video, and compressed file depends on his ideas.

Shannon cared about machine intelligence too. In 1950 he published "Programming a Computer for Playing Chess," outlining the basic strategy of searching a tree of possible moves and scoring positions, which became the foundation of computer chess. He also built Theseus, a small mechanical mouse that could learn to find its way through a maze, one of the first demonstrations of a machine that visibly learned from experience.

His mathematics also hides a connection that will matter greatly in modern AI. The measure of information, entropy, is the same quantity that today's language models are trained to minimize (in the form of cross-entropy loss) when they predict the next word. Shannon was already experimenting in 1948 with predicting English text one letter at a time using statistical models. In a very real sense, today's chatbots are descendants of those experiments.

Hebb: How Brains Learn

McCulloch and Pitts left a gap. Their neurons had fixed connections. But real brains learn. How?

In 1949, the Canadian psychologist Donald Hebb proposed a simple and influential answer in his book The Organization of Behavior. In his words:

When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.

It is usually summarized, a little loosely, as: "neurons that fire together, wire together." (That catchy phrase actually came later, from the neuroscientist Carla Shatz.)

The idea is that connections between neurons strengthen when the neurons are active at the same time. This is Hebbian learning, and it was the first plausible learning rule for artificial neural networks.

A tiny demonstration:

import numpy as np
# Learn an association between two patterns using Hebb's rule:
# weight change = learning_rate * input * output
patterns = [(np.array([1, 0, 1]), np.array([1])), # input pattern -> "fire"
(np.array([0, 1, 0]), np.array([0])), # input pattern -> "silent"
]
w = np.zeros(3)
lr = 1.0
for x, y in patterns:
    w += lr * x * y # strengthen weights where input and output are both active
print("w