Grey Walter’s Anticipatory Tortoises
Grey Walter and the Ratio Club
The British physiologist William Grey Walter (1910–1977) was an early member of the interdisciplinary Ratio Club. This was a small dining club that met several times a year from 1949 to 1955, with a nostalgic final meeting in 1958, at London’s National Hospital for Neurological Diseases. The founder-secretary was the neurosurgeon John Bates, who had worked (alongside the psychologist Kenneth Craik) on servomechanisms for gun turrets during the war.
The club was a pioneering source of ideas in what Norbert Wiener had recently dubbed ‘cybernetics’.1 Indeed, Bates’ archive shows that the letter inviting membership spoke of ‘people who had Wiener’s ideas before Wiener’s book appeared’.2 In fact, its founders had considered calling it the Craik Club, in memory of Craik’s work—not least, his stress on ‘synthetic’ models of psychological theories.3 In short, the club was the nucleus of a thriving British tradition of cybernetics, started independently of the transatlantic version.
The Ratio members—about twenty at any given time—were a very carefully chosen group. Several of them had been involved in wartime signals research or intelligence work at Bletchley Park, where Alan Turing had used primitive computers to decipher the Nazis’ Enigma code.4 They were drawn from a wide range of disciplines: clinical psychiatry and neurology, physiology, neuroanatomy, mathematics/statistics, physics, astrophysics, and the new areas of control engineering and computer science.5
The aim was to discuss novel ideas: their own, and those of guests—such as Warren McCulloch. Indeed, McCulloch—the prime author, a few years earlier, of what became the seminal paper in cognitive science (McCulloch and Pitts 1943)—was their very first speaker in December 1949. (Bates and Donald MacKay, who’d hatched the idea of the club on a shared train journey after visiting Grey Walter, knew that McCulloch was due to visit England and timed the first meeting accordingly.) Turing himself gave a guest talk on Educating a Digital Computer exactly a year later, and soon became a member. (His other talk to the club was on morphogenesis.) Professors were barred, to protect the openness of speculative discussion. So the imaginative anatomist J. Z. Young (who’d discovered the squid’s giant neurones, and later suggested the ‘selective’ account of learning)6 couldn’t join the club, but gave a talk as a guest.
The club’s archives contain a list of thirty possible discussion topics drawn up by Ashby (Owen Holland p.c.). Virtually all of these are still current. What’s more, if one ignores the details, they can’t be better answered now than they could in those days. These wide-ranging meetings were enormously influential, making intellectual waves that are still spreading in various areas of cognitive science. The neurophysiologist Horace Barlow (p.c.) now sees them as crucial for his own intellectual development, in leading him to think about the nervous system in terms of information theory.7 And Giles Brindley, another important neuroscientist,8 who was brought along as a guest by Barlow before joining for a short time, also remembers them as hugely exciting occasions.9 Our specific interest here, however, is in the machines built by one member of the Ratio Club: Grey Walter’s tortoises. These were intended to model general aspects of purpose and learning. Considered as physical objects, they were intriguing gadgets that attracted enormous publicity (frowned on by some Ratio members) in the newspapers. Their theoretical interest, however, was significant.
More ‘interest’, perhaps, than immediate influence. With hindsight we can now see how hugely insightful they were. But that 20-20 vision wasn’t available to their contemporaries. Some people got the point, to be sure. The French cybernetician Pierre de Latil (1953), for instance, regarded them as ‘revolutionary’, and wrote about their wider scientific—and philosophical—implications at length. (His book was soon translated into English by a relation of Grey Walter’s boss, the neurologist Frederick Golla.) However, the primitive state of electronic technology didn’t enable those implications to be explored in practice.
That wouldn’t be possible until the late 1980s, with the development of behaviour-based (situated) robotics and computational neuroethology.10
Robots at the Festival
Grey Walter was the first Director of Physiology (from 1939) at Golla’s newly-founded Burden Neurological Institute, Bristol. He was a highly influential electro-encephalographer. For instance, he discovered the delta and theta rhythms, and designed several pioneering EEG-measuring instruments. In addition, he founded the EEG Society (in 1943), organized the first EEG Congress (1947), and started the EEG Journal (also 1947). With his EEG expert’s hat on, he focussed on the overall effects of large populations of neurones rather than on specific cell connections. But his robots, as we’ll see, used as few ‘neurones’ as possible.
His interest in the EEG dated from his time at Cambridge in the early 1930s, when he worked on muscle contraction with Edgar D. Adrian. Ever since the first paper on EEG (by Hans Berger) in 1929, Adrian was a key pioneer in the field. He discovered body mappings (of the limbs, for instance) in both cerebellar and cerebral cortex. And he predicted that improved brain-monitoring technologies would one day—fifty years later, as it turned out11—enable neuroscientists to study the cerebral changes associated with thinking.12
In his youth, Grey Walter also studied conditioning with a team of Ivan Pavlov’s students visiting from St. Petersburg. Indeed, he met Pavlov briefly. But his prime neurological interest was in the activity of the brain as a whole.
Besides these psycho-physiological skills, he was a skilled speaker and writer in several languages. He was much in demand to talk on both professional and political issues. Initially a communist, he later veered towards anarchism: that is, the rejection of top-down control. But he was so full of ideas that, as his son Nicolas remembers, ‘he found it difficult to produce more sustained work, and both of his two books were actually written by his father from his notes and conversations’.13
From 1949 onwards, Grey Walter built several intriguing cybernetic machines. These were intended to throw light on the behaviour of biological organisms—although he did point out that they could be adapted for use as ‘a better “self-directing missile”’.14 Unlike actual missiles, however, his machines displayed a range of different behaviours.
He’d been inspired, in part, by a wartime conversation with Craik. Craik was then working on scanning and gun aiming, and visited Grey Walter at the Burden Institute to use some of his state-of-the-art electronic equipment. During his visit he suggested that the EEG might be a cortical scanner, affected by sensory stimuli. This idea became influential in neuroscientific circles.15 And it was later modelled by Grey Walter as a rotating photoelectric cell, whose ‘scanning’ stopped when its robot carrier locked onto a light source.
Wiener’s influence on him was less effective. Thus in a letter to Adrian written in June 1947 (after Craik’s early death), Grey Walter said:
We had a visit yesterday from a Professor Wiener, from Boston. I met him over there last winter and found his views somewhat difficult to absorb, but he represents quite a large group in the States These people are thinking on very much the same lines as Kenneth Craik did, but with much less sparkle and humour.16
Wiener himself was more generous—or perhaps just more polite. In a letter thanking Grey Walter for his hospitality during this brief visit, he wrote ‘I got a great deal out of our trip, and am certain that it will be possible to renew our contact at some future date’.17
In particular, Grey Walter sought to model goal seeking and, later, learning. But he did so as economically as he could—in both the financial and the theoretical sense. Not only did he want to save money (the creatures were cobbled together from war-surplus items and bits of old alarm clocks), but he was determined to wield Occam’s razor. That is, he aimed to posit as simple a mechanism as possible to explain apparently complex behaviour.
And simple, here, meant simple. His wheeled robots, or ‘tortoises’, had two valves, two relays, two motors, two condensers, and one sensor (for light or for touch). In effect, then, a Grey Walter tortoise had only two neurones. For, crucially, the tortoises weren’t mere toys but models of (very simple) nervous systems.
Robot toys with simple tropisms were already common at the time, at public exhibitions if not in the toyshops. For instance, a French-made ‘Philidog’ at the 1929 International Radio Exhibition in Paris would follow the light from an electric torch—until it was brought too near to its nose, when it started to bark.18 Ten years later, visitors to the New York World Fair in 1939 were sadly robbed of their chance to be enchanted by another robot dog. It had committed suicide a few days earlier:
[The ‘electrical dog’] was to be sensitive to heat and was to have attacked visitors and bitten their calves, but just before the opening of the exhibition it died, the victim of its own sensitivity. Through an open door it perceived the lights of a passing car and rushed headlong towards it and was run over, despite the efforts of the driver to avoid it.19
Great fun—except perhaps for the dog! But nothing to do with neuroscience. Grey Walter, by contrast, was pioneering biologically-based robotics, an activity taken up many years later by (among others) Michael Arbib, Valentino Braitenberg, Rodney Brooks, Randall Beer, and Barbara Webb.20
One of his tortoises, the Machina speculatrix showed surprisingly llifelike behaviour. ‘Lifelike’ rather than (human) mindlike—the Latin word meant exploration, not speculation. But Grey Walter clearly had his sights on psychology as well as physiology. This was the first step in a research programme aimed at building a model having:
these or some measure of these attributes: exploration, curiosity, free-will in the sense of unpredictability, goal-seeking, self-regulation, avoidance of dilemmas, foresight, memory, learning, forgetting, association of ideas, form recognition, and the elements of social accommodation.21
‘Avoidance of dilemmas and ‘free-will’ were supposedly modelled by the tortoise’s ability to choose between two equally attractive light sources.22 Unlike Buridan’s ass, forever poised between two identical bundles of hay, the tortoise would unknowingly exploit its scanning mechanism to notice, and so to follow, one light before the other. ‘Learning’, ‘association of ideas’, and ‘social accommodation’ came later (see below). Meanwhile, the Latin tag being too much of a mouthful, this early tortoise was quickly named ELSIE (from Electro-mechanical robot, Light-Sensitive with Internal and External stability). The prototype, which was very similar, was dubbed ELMER: ELectro-MEchanical Robot.
ELSIE soon became something of a celebrity. Much as Jacques de Vaucanson’s flute player had delighted visitors to London’s Haymarket two hundred years before,23 so ELSIE amazed visitors to the Festival of Britain held—only a couple of miles away—in 1951. A few years later, it caused great amusement at a meeting of Charles’ Babbage’s brainchild, the British Association for the Advancement of Science. For it displayed an unseemly fascination with women’s legs, presumably because of the light reflected from their nylon stockings.24 (Much later, it was resurrected at the Science Museum for the Millennium Exhibition, and for the centenary of the British Psychological Society in 2001).
Members of the public unable to reach the Festival site were soon able to read about ELSIE in a pot-boiler entitled The Robots Are Amongst Us.25 A more serious account—but including a photograph of ELSIE and her infant human ‘brother’ in Grey Walter’s living room—had already come off the press in France, and appeared in English in 1956.26 Grey Walter himself became something of a celebrity too. He was often invited to speak by the BBC, sometimes appearing as a panellist on the popular radio programme The Brains Trust.
Of Wheels and Whiskers
The Festival robot ELSIE explored its environment. It used a scanning photelectric cell (coupled to the steering wheels) to seek light and to guide the wheels towards the illumination. If the light wasn’t too bright, the tortoise would stay in front of it, oscillating very slightly to left and right. But a strong light would cause the creature to continue scanning. In that case, its attention—and its movement—might be attracted by another, perhaps a weaker, light. As Grey Walter put it (in deliberately biological language), the tortoise showed a positive tropism to moderate light, but negative tropisms with respect to both strong lights and darkness.
Besides these basic tropisms, Grey Walter’s robot displayed simple forms of approach and avoidance. Its (slightly moveable) ‘shell’ acted as a 3-dimensional whisker, or pressure sensor: it closed an electrical circuit whenever it encountered a mechanical obstacle, causing the creature to back away from walls, furniture, or people’s fingers. Since the robot also carried a pilot light, it would approach its own image in a mirror, or another light-bearing tortoise. On touching the mirror, or the mate, it would automatically move away—only to be drawn back again by the other’s pilot light. The mechanical minuet that resulted was, for a while, fascinating to watch.
Another cybernetic device designed (in 1950) by Grey Walter was an electrical learning circuit named CORA (‘Conditioned Reflex Analogue’), now kept in London’s Science Museum. Largely ‘cannibalized’ from ELSIE, this functioned only as a static box on the workbench. It wasn’t incorporated by Grey Walter himself into a mobile tortoise, although there’s some evidence that he’d intended to do so.
That may explain why ‘in spite of his efforts, CORA had only a small fraction of the impact of the tortoises, and made little lasting impression’.27 This shouldn’t have mattered, at least outside the exhibition halls. For, quite apart from its theoretical interest, Grey Walter did connect it to the circuitry of a moving tortoise, so presumably was able to demonstrate learning in action. But the rhetorical effect was less dramatic, and even the scientists failed to see the point.
The ‘point’, here, was Pavlovian. CORA was based in neurophysiology, being a development of an earlier circuit named NERISSA (Nerve Excitation, Inhibition, and Synaptic Analogue). It was intended as a model of the sort of conditioning that Pavlov28 had reported in his bell-salivating dogs—and which had fascinated Grey Walter in his Cambridge days.
If CORA was repeatedly presented with two stimuli in quick succession, only the second of which ‘naturally’ caused a particular response, it would eventually produce that response even when the first stimulus occurred alone. Like Pavlov’s dogs, CORA needed occasional reinforcement (wherein the first stimulus is again followed by the second) in order to maintain the conditioning. It was based on a probabilistic theory, and may have owed something to Grey Walter’s fellow Ratio member, Albert Uttley.29 In short, CORA reflected current views on neural communication and learning—and it was described by Grey Walter in his account of the living brain.30
Grey Walter pointed out that CORA could be combined with machina speculatrix to produce a robot capable of learning: machina docilis. A tortoise equipped with CORA and an auditory sensor would learn to approach at the call of a whistle, if the whistle was repeatedly blown just before a flash of light. Similarly, it would learn to move away from its current position when ‘whistled’, if the whistle had often been blown just before it touched an obstacle. One follower of Grey Walter built tortoises, called machina reproducatrix, sensitive to various combinations of lamp, flute, and whistle.31
Significantly, Grey Walter noted that his model of associative learning could ‘respond to a part of the significant association as if the whole were present’.32 This, he said, was ‘essentially the same process’ as pattern recognition—what McCulloch33 had called knowledge of universals. He didn’t say, because he couldn’t know, that part-to-whole generalization would be hailed thirty years later as one of the triumphs of parallel distributed processing.34 Grey Walter’s last reported device, built in about 1953 and shown to the Ratio Club in 1955, was called IRMA: Innate Releasing Mechanism Analogue.35 As the name implies, this was designed to model the ethologists’ notion of an IRM: an innate propensity to respond in a specific way to a specific stimulus. In developing IRMA, Grey Walter was especially interested in stimuli originating in the action of some other robot, so that the activity of two (or more) creatures could be coordinated in adaptive ways. Again, this robotic research would be largely forgotten, only to be taken up several decades later.36
If IRMA was the last device to be built by Grey Walter, it wasn’t the last to be envisaged by him. Near the end of his working life (his research ceased when he was seriously injured in 1970), he remarked on ‘a new era’ made possible by transistor technology:
We are now envisaging the construction of a creature which instead of looking as the original did, like a rather large and clumsy tortoise, resembles more closely a small eager, active and rather intelligent beetle.
There seems to be no limit to which this miniaturisation could go. Already designers are thinking in terms of circuits in which the actual scale of the active elements will not be much larger, perhaps even smaller, than the nerve cells of the living brain itself. This opens a truly fantastic vista of exploration and high adventure 37
Grey Walter’s intriguing tortoises, despite their valve technology and clumsiness, were early versions of what would much later be called Vehicles,38 autonomous agents, situated robots, or animats. They illustrated the emergence of relatively complex motor behaviour—analogous to positive and negative tropisms, goal seeking, perception, learning, and even sociability—out of simple responses guided and stabilized by negative feedback.
The ‘tropisms’ of machina speculatrix, for instance, emerged from a few core rules linking the speeds of the two motors to the level of illumination. In the dark, the drive motor would run at half-speed and the steering motor at full speed. In a moderate light, the drive motor would run at full speed while the steering motor was switched off. And in strong illumination, the drive motor and steering motor would run at full and half speeds, respectively. These simple mechanisms gave rise to a wide range of observable behaviour.
Even more to the point, in an unpublished manuscript of about 1961, Grey Walter described a complete behaviour—finding the way past an obstacle to reach a light source—as being achieved by four reflex ‘behaviour patterns’, some of which were ‘prepotent’ over others.39 The four basic patterns envisaged were exploration, positive and negative phototropisms, and obstacle avoidance. His analysis, in this case, was equivalent to that used in the ‘subsumption architecture’ of modern behaviour-based robotics.40
The Perils of Popularity
During Grey Walter’s lifetime, his tortoises—like Vaucanson’s flute player, which in fact had also been theoretically motivated41—were commonly dismissed by professional scientists as mere robotic ‘toys’. That remained true for nearly thirty years. During all that time, the general verdict was that they were superficially engaging gizmos, of little scientific interest.
This largely negative reception was due partly to the vulgar publicity they’d attracted in the mass media around 1950. The brou-ha-ha surrounding the tortoises put off even some of Grey Walter’s fellow Ratio members, who were better placed than anyone to appreciate their significance.
He wasn’t the last to endure the counterproductive effects of seemingly favourable publicity. The connectionist psychologist Frank Rosenblatt would do so too, only a few years later, when his media-applauded work on ‘perceptrons’42 was savagely criticized by two devotees of symbolic AI.43 And symbolic AI, in turn, would suffer similarly in the early 1970s.44
But excessive media attention wasn’t the only obstacle. As remarked above, Grey Walter himself never provided an extended account of his tortoises’ theoretical implications. Having his father draft his books from his notes and conversations simply wasn’t enough to engage his peers’ respect. Were it not for the end-of-century work in situated robotics and computational ethology, his light in cognitive science today would be very much dimmer than it is.
In fact, his anticipatory work is now sometimes praised as
the pioneering effort in ‘Real Artificial Life’.45 One might quibble
about the laudatory definite article. For his fellow Ratio-member William Ross
Ashby, inventor of the—much less entertaining—Homeostat machine, arguably
has an equal right to the accolade. That, however, is a different story.46 What’s not deniable is that Grey Walter’s
engaging little tortoises had a serious scientific purpose that’s widely
Images appear courtesy of University of the West of England, Bristol
Adrian, E. D. (1936), ‘The Electrical Activity of the Cortex’,
Proceedings of the Royal Society of Medicine, 29: 197-200.
Angyan, A. J. (1959), ‘Machina Reproducatrix’, in Blake and Uttley
1959, vol. 2: 933-944.
Barlow, H. B. (1959), ‘Sensory Mechanisms, the Reduction of Redundancy, and Intelligence’, in D. V. Blake and A. M. Uttley (eds.),
The Mechanization of Thought Processes, Proceedings of a Symposium held at the National Physical Laboratory on 24-27 November 1958 (London: Her Majesty’s Stationery Office), vol. 2: 535-539.
Boden, M. A. (2006), Mind as Machine: A History of Cognitive Science
(Oxford: Oxford University Press).
Braitenberg, V. (1984), Vehicles: Essays in Synthetic Psychology
(Cambridge, Mass.: MIT Press).
Brooks, R. A. (1986), ‘A Robust Layered Control System for a Mobile Robot’, Institute of Electronics and Electrical Engineers Journal of Robotics and Automation, 2: 14-23.
Brooks, R. A. (1991a), ‘Intelligence Without Representation’,
Artificial Intelligence, 47: 139-159. (First published as an MIT AI-Memo in 1987.)
Brooks, R. A. (1991b), ‘Intelligence Without Reason’, Proceedings of the Twelfth International Joint Conference on Artificial Intelligence, Sydney, 1-27.
Craik, K. J. W. (1943), The Nature of Explanation (Cambridge: Cambridge University Press).
De Latil, P. (1953), La Pensee Artificielle (Paris: Gallimard), 1953. (Trans. Y. M. Golla as Thinking by Machine: A Study of Cybernetics. (London: Sidgwick and Jackson), 1956.)
Dreyfus, H. L. (1972), What Computers Can’t Do: A Critique of Artificial Reason (New York: Harper & Row).
Grey Walter, W. (1950a), ‘An Electro-Mechanical Animal’, Discovery,
Grey Walter, W. (1950b), ‘An Imitation of Life’, Scientific American,
182 (5): 42-45.
Grey Walter, W. (1953), The Living Brain (London: Duckworth).
Grey Walter, W. (1956), ‘The Imitation of Mentality’, Proceedings of the Royal Institution, 36: 365-9.
Grey Walter, W. (c. 1968), ‘The Future of Machina Speculatrix’. Unpublished ms., in the Burden Neurological Institute archives.
Hayward, R. (2001), ‘The Tortoise and the Love-Machine: Grey
Walter and the Politics of Electro-Encephalography’, in C. Borck and M. Hagner (eds.), Mindful Practices: On the Neurosciences in the Twentieth Century. Special volume of Science in Context,
Holland, O. (1997), ‘Grey Walter: The Pioneer of Real Artificial Life’,
in C. G. Langton and K. Shimohara (eds.), Artificial Life V: Proceedings of the Fifth International Workshop on the Synthesis and Simulation of Living Systems (Cambridge, Mass: MIT Press), 34-41.
Holland, O. (2002), ‘Grey Walter: The Imitator of Life’, in R. Damper and D. Cliff (eds.), Biologically-Inspired Robotics: The Legacy of W. Grey Walter, Proceedings of the EPSRC/BBSRC International Workshop WGW-02, 32-48.
Revised version: ‘Exploration and High Adventure: The Legacy of Grey Walter’, Philosophical Transactions of the Royal Society of London A, 361 (2003): 2085-2121. (Special issue on ‘Biologically Inspired Robotics’.)
Minsky, M. L., and Papert, S. A. (1969), Perceptrons: An Introduction to Computational Geometry (Cambridge, Mass.: MIT Press).
Pavlov, I. P. (1897/1902), The Work of the Digestive Glands: Lectures by J. P. Pawlow, trans. W. H. Thompson (London: Charles Griffin), 1902. First published in Russian in 1897.
Pavlov, I. P. (1923/1927), Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex, trans. G. V. Anrep (London: Humphrey Milford), 1927. First published in Russian in 1923.
Rosenblatt, F. (1958), ‘The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain’, Psychological Review, 65: 386-408.
Rosenblatt, F. (1962), Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms (Washington, DC: Spartan).
Stehl, R. (1955), The Robots Are Amongst Us (London: Arco).
Uttley, A. M. (1954), ‘The Classification of Signals in the Nervous System’, EEG and Clinical Neurophysiology, 6: 479-494.
Uttley, A. M. (1956), ‘Conditional Probability Machines and Conditioned Reflexes’, in C. E. Shannon and J. McCarthy (eds.),
Automata Studies (Princeton: Princeton University Press), 253-275.
Vaucanson, J. de (1738/1742), An Account of the Mechanism of an Automaton or Image Playing on the German Flute. Trans., with a letter to the Abbe de Fontaine, J. T. Desaguliers (London: Parker). French original, Paris: Guerin, 1738. Facsimile reprints of both from Buren: Fritz Knuf, 1979.
Walter, N. (1990), ‘Personal Notes for the Dictionary of National Biography’.
(Copy given to MAB by Owen Holland.)
Weizenbaum, J. (1976), Computer Power and Human Reason: From Judgment to Calculation (San Francisco: Freeman).
Wiener, N. (1947), Letter to Grey Walter, 12th June. (Copy given to MAB by
Wiener, N. (1948), Cybernetics: or Control and Communication in the Animal and the Machine (Cambridge, Mass.: MIT Press). Printed offset from the 1948 French
edition (Paris: Hermann et Cie).
1 Wiener 1948
2 P. Husbands, p.c.
3 Craik 1943; Boden 2006: 4.vi
4 Boden 2006: 3.v.d
5 For a discussion of the neuroscientists and psychologists involved, see Boden 2006: 12.ii.c.
6 Boden 2006: 2.viii.e and 13.ix.d
7 Barlow 1959; Boden 2006: 12.ii
8 Boden 2006: 14.iii.c
9 P. Husbands, p.c.
10 see Boden 2006: 14.vii and 15.vii-viii.a
11 Boden 2006: 14.x.c
12 Adrian 1936: 199
13 N. Walter 1990
14 Grey Walter 1950a,b, 1953; see also Holland 1997, 2002
15 Boden 2006: 13.ii.a
16 Holland 2002: 36
17 Wiener 1947
18 De Latil 1953: 240f
19 De Latil 1953: 241
20 Boden 2006: 13.iii.b, 14.vii.c, and 15.vii
21 Grey Walter 1953: 120f
22 Holland 2002: 44f.
23 Boden 2006: 2.iv.a
24 Hayward 2001
25 Stehl 1955
26 De Latil 1953
27 Holland 2002: 46
28 Pavlov 1897/1902, 1923/1927
29 Uttley 1954, 1956; Boden 2006: 12.ii.c and 13.ii.b
30 Grey Walter 1953: 203-7
31 Angyan 1959
32 Grey Walter 1956: 368
33 Pitts and McCulloch 1947
34 Boden 2006: 12.vi
35 Grey Walter 1956: 367f.
36 Boden 2006: 13.iii.e
37 Grey Walter c. 1968: 7
38 Braitenberg 1984
39 Holland, p.c.
40 Brooks 1986, 1991a,b
41 Vaucanson 1738/1742; Boden 2006: 2.iv
42 Rosenblatt 1958, 1962
43 Minsky and Papert 1969; Boden 2006: 12.iii
44 Dreyfus 1972; Weizenbaum 1976; Boden 2006: 11.iv
45 Holland 1997
46 Boden 2006: 4.viii.c-d and 15.xi.a
47 This paper is based on Chapter 4.viii.a-b of my book Mind as Machine: A History of Cognitive Science Oxford: Oxford University Press, 2006)