The Driving Forces Behind Experimental Economics

Since its emergence in the 1960s, experimental economics has been at the forefront of investigating the limits of homo economicus and the role of social norms in market and non-market settings. My paper analyzes key motives and pivotal experiences — which I label driving forces —that turned non-experimental economists into early pioneers of experimental economics.

The four driving forces are: 1) integrity – the expansion of the permissible type of data in economics by introducing experimental data and advocating its advantages; 2) rigorousness – the personal collection of data under controlled conditions; 3) the virtuous circle – the realization that experimental research is most potent when it goes in tandem with economic theory; and 4) symmetry – placing experimental data on a par with economic theory.

Each driving force is introduced through the story of an economist who eventually, in no small part through the described experience, embarked on a path of experimental life. These four driving forces are generalizations of the motives and memories that I encountered in interviewing over eighty experimental economists.

James Cox received standard economics training at Harvard in the second half of the 1960s. He experienced many imperfections of field data which could be remedied once he learned from Vernon Smith about the possibility of producing experimental data. This attractiveness of experimental data lies at heart of the integrity driving force.

The early experiences of Charles Plott when studying group decision making and preference reversals share a common element — his desire to produce the experimental data himself — since he did not trust what others observed. His case is used to highlight the issue of how and who produces experimental data — the driving force of rigorousness.

The virtuous circle driving force is introduced through Reinhard Selten’s story of how the seminal concept of sub-game perfect equilibrium came about through the interaction of experiments and theoretical considerations which eventually led to his Nobel Prize in 1994.

Finally, John Ledyard’s observation of Vernon Smith’s ability to modify an impromptu experiment showed him that data created in controlled experimental conditions cannot be dismissed as irrelevant for his theory of free-riding and provision of public goods. Rather he needed to modify his theoretical approach to account for the new, replicable empirical evidence. Ledyard’s case deals with the data-theory symmetry driving force.

NB—here ends the short abstract and the extended abstract continues.

The four driving forces can be split into two groups. Two relate primarily to theory and two to data. The symmetry driving force is about seeing that experimental data matter for theory, but not seeing how this impact goes on. This is shown in the virtuous circle of Selten where theory is not merely evaluated, but is a new element to be created. John Ledyard’s example is particularly suitable for demonstrating the driving force of symmetry. The interaction between Smith and Ledyard comprises the accumulated knowledge in Smith’s experience, by then two decades long, with the experimental method.

The other two driving forces relate to data. The rigorousness driving force deals with the presence of the experimenter and her direct involvement in data production. Being rigorous is also about questioning the beliefs of oneself and others. But it is wider than proper scientific conduct; it is about control. The integrity driving force illustrated by James Cox’s experience with non-experimental data deals with the quality of data. Taking experimental data seriously meant discarding the possibility that it is only the data that are flawed and not the theory to which they are related.

All four driving forces present different ways of thinking about the triangle ‘scientist-theory-data’. The replicability of and amenability of experimental design countered objections about the reliability of experimental data and speeded up the interaction between theory/model building and empirical evidence. The four examples of conversion show how experimentation allowed economists to rethink their understanding of what it means to be a good economist not simply in one’s abstract commitment to epistemic tenets, but, more importantly, in practice as well.

These pioneers of experimental economics are theorists by training, more specifically mathematical economists, which was not typical in the 1950s and 1960s when many of them graduated. This may be seen to confirm the received idea that the early experiments in economics were predominantly theory driven; intended to test theory. But a close reading of four driving force cases reveals the relation between theory and experiment in a different light. These driving forces are equally about theory and data, and in particular about elevating data to the focus of an economist’s work. Selten would not have reached the idea of sub-game perfect equilibrium had it not been for the experiments he performed. Experiments allowed Plott to discriminate between theories. Giving a theory its “best shot” is not about the theory, but about the data, the quality of the data. James Friedman left experimentation and turned into a theorist because of a lack of connection between his experiments and the theory that he wanted to test.

For experimentalists the question of what comes first — theory or data — became far less important once they experienced the four driving forces. The question of what comes next became far more important: how to modify theory, how to better control experiments, how to design relevant experiments, etc. In effect, experiments became question generating machines for economics research. Experiments transcended the theory or data that they were based on. The four driving forces helped to start the experimental engine.

The four case studies clearly show how interconnected and closely related the American group of early experimentalists was. Smith, Ledyard, and Plott are all connected via Purdue. There are other Purdue graduates who became early experimentalists — John Kagel and Raymond Battalio. When they came to Purdue as graduate students at the end of the 1960s, Smith had already left and there was no institutional memory of the experiments he had performed early in the decade. Even during Ledyard’s studies at Purdue in the mid-1960s he encountered no experiments at all. James Friedman — who wrote perhaps the first economics dissertation using economic experiments (Yale, 1963)— Vernon Smith and Reinhard Selten are all connected through Austin Hoggatt, but whereas Friedman’s experiences with Hoggatt made him more or less give up on experiments and move to game theory, Smith and Selten were inspired by Hoggatt’s laboratory and once the opportunity to build their own laboratory presented itself, they did not hesitate.

Both Purdue and Caltech were important institutes for the emergence of experiments in economics in the 1960s and 1970s, respectively. In contrast to traditional economics departments, both fostered research that went beyond the traditional boundaries of well-defined economic subfields at the time.

The individual trajectories of the early experimentalists traversed academic space-time, intersected and from individual researchers built themselves into a community of economists who through their repeated interaction and continuous engagement with the experimental method were gradually grasping the full extent of the turn’s implications.  While the driving forces operated at a personal, individual level, without a community of like-minded scholars who actively practiced the experimental method, notably in close relationship to economic theory, a continuous style of experimentation could not emerge and the experimental engine would have to halt, as it did in the US in the 1960s. Despite a number of researchers such as James Friedman, Lester Lave, Trenery Dolbear, Austin Hoggatt, Martin Shubik, William Starbuck and Roger Sherman pursuing experimental work, Smith’s attempts did not succeed, unlike those in Frankfurt where Selten and Sauermann managed to foster a community of experimentalists.

The driving forces resulted in an expansion of permissible empirical evidence by including data created in controlled experimental and later laboratory conditions under the personal supervision of the (experimental) economist, who did not have to rely on data gathered and processed externally, for example by statistical offices. This added an element of personal responsibility and even ownership of the experimental data that precipitated a serious view of the data and discarded the possibility that the data alone are flawed and not the theory to which they are related. The establishment of dedicated spaces for experimental research, economics laboratories, further nurtured the ability to replicate and amend experimental design. These features of experimental research countered objections about the reliability of experimental data, thereby strengthening the symmetry between theory and data as well as speeding up the interaction between theory/model building and empirical evidence.

Both theory and experimental data were envisioned to be equal parts of a virtuous circle. Economic theory had several functions in the experimental turn. First, the testing of theory as a starting point led to a style of continuous experimentation that encouraged the emergence of the identity of experimental economists. Second, in the 1980s various theories were examined experimentally for the first time in a process which can be aptly described as experimental imperialism. Third, and in one sense a response to questions of the relevance and novelty of experimental research, experimentalists aimed at the most recent and advanced theory with the goal of entering into conversation with theorists. Together with what the experimentalists perceived as the equal standing of experimental data and theory, this was needed to start up a virtuous circle of theory-data development. Last, but quite as important, the fact that Smith, Plott, Battalio and Kagel observed supporting evidence for economic theory - in certain experimental situations and often under much weaker assumptions than traditional theory presupposes — was a stimulus that helped them to persevere in their experimental research.

Thus far my brief discussion of the experimental turn has focused only on the individual experiences of particular early experimentalists. However, the impact of the driving forces was not felt on this individual level only. As soon as the experimentalists entered into a dialogue with theory and tried to convince other economists who had no direct experience with experiments about the merits of the experimental method, the novel type of data, and the intriguing results obtained through them, the experimental turn reverberated on both the local and experimental community levels and eventually percolated through the whole discipline of economics. The local level was constituted by the immediate environment of the emerging experimentalists — their colleagues, students, and co-authors — and soon centered on the experimental economics laboratories. As more economists were going through their personal transformations and became more geographically dispersed, the sense of a community and later an identity emerged. This also speeded up the stabilization of experimental practices and methods that in turn launched a continuous engagement with experimentation.

The communal level, in contrast to the previous two levels, can be characterized as a process of the passive reception of experimental economics. While active reception defined the community of experimental economists and influenced its internal growth, passive reception defined for the rest of the profession the boundaries, distance and image of experimental economics. The boundaries were negotiated on two fronts: publications in general economics journals and funding. Success in passive reception enabled experimental economics to join the economic mainstream, i.e. the experimental method became an accepted, if not widely used, method in economics.

In this respect, the second half of the 1980s is the crucial and hitherto unacknowledged period in the history of economics. In the course of my investigation I demonstrate that during this period experimentation stabilized as a new knowledge generating method and gained acceptance; an experimental community emerged and became institutionalized through the Economics Science Association; the computerized economic laboratory developed and rapidly proliferated; publications in leading journals soared; sources of funding were continuously secured; successful applied experimental projects were carried out; the experimental economists experienced a major internal dispute that consolidated their credibility in the eyes of the rest of the profession; and, not least in the line, separation and methodological tensions with the emergent behavioral economists were set in motion. In short, the experimental turn was complete on all the various levels — individual, local, communal and discipline wide — that it operated on.