Edited by Michael H. Birnbaum

San Diego: Academic Press, 2000

In the last few years, it has become possible to conduct meaningful behavioral research via the Internet. As of June 17, 1998, there were 35 Internet experiments and surveys in the American Psychological Society list of Psychological Research on the Net, maintained by John Krantz, URL [http://psych.hanover.edu/APS/exponnet.html]. By May 11, 1999, this figure had grown to 65, suggesting a growth rate of about 100% per year. I expect that this book and others like it will accelerate this growth. Of the experiments listed in the APS Web site, there were 24 in social psychology, 13 in cognitive, 8 in sensation/perception, 5 in health psychology, 4 in developmental, 3 in clinical, 3 in personality and industrial-organizational, 2 in biological, 2 in emotions, and one in general psychology. Although this list does not include all experiments, it gives a proportional estimate that indicates the growth of research conducted via the Web.

Early "pioneers" of Internet research soon learned that it was not only possible to conduct research this way, but that it was also feasible to collect large samples of high quality data in a short period of time. This book is intended for psychologists who are interested in learning from the experiences of those who have been engaged in this type of research. The book includes a great deal of good advice from those who have learned by experience.

In reading the book, you should follow suggested links on your computer, which should be your window to the Web and your study companion. This Web page includes the most important links from the book, which will save you the trouble of typing in the URLs (the addresses of the sites on the Web). One advantage of Web-based research is the ease with which another can see exactly what the participants experienced and also learn how the experimenter carried it out. To save space (and trees), the authors have made a great deal of information available to you electronically via the Internet. Terms used in the book unique to this type of research (e.g., HTTP, HTML, FTP, etc.) are defined in a glossary at the end of the book.

The book has three sections. The first deals with general questions such as do the results of Web experiments agree with those of laboratory experiments? Who are the people who volunteer to participate via the Internet? What were the developments that led to the first Web studies and what did the early Web researchers experience? What are the methodological considerations in doing research by "remote control?" The second section considers studies of individual differences and cultural differences. Because the Internet provides a means to reach large and diverse samples, it seems ideally suited for these purposes. The third section covers advanced computer techniques that allow for greater control of Internet experiments. These include the dynamic creation and display of graphics, randomization, and timing in experiments such as those in cognitive experimental psychology. In addition, methods for scoring and feedback on surveys or tests, tracking of participants, security, and saving of data on the server are discussed.

Michael H. Birnbaum

This chapter provides a preview of several of the issues and questions that are themes of this book: (1) How to conduct Internet research; (2) recruitment of special populations; (3) how do Internet samples differ demographically from college subject pools? (4) How do results of Internet and lab experiments compare? The chapter also describes a program of research in decision-making. The appendix of Chapter 1 illustrates how experiments that might be done by paper-and-pencil methods can be easily conducted using the technique of forms available in HTML (Hyper Text Markup Language, see glossary). The chapter reviews experiments on decision making that were done with three samples, one conducted with undergraduates in the laboratory, one recruited from experts in judgment and decision making, and another recruited by Web sites that advertise games and drawings with prizes. The decision-making experiments of this chapter can be viewed at URL [http://psych.fullerton.edu/mbirnbaum/exp2a.htm] and URL [http://psych.fullerton.edu/mbirnbaum/exp2b.htm].

John H. Krantz & Reeshad Dalal

Krantz and Dalal review studies that assess the validity of Web experiments. Whereas laboratory studies typically use a small, homogeneous sample tested in controlled conditions, the Internet study typically uses a large, heterogeneous sample tested in less well-controlled conditions. Krantz and Dalal define validity in terms of the correspondence of results between experiments conducted via the Web and those done in the laboratory. The trend emerging from the early research on this problem is that Internet studies yield the same conclusions as studies done in the lab.

Jochen Musch & Ulf-Dietrich Reips

Musch and Reips review the brief history of experimentation on the WWW. In 1995, several developments (Java, JavaScript, HTML 2 with forms) combined to facilitate Web research. Musch and Reips surveyed those who conducted the first Web experiments and report their results in their chapter. Most of those who responded to the survey concluded that their research projects were successful and planned to continue research via the Web.

Ulf-Dietrich Reips

Reips reviews methodological issues in Internet research. Reips reverses the definition of validity given by Krantz and Dalal, arguing that one should be more skeptical of traditional laboratory research than of Web-based research, because of certain problems of traditional research. For instance, because large and diverse samples are obtained from the Web, one can separately analyze a research question in each demographic sub-sample, to ensure that conclusions that are found with 19-year-old college students also hold in other demographic groups. The chapter reviews pros and cons of Web research, concluding that advantages of Internet research outweigh the disadvantages. Reips also describes techniques to deal with potential problems, illustrating certain issues with data from his Web Experimental Psychology Lab, URL [http://www.genpsylab.unizh.ch/ http://www.genpsylab.unizh.ch]. This site, first established at Tuebingen, is now in Zurich. The site contains a number of Web experiments, an archive of past experiments, and commentaries on the Web experiment method.

Tom Buchanan

Buchanan reviews the psychometric properties of tests given via the Internet. An important approach to validity is the use of criterion groups. For example, a test of mental illness should distinguish people who are patients in mental institutions from those who work there. Buchanan uses natural criterion groups existing in the form of subscribers to contrasting newsgroups. People who choose to join different groups should differ systematically in specific aspects of their personalities. Illustration of a personality test given by Internet is available in URL [deleted]. From that site, one can also follow links to join the psy-net-research list (for Internet researchers).

Robert D. Bailey, Winona E. Foote, & Barbara Throckmorton

Bailey, Foote, and Throckmorton report a survey of human sexual behavior and attitudes, URL [http://psych.fullerton.edu/throck]. Their chapter considers several difficult issues in sex surveys. Are people honest and unbiased when they answer the questions? Would people be more honest with a computer than they would be with paper and pencil or face-to-face interviews? The chapter compares data obtained by requesting university students to complete the survey in class and those obtained by passive recruitment from the Internet. How do such samples differ, and are these differences correlated with sexual attitudes, knowledge and behavior? Since 1995, over 10,000 people have already responded to a questionnaire of over 400 items.

Donatella Pagani, & Luigi Lombardi

The Internet makes an experiment available to people from all parts of the world. Pagani and Lombardi take advantage of this new opportunity to conduct a cross-cultural examination of perceptions of the expression of surprise depicted in schematic faces. To experience the experiment, visit the Padua on-line laboratory, URL [http://www.psy.unipd.it/personal/laboratorio/surprise/htmltesi/first3.html http://www.psy.unipd.it/personal/laboratorio/surprise/htmltesi/first3.html]. By manipulating features in schematic faces, they are able to change judgments of the degree of surprise. For people of all cultures, a general expansion of the features of the upper face produces higher judgments of degree of surprise. Interestingly, there are differences in judgment that are correlated with the culture or region of the participants. North Americans and North Europeans give very similar judgments, but these differ from Asians, who appear to give relatively more weight to the eyes than the eyebrows. Southern Europeans give results that are intermediate between those of Asians and North Europeans.

John H. Mueller, D. Michele Jacobsen, & Ralf Schwarzer

Mueller, Jacobsen, and Schwarzer ask if experiences in controlling a computer are correlated with greater self-efficacy. Perhaps people who learn that by following a scheme one can control a computer also learn that they can control other aspects of their lives. The Internet seems a good place to recruit people who either have or have not learned to program. The Internet also allows one to collect large samples in which small correlations can be detected. The survey can be experienced by visiting URL [http://www.acs.ucalgary.ca/~mueller/tai-consent.html http://www.acs.ucalgary.ca/~mueller/tai-consent.html].

The third section of the book reviews techniques that allow greater control of an Internet experiment than is afforded by simple HTML. Each technique has certain advantages and potential difficulties compared to other techniques that might accomplish the same goals. These various techniques are discussed in different chapters.

Kenneth O. McGraw, Mark D. Tew, & John E. Williams

McGraw, Tew, and Williams discuss their use of Shockwave and Authorware to develop their on-line psychology laboratory, PsychExps. Shockwave, now known as the Authorware Web Player, is a plug-in that plays experiments created by Authorware. The plug-in is available free from http://www.macromedia.com or via links at http://www.olemiss.edu/PsychExps">http://www.olemiss.edu/PsychExps, the PsychExps site. Authorware is a Macromedia product that McGraw and his colleagues have found to be an excellent tool for developing computer-controlled psychology experiments. They believe that Authorware is easy enough to learn that a person without programming experience can develop sophisticated experiments. They hope that PsychExps will become a collaborative site developed through the mututal efforts of psychology instructors who will share Authorware experiments, and provide data gathering opportunities for students and colleagues. In their chapter, they present this vision and an overview of the technology that makes it possible.

Jonathan Baron & Michael Siepmann

Baron and Siepmann describe how one can use JavaScript to randomize and control Web questionnaires. JavaScript is a scripting language that is distinct from the Java language. JavaScript code can be included as source code in a Web page. Advantages of including the source code in the Web page include the openness of research to the scientific community and the generally fast loading times of such pages. Examples illustrating the techniques described in their chapter can be viewed in URL [http://www.psych.upenn.edu/~baron/examples/].

Gregory Francis, Ian Neath, & Aimee M. Surprenant

The Cognitive Psychology OnLine Laboratory of Purdue is described in the chapter by Francis, Neath, and Surprenant, URL [

Chapter 12. The Server-Side of Psychology Web Experiments

William C. Schmidt

Schmidt discusses ways in which programs on the server can be used to help control Internet experiments. The server is the computer that delivers (or "serves") Web pages to be viewed on the client's (the visitor's) browser. Although some functions such as error checking or question randomization can be carried out using JavaScript, using server programs provides a universal solution guaranteed to work even if JavaScript is not installed on the browser. Other functions such as tracking participants, summarizing group data, password/security control, or the saving of data require server programming. Schmidt has written software that eliminates the need to program when administering Web surveys and tests; it creates both the Web page and the server-side script. This software can be accessed from URL [http://survey.psy.buffalo.edu].

Return to Michael Birnbaum's Home Page