Conducting Social Experiments Online

Berkman Fellow Jerome Hergueux discusses a platform for conducting social experiments online and present some experiment results during today’s Berkman lunch.

In the most simple experiments, anonymous people play something online for a few minutes, trying to illustrate behavior in the real world. Experimental economics is a useful frame for scientific discourse because it allows reproduction of and adjusting experiments.

He uses an acronym for WEIRD for some of his research, but Firefox crashed as he explained it, so I missed it.

Cyberspace is a social place, so running social experiments there is not such an odd thing.

He illustrates how cyberspace has merged with the real world using a clip from South Park’s episode about World of Warcraft where a dad interrupts a player and gets a string of acronyms and slang about what the son is doing with the game and his team of friends playing along.

Social preferences economics can shed some light on the data from online social experiments.

His current paper is primarily about methodology of these online social experiments.

Is it possible by designing a proper online lab to get consistent results between experiments?

People playing in real life might play faster and with less careful thought than people who play as part of an experiment.

(I’m in Berkman’s overflow space—an alcove in a hall—watching the talk broadcast over the Internet. The camera person is spending more time showing Jerome than his presentation slides, which makes note taking a bit challenging.)

People give up private benefits to achieve a socially efficient goal. Waiting in line, for example, to buy food when you’re really hungry involves some sacrifice. Not running a red light when you’re late is a similar decision.

It sounds like he does a version of the experiment where a participant gets an amount of money, then chooses whether to keep the money or contribute the money to the common good. If other participants contribute to the general pool, everyone gets rewarded. If no one does, people are rewarded individually. People often contribute 10-15% of their money to the common good.

The Dictator game measures someone’s altruism toward strangers. The Bargaining game explores whether people will bargain in a way to close a deal. (My computer is apparently having a bad day because each program I was running just crashed again, including the OS. So I rebooted and missed a few of the games Jerome explained.)

Are online participants less diverse than participants brought to a lab?

Risk Aversion game causes people to choose between amounts of money based on different probability of getting more money. For example, in one scenario, one might be choosing between $20 and $30 with a 50/50 chance. In another, the selection may be between $1 and $50 with a 20/80 chance of getting $50. An audience question explores whether they can measure whether someone’s including risk in their actions or just doing what they would always do. Jerome says he’s not sure, but he doesn’t think it’s a strong factor. Someone else tries to deflect the question by observing Jerome’s work isn’t about the kind/style of experiment, that these games are classic experiments in some fields, and all Jerome is doing is looking at how they function online.

The interface is one they can use in online games and in the lab, removing any potential problems from having different interfaces online and offline. Earnings calculators allow some people to explore what outcomes are possible in an experiment. Exploration before making a decision can be helpful and informative.

Procedures: subjects self-select and register to an experiment. The researchers decide who’ll come to the lab and who will perform the experiment online based on the order in which the people register for the experiment.

They pay subjects in cash or via PayPal transfers, depending on whether they came to a lab or participated online.

The lab is a bunch of cubicles with one computer per cubicle. The participants all see each other at the beginning and are debriefed together. Then, they are assigned computers at random, sit down, and begin their work. I think he said they don’t necessarily know against/with whom they’re playing. The lab is in France.

It’s hard to hear people asking questions because there aren’t microphones for the audience. Jerome does not repeat the questions. Questions are likely to be missing below, but I can try to summarize his answers.

A: It seems like most people read the instructions, but sometimes we can tell not everyone does.

A: People online seem to act as if they’re most social than what they appear to be while online. Online subjects seem to be less risk averse, too.

A: People as if they had higher trust, more consideration of social norms. But the study didn’t cover these factors, so we didn’t examine them.

A: Because the laboratory has become more recognized in recent years, they’ve gained a large pool of potential test subjects.

A: Jerome explained a little bit of the interface.

A: They’re all WEIRD subjects.

Be Sociable, Share!

Leave a Reply

You must be logged in to post a comment.