One the most important exchanges in this session was a challenge from Eric Zimmerman during Q&A as to whether foregrounding assessment hampers the cultural expression of the project. “How would you assess Maus?” Several in the audience applaused.
Shelly Pasnik (EDC): If assessment is about what we know, we need to be more sophisticated about describing what we know.
Karin Hillhouse (Ashoka) gave the example of Wired and the potential for changing hearts and minds. If Wired had been tested and focus-grouped it would never had been on the air.
Fran C. Blumberg (Fordham) gave a very focused presentation on nuts and bolts of evaluation for games. Will playing games induce attitude change? Inducing such change is very difficult — especially if message is inconsistent with long-ingrained values/attitudes (Eagly & Chaiken, 1993, Petty & Cacioppo, 1986). Interest do go up from the games (Gee, Squire)… but what are they actually learning — this is still in investigation.
What can be measured: 3×3 grid:
Stage: before / during / after
Effect: Motivation to elaborate content of desired social change / knowledge acquisition, comprehension / attitude change
DURING
Motivation: Enjoyment promotes greater attention to social message
Knowledge: Narrative facilitates comprehension; multi-player game promotes communication about the content
AFTER (Transfer)
Motivation: Enjoyment promotes repeated play. Enjoyment prompts greater thinking about the game between play. Enjoyment prompts talking about social messages with others.
Knowledge: Immersive nature of game enhances likelihood of transfer to real world.
VEHICLES FOR ASSESSMENT
- Play diaries (blogs)
- Focus groups
- Surveys on frequency of play, frequency of behaviors initiated as a result of game play
Recall of game information
Shelly Pasnik presented case study evaluations of Global Kids games (Ayiti, ICED). Perspective-taking in ICED changed the way players thought about the issue: identifying with, rather than distant. Ethical scenarios strengthened students’ written feedback.