You are viewing a read-only archive of the Blogs.Harvard network. Learn more.

Is reputation obsolete?

Essay by Judith Donath.
Continue the conversation on online reputation with Chris Dellarocas.

In the past, most conversations were ephemeral: spoken words quickly slipped into the past, resurrected only if a listener later repeated them from memory. Today, many discussions and transactions live on indefinitely. Online conversations are often permanently archived and events in the face-to-face world are frequently recorded. We photograph each other at events both significant and mundane, and upload the images to public media-feeds. Records of our travel times, purchases, health conditions, phone calls, and more exist in vast corporate and government databases. Today, I often no longer have to rely on someone else’s account of your past behavior: I can see for myself.

In a world in which all action is recorded, is there still need for reputation information? If I can see the events of the past for myself, is getting other people’s potentially biased and self-serving opinions about it worth anything? Or, has reputation become obsolete?

In some cases, the answer is yes.

For example, when buying something on eBay, if a seller’s reputation is poor I’ll go elsewhere. However, some ratings are falsely low because of retaliation, or inaccurately high due to fear of retaliation. The reported opinions of others are therefore dubious.* What I really want to know are the facts about the seller’s past transactions: Were the items sent on time? Were they in good condition? The ideal would be if UPS had a service where they photographed an item to be shipped, wrapped it themselves, and posted the photo and shipping info; I could then look up a verified record of the seller’s actions (though items whose condition is not readily apparent from a photo – a laptop, for instance – would need a more extensive evaluation). In other words, I’m more interested in history here than in reputation.

Yet even when the facts of an event are clear,interpretations of them can be important. A politician’s publicly broadcast speech is subsequently argued over by journalists, bloggers, and taxicab drivers; different communities interpret the same words and gestures in vastly different ways. In academia, the committee members evaluating a professor for promotion can read his C.V. and publications, but they also rely heavily on letters from colleagues assessing the significance of the candidate’s work.

Reputation is central to community formation and cooperation (Emler 2001; Gluckman 1963; Hardin 2003). Through discussion about others’ actions, people establish and learn about the community’s standards. Reputation is the core of rewards and sanctioning – it amplifies the benefits of behaving well and the costs of misbehavior. If I work with someone who turns out to be lazy and dishonest, by telling my friends about it, they are spared from a similar bad experience. Having access to reputation information is a big benefit of community membership: insiders know who to trust and how to act toward each other, while strangers do not get the benefit of other’s past experiences. Our ability to share reputation information makes society possible (Dunbar 1996).

In light of this, it would seem that the answer to the question “Is reputation obsolete?” is “No”.

Yet reputation is subject to manipulation, for various reasons. People use it to influence opinion to advance their own causes, to maliciously harm someone, or to curry favor by providing entertaining or seemingly confidential material. We need to understand what circumstances make reputation reliable.

Reputation information exchanged within close-knit communities is more reliable, and members learn when assessments are biased. A colleague recently mentioned that she would never trust another recommendation letter from Professor X again – she’d seen too many in which he claimed that different students were “the top scholar I’ve known”. In overzealously promoting the careers of his students, Professor X acquired a poor reputation for inflated praising. Most letter writers temper the desire to over-enthusiastically praise in order to remain credible in the eyes of their peers, realizing that this close-knit community assesses the assessors. Without community ties, reputation is generally less useful. On public rating sites such as eBay, where no community binds the rater and the reader of ratings, there is no check on reliability and the ratings function primarily as a social exchange between the rater and subject (David & Pinch 2006).

So, is reputation obsolete in an increasingly archival world? The answer, it appears, is “sometimes”. When the immediate facts are primary, we should make use of the vast amount of archived material available. But when situations are ambiguous, when there are conflicting versions of events or codes of behavior, and when developing a shared culture is important (Merry 1997), reputation and the communicative, community-building process of creating it is far from obsolete.

Online, new factors affect the balance between reputation and history. One big issue is “portable identity”: if I spend countless hours on a site being a gracious and well-informed companion, shouldn’t I be able to take that personal history and reputation with me to another site? Many would argue yes, that “you own your own words”. (More controversial is the question of whether you *must* take your history with you: people prefer to port only positive pasts.) But porting reputation is a different matter. Your reputation is information about you, but it is not by you. If you own your own words, then your reputation is owned not by you, but by the people who talk about you. Furthermore, it is a subjective judgment made it a specific context that may not translate well into another. History is portable in ways that reputation is not.

An online site can encourage reliance on history by making search easy and by providing visualizations of patterns within its archive. Or it can encourage the use of reputation by providing both public and private communication channels, as well as feedback about the value of the reputation information people have provided. In technologically mediated societies, evaluating the relative merits of history and reputation is especially important, for the habits of such communities are shaped by deliberate design.

*People do use these ratings and they affect price (Resnick et al. 2006) but this is in the absence of better information.

Judith Donath is a Berkman Faculty Fellow and the director of the Sociable Media Group at the MIT Media Lab. Her work focuses on the social side of computing, synthesizing knowledge from fields such as graphic design, urban studies and cognitive science to build innovative interfaces for online communities and virtual identities. She is known internationally for pioneering research in social visualization, interface design, and computer mediated interaction.

References

David, Shay and Trevor John Pinch. 2006. Six Degrees of Reputation: The Use and Abuse of Online Review and Recommendation Systems. First Monday 11, no. 3.

Dunbar, Robin I. M. 1996. Grooming, Gossip, and the Evolution of Language. Cambridge, MA: Harvard University Press.

Emler, Nicholas. 2001. Gossiping. In The New Handbook of Language and Social Psychology, ed. W. P. Robinson and H. Giles:317–338. New York Wiley.

Gluckman, Max. 1963. Gossip and Scandal (Papers in Honor of Melville J. Herskovits). Current Anthropology 4, no. 3: 307-316.

Hardin, Russell. 2003. Gaming trust. In Trust and reciprocity: Interdisciplinary lessons from experimental research, ed. Elinor Ostrom and James Walker:80-101. New York: Russell Sage Foundation.

Merry, Sally Engle. 1997. Rethinking gossip and scandal. In Reputation: Studies in the Voluntary Elicitation of Good Conduct, ed. Daniel B. Klein:47-74. Ann Arbor: University of Michigan Press.
Resnick, P., R. Zeckhauser, J. Swanson, and K. Lockwood. 2006. The value of reputation on eBay: A controlled experiment. Experimental Economics 9, no. 2: 79-101.

Comments 3

  1. Bertil Hatt wrote:

    I’m not sure I would split the world in two: history from one side, reputation on the other. Websites (and any exchange institution since the 10th century is you believe Avner Greif’s Magrhibi Traders) tend to decide and refine signals and enforce sanctions: eBay has to decide on what time period/how many transactions to average the most visible grade, but they could offer a more detailed representation; one can use labels, grades, decide to filter out certain elements and make other hard to find; every platform decides what to record. . . Your history versus reputation axis is really a question of how much does one condense information into simple stats — but many other aspects are also important: how explicit the projection is? PageRank can’t be transparent, by fear of gaming; eBay though they could avoid it, but had to stop sellers from rating each other for that. Can the users decide on what type of information to report, rather then their value — they can use that to boast their appearance, but would most likely specialise in certain behaviours, encourage by the metrics they chose.

    There is a large literature by statisticians on what makes a good measure — and how it can influence policy. I’ll try to find translations if you are interested.

    Posted 17 Oct 2008 at 8:02 pm
  2. Dave Karpf wrote:

    It seems to me the title of this post should be “Is Reputation Tracking Obsolete?” In that case, the answer would be “for the most part, yes.”

    Reputation in its pure form is ephemeral. Reputation management systems seek to establish proxies for reputation assessments. The quality of reputation tracking is going to hinge on three elements, then: (1)relevance of the proxy. How good of an approximation does the online rating mechanism provide? (2)Traffic levels. I’m always entertained by low-traffic blogs that include recommended diary structures and such. Online reputation tracking assumes huge inputs, but given the power law distributions of web traffic, we know that there are only going to be a select few webspaces that obtain that level of traffic. (3)Gaming of the system or lack thereof. And this last one is long-term problematic. Any high-traffic webspace is going to represent valuable online real estate. The perverse incentives are there for actors to try to figure out the rules of the game and then innovate ways to get around them. We haven’t seen a lot of innovations in reputations systems for years, and most of the literature seems to be focused solely on eBay. So reputation tracking systems are probably obsolete at this point, because every system is going to have weaknesses and vulnerabilities, and there haven’t been many new developments (at least that I’m aware of — which is a decent indicator that if something great is out there, it sure hasn’t diffused very widely yet).

    What we really need is reputation systems that take advantage of Metcalfe’s Law. As processing speed and memory continue to double — as Information Abundance becomes still more abundant — we need to develop reputation tracking systems that use better proxies. You ask above whether “in a world where all action is recorded, is there still need for reputational information?” I would respond Yes, all the moreso! Reputation data should be broadly understood as a form of filtering and content management. In a world where all action is recorded, reputational information is all the more necessary so we can sort through the mess. But likewise, as more types of data become available, we need to diversify the types of proxies we use for assessing reputation. As the mobile web comes into wider use, whole new classes of data will become available. What we need is to figure out how to sort and use that data, particularly keeping in mind the competing needs for reputational assessment/filtering and privacy. The weaker the privacy norms, the stronger the reputation tracking can be. That tradeoff has steeply decreasing returns at some point, and it’s up to us as public scholars to help identify what that point might be.

    Posted 22 Oct 2008 at 9:15 am
  3. Nick DiGiacomo wrote:

    Your points about the widespread availability of information on-line are well taken. Reputation, however, is a social evaluation. And the evaluation part is a) hard, and b) not being done well now, as far as we can see.

    The challenge in measuring reputation is to model a complex, non-linear social process. Any evaluation of reputation must start with the way our individual beliefs and biases turn fact into opinion. It must also take into consideration the way those opinions (and opinions about opinions) move through social networks. Doing this with classical statistical methods isn’t really possible, since most on-line opinions are highly correlated – voiding the underlying (but often ignored) requirement of event independence.

    So we’ve taken some aspects of Bayesian belief revision, and build an algorithm that turns user input about companies and their reputations (stories, votes, comments) into Reputation Scores and an overall Reputation Index for thousands of companies.

    We’re in private beta now, but I’ve created an invitation link if you and your readers would like to have a look: http://vanno.com/private/signup/Publius_Project
    Otherwise, we’ll be launching to the public in mid-November 2008.

    Posted 27 Oct 2008 at 9:49 am