There’s a truism that bothers many (except economists): if there is a good or service that has value to some and can be produced at a cost below that value by someone else, there will be a market. This is disturbing to many because it is as true for areas of dubious morality such as sexual transactions, clear immorality (human trafficking and slavery) as it is for lawn mowing and automobiles.
Likewise for online activities, as I’ve documented many times here. You can buy twitter followers, Yelp reviews, likes on Facebook, votes on Reddit. And, of course, Wikipedia, where you can buy pages or edits, or even (shades of The Sopranos), “protection”.
Here is an article that reports at some length on large scale, commercialized Wikipedia editing and page management services. Surprised? Just another PR service, like social media management services provided by every advertising / marketing / image management service today.
There is an article in the new issue of CACM on “Crowdsourcing systems on the World-Wide Web”, by Anhai Doan, Raghu Ramakrishnan, and Alon Y. Halevy. In it they offer a definition of crowdsourcing systems, characterize them along nine dimensions, and discuss some of the dimensions as challenges.
It’s a useful review article, with many examples and a good bibliography. The characterization in nine dimensions is clear and I think mostly useful.
I’m particularly pleased to see that they have given prominent attention to the incentive-centered design issues on which I (and this blog) have focused for years. Indeed, they define crowdsourcing systems in terms of four incentive problems that must be solved (distinguishing them from, say, crowd management systems that only address three of the questions). They define crowdsourcing as “A system that enlists humans to help solve a problem defined by the system owners, if in doing so it addresses four fundamental challenges:
- How to recruit and retain users?
- What contributions can users make?
- How to combine user contributions to solve the target problems?
- How to evaluate users and their contributions?
The first and second are the “getting stuff in” (contribution) problem about which I write. How to get people to make effort to contribute something to the good of others? The fourth is the quality incentive problem, which I usually separate into “getting good stuff in” (positive quality), and “keeping bad stuff out”.
Here is a nice article in Wired about Googlenomics, featuring my co-author and friend Hal Varian. This describes a number of ways that Google has combined vast data mining resources with economics to do some incentive-centered design.
Hal is a micreconomist par excellence, who has made important contributions to both theory and empirical work. He was one of the first economists who took the study of the Internet and related phenomena seriously.
He was my colleague at Michigan, and involved in some of the early meetings in which a group of us developed the plan to create the School of Information. The year we launched, however, he departed for Berkeley, where a year later he was dean of their new School of Information (called SIMS at the time, but since renamed). For the past few years he has been on leave from Berkeley to be the Chief Economist at Google.
Sir Tim Berners-Lee and his colleagues have been advocating for a field of “web science” for several years. They describe it as a multidisciplinary, systems science needed to understand and engineer the future Web.
I’ve not fully grokked what they are proposing: the research questions they suggest to define seem a bit vague, and I don’t quite see what defines this “science” other than a set of topics (the semantic web prominent among them, natch) in which this group of people is interested. I’m not saying I think it’s not science, I’m just not sure what field definition they are proposing (so that, for example, many universities could start offering courses, or even creating departments of web science) — it feels like the space of current interest to a particular research center (and indeed, there is a Web Science Research Institute).
They have published another manifesto (also available from the WSRI site) (there have been several in the past few years), this time in the Communications of the ACM. Maybe I’m just paying more attention, but I think I’m starting to get parts of it. In any case, one thing seems clear to me: incentive-centered design (ICD) fits comfortably in their framework. It’s a subset of their very ambitious agenda, but I think it’s clearly a central piece of. Put another way, they are saying some of the same things our ICD group has been saying independently over the last several years.
We show there is significant interplay among the social interactions enabled by the Web’s design….However, the study of the relationships among these levels is often hampered by the disciplinary boundaries that tend to separate the study of the underlying networking from the study of the social applications.
I agree, and this has been part of the ICD manifesto from the beginning. It is rather vague, but here’s a clearer statement of the common starting point: “It is the interaction of human beings creating, linking, and consuming information that generates the Web’s behavior.”
They suggest, as do we, that this inquiry should rely (among other things) on the sciences of motivated behavior, such as economics and psychology. However, I think there is one way in which we diverge: for the most part, I have seen these authors talking about computer scientists and web engineers needing to understand how people behave so they will understand the consequences of web design decisions. But, I have not really seen much evidence that they recognize the role of incorporating motivated human behavior directly into the design loop, which is the essence of ICD: design incentives or motivations for the humans who will be interacting with and over the web in order to obtain desired consequences. What we propose is more a more central recognition of the malleability of human behavior, and the social (or commercial) value to be gained from designing for that malleability.
On the other hand, one large area of research (and not the only one) that the “web science” promoters claim that is not inside the boundaries of what we call ICD, is a micro-behavioral science of understanding and explaining observed macro web phenomena. For example, they point out that social network analysts (like my SI colleague Lada Adamic, U Mich’s Mark Newman, or HP Labs’s Bernardo Huberman) rarely explore or test the underlying human behavior that generates the macro phenomena they observe and characterize.
In another June 2008 American Economic Review article, Ellingsen and Johannesson introduce a standard concept from social psychology into a standard economic model of incentives, and find that it helps explain some well-known empirical puzzles.
This is not at all the first article in the economics literature that explores the role of social motivations, and the authors provide a good discussion of prior work.
“In Pride and Prejudice: The Human Side of Incentive Theory“, Ellingsen and Johannesson add two motivational premises to the standard principal-aget model: people value social esteem, and the value they experience depends symmetrically on who provides the esteem: they value esteem more from those who they themselves esteem.
Their main result is to show how an incentive that otherwise would have a positive effect on behavior can have a negative effect for some people because of what the incentive tells the agent about the principal. For example, they suggest this as an explanation for “the incentive intensity puzzle that stronger material incentives and closer control sometimes induce worse performance” (p. 990).
Students often ask me what they can read to learn about ICD. I’ve not had a terribly good answer to that. On the one hand, the foundations — especially mechanism design in economics, and game theory, and engineering design theory, and social psychology — are ancient (well, a few decades old), and have very rich literatures. But I haven’t seen (haven’t really searched for) good intros. And, these are the building blocks of ICD, but the particular area in which we focus — incentive-centered design for information systems — and the particular multi-disciplinary approach we take — is rather new. I don’t know that folks have written any good overviews yet.
However, three quite nice articles just appeared in the American Economic Review that are a step in the right direction. They are focused on mechanism design and microeconomics (not social psychology, computation theory, nor specifically applications to information system design). But they are accessible, short, and written by giants in the field; in fact, they are revised versions of the Nobel lectures given the by three laureates recently cited for creating the foundations of mechanism design theory: Leonard Hurwicz, Eric Maskin and Roger Myerson.
Maskin’s overview, “Mechanism Design: How to Implement Social Goals“, doesn’t require any math. He introduces implementation theory, “which, given a social goal, characterizes when we can design a mechanism whose predicted outcomes (i.e., the set of equilibrium outcomes) coincide with the desirable outcomes” (p. 567).
Myerson’s article, “Perspectives on Mechanism Design in Economic Theory“, begins to introduce some of the basic modeling elements from the theory, so it has a bit more math, but it’s not heavy going for those who have had an intermediate microeconomics class. He introduces some of the classic applications from economics: bilateral trade with advsere selection (hidden information), and project management with moral hazard (hidden action).
Yahoo! Research invited me to speak in their “Big Thinkers” series at the Santa Clara campus on 12 March 2008. My talk was “Incentive-centered design for user-contributed content: Getting the good stuff in, Keeping the bad stuff out.”
My hosts wrote a summary of the talk (that is a bit incorrect in places and skips some of the main points, but is reasonably good), and posted a video they took of the talk. The video, unfortunately, focuses mostly on me without my visual presentation, panning only occasionally to show a handful of the 140 or so illustrations I used. The talk is, I think, much more effective with the visual component. (In particular, it reduces the impact of the amount of time I spend glancing down to check my speaker notes!)
In the talk I present a three-part story: UCC problems are unavoidably ICD problems; ICD offers a principled approach to design; and ICD works in practical settings. I described three main incentives challenges for UCC design: getting people to contribute; motivating quality and variety of contributions; and discouraging “polluters” from using the UCC platform as an opportunity to publish off-topic content (such as commercial ads, or spam). I illustrated with a number of examples in the wild, and a number of emerging research projects on which my students and I are working.