Encyclopedia of Life

The Encyclopedia of Life is a rather new project to create an online and every growing encyclopedia of the species of life on earth. This is just a narrow slice of what Wikipedia nominally covers (everything!), but its ambition highlights the fact that we should expect to see more and more specialized encyclopedias growing through user-contributed content (that is, Wikipedia cannot be a successful single source of knowledge).
The task: there are currently about 1.8 million known species; the relevant scientific community thinks there are about 18 million more to be discovered. And an encyclopedia does not merely name the speciies: it compiles a wide range of useful information (some of the current developed pages have 20 or more subheadings, multiple photographs, extensive bibliographies, even references to the species in literature; see, e.g., Eastern White Pine).
And what about the usual incentives problems: why contribute? what make the effort necessary for quality contributions? how to limit pollution? By focusing on a specialized and visible community of scientists, some of these problems may be smaller than in other settings. In particular, I expect that quality will be handled by a mix of contributors not wanting to look foolish to their colleagues, and other contributors delighting in showing how much better their knowledge is as they make quality-improving edits. The rewards are similar to the standard rewards of recognition and satisfaction with documenting and adding to knowledge that have kept academia moving for the past several hundred years.
Pollution and quality also will be managed, it appears, by having volunteer experts assigned as “curators”, or moderators for each page. This is reminiscent of the method that seems to work well on Slashdot, for example.
But what about inducing contributions in the first place? The project’s Executive Director, Prof. James Edwards, said

“We have not given enough thought to the people who provide the information on which the Encyclopedia of Life is built. “We are looking into ways to keep that community going.��?

(This quote and other material above from today’s New York Times article on the EOL.)
Indeed, in a twist not often heard when talking about getting people to contribute to, say, Wikipedia, the founders are worried about the community dying off:

“The ranks of taxonomists — the scientists who describe species and revise old descriptions — have been shrinking steadily for decades. Dr. Wilson hopes the Encyclopedia of Life will foster the growth of that group.”

(Carl Zimmer, NYT 26 Feb 2008.)
My student, Lian Jian, is currently working on a project to discover reasons why contributors “exit” (stop contributing to) Wikipedia. Here is a poster describing her preliminary work. As the many new and exciting user-contributed content projects mature, inducing a flow of new contributors to replace those who exit, and passing along the organizational memory and routines, will become important determinants of long-term success.

Followup: Second GiveWell founder admits deception

Earlier I noted that the founder of nonprofit GiveWell had been demoted and financially penalized when it was learned that he used a pseudonym online to recommend his own organization. The New York Times reports today that a second founder has admitted he also used a false name in an online posting recommending GiveWell.
These are provocative examples of the manipulation problem — one species of managing the quality of user-contributed content — because GiveWell’s business is evaluating the reliability and quality of other nonprofits in order to provide advice on where to give one’s charitable donations.

ICD: A 5-step program?

Bob Gibbons presented an intriguing framework for an incentive-centered design program during a talk he gave to our STIET seminar on 31 January. He wasn’t thinking about information system design problems, but organizational design. But his fundamental concern was the same: an organization’s performance depends on the incentives and how agents respond to them. Like us, he took an explicitly multidisciplinary perspective.

Before I summarize his framework,I’ll mention that his multidisciplinary lens is somewhat different than what my group of colleagues and students and I usually do. Our focus so far has been on the interaction between economics (rational choice theory) and computation (information processing). We’ve talked for a while, and a few of us are starting to integration social psychology perspectives as well (especially to think about non-monetary and intrinsic incentives). We also see a role for personality psychology, and maybe cognitive.

Bob, like me, starts from the foundation of economics. (By the way, Bob was hired by MIT as an assistant professor in Economics while I was there as a grad student, so we got to know each other a bit — as he reminded me the other day, we played basketball together. He’s now a full professor in the business economics group of the Sloan School of Management at MIT.) But then he moves into politics and complex systems perspectives. In particular, he relies on March’s approach to control and hierarchies in organizations, and on Winter’s work that predicts path dependence.

On to Bob’s 5-step program:

  1. The formal is flawed
  2. The relational is required
  3. The formal and relational interact
  4. Institutional design
  5. Building & changing relationships

These suggest a way of explaining why cross-disciplinary approaches are important for ICD, and a framework for moving forward. I can barely do these justice in a short note; Bob gave a detailed 80-minute talk with this outline. I’ll try:

Formal is flawed. Formal models that rely on a few narrowly drawn incentive instruments are incapable of doing a very convincing job of describing complex incentive problems. For example, for the standard price model, Bob poses this challenge: “Find an employee with fabulous incentives created solely by a formula.” Formulae are not enough: there are too many measurement problems, contingencies, etc.

Relational is required. A quote from Leamer (2007) summarizes the point better than I can: “Most exchanges take place within the context of long-term relationships that create the language needed for buyer and seller to communicate, that establish the trust needed to carry out the exchange, that allow ongoing servicing of implicit or explicit guarantees, that monitor the truthfulness of both parties, and that punish those who mislead.�? A point that Bob made is that most organizational interactions are more like long-term repeated games than they are like one-shot strategic interactions. As a general matter, long-term relationships can lead to a wide variety of outcomes (cf. the Folk Theorem). And, another general implication of repeated games is that the “shadow of the future” is pivotal, so investing in and respecting the relationship is crucial.

Formal and relational interact. The way that relations are structured can have a strong (even dispositive) impact on the effectiveness of the formal incentives. For example, because of the shadow of the future, it can make sense to include a subjective bonus in a compensation plan (that is, the principal says something like “trust me to honestly assess your performance and pay you an ex post bonus based on it” — trust because the performance is not verifiable by a court and thus not contractible).

(Intermediate summary: Some prices can be chosen, but not the right ones because of gap between performance goals and contractible measures. Relationships help, but not enough. Reliance on relationships affects the desired structure of formal incentives.)

Institutional design. Cyert and March (1963): An organization “is basically a coalition without a generally shared, consistent set of goals. Consequently, we cannot assume that a rational manager can treat the organization as a simple instrument in his dealings with the external world. Just as he needs to predict and attempt to manipulate the ‘external’ environment, he must predict and attempt to manipulate his own firm.�? And here’s where politics, authority and control come in: Pfeffer (1981): “it is necessary to understand who participates in decision making, what determines each player’s stand on the issues, what determines each actor’s relative power, and how the decision process arrives at a decision.�? Gibbons’ conclusion: Choose the formal to facilitate the relational. He didn’t spend much time on this idea, but one of his examples is that the best allocation of control for spot conditions may not be best for relational decisions.

Building and changing relationships. The driving point here is that seemingly similar organizations experience persistent performance differences. Bob explains this as a consequence of path dependence. The paths may differ in (among other things?) the extent to which they rely on formal and on relational incentives. He suggests a stylized, extreme case, in which a concave possibilities frontier between a very controlled firm and a very decentralized firm is traced by relational restructurings, whereas primary reliance on formal incentives carves out a path convex and far inside the frontier. Where the firm ends up, then, depends on the mix of formal and relational incentive structures it employs. (See his slides 49 and 50.)

This is all a bit vague, largely because I don’t have a good grasp of the ideas yet. I’ll follow Bob’s work and see if I can make it more concretely useful for the ICD research programme.

The Industry Standard returns as prediction market

The Industry Standard was one of the go-to news sources during the late 1990s dot-com boom. Like so many of the companies on which it reported, it went bankrupt in 2001. However, the computer industry trade publisher International Data Group brought it back to life today.
In a novel twist, The Industry Standard is now combined online newspaper and a prediction market. With a free account, readers can place (fake currency) side bets on various predictions about industry events. For example, markets open today include whether Yahoo! will accept Microsoft’s bid by the end of the week, and whether Google, Yahoo! or Microsoft will buy Tivo by the end of the summer.
Prediction markets are a type of incentive-centered design that have become widespread and hot in the past few years. The first significant example I recall is the 1988 Idea Futures market by Robin Hanson, though academic articles by Hofstee (1984) and Leamer (1986) came a bit earlier.
There are quite a few well-known and active prediction markets now, such as Hollywood Stock Exchange, TradeSports, and News Futures. However, the new Industry Standard is the first time I’ve seen a significant market that is so closely linked to an online news source (though others sometimes provide a limited news feed). It seems like a natural idea: an industry-focused news site (if successful) is bringing in knowledgeable people with an interest in the likelihood of industry events.
Why is a prediction market an incentive-centered design? It is providing incentives to induce knowledgeable people to make the effort to reveal their knowledge and beliefs, and to do so honestly (so they address problems of both hidden action and hidden characteristics). Traditional stock markets provide this information revelation and aggregation role for the futures of publicly-traded companies; prediction markets play the same role for other outcomes.
Most of them, due to gambling and securities laws in the US, run based on funny money (valueless currency). Thus, the substance of the incentives is not immediately clear, and it is a topic for future research to determine how well these markets do induce effort and truth revelation. (The Iowa Electronic Markets, which has approval to trade in small amounts of money, mostly on political outcomes, has performed extremely well — almost always better than professional polling organizations such as Gallup — for many years.) It is going to be interesting and instructive to analyze participant responses to non-pecuniary incentives.
Former Michigan Ph.D. student Dave Pennock (now with Yahoo! Research) publishes an excellent blog about predicion markets: Oddhead.
(Image courtesy of “Financial Aid Podcast” on Flickr.