Chapter 11 - Intelligent Contributions

Summary

Chapter 11 address how contributions are combined in different ways when designing CI. One approach utilize many different perspectives on the same work, like in collective work on the same Wikipedia article. Multidisciplinary innovation teams also include a diversity of perspectives in creative problem solving. Second, contributions can be combined under the assumption that the golden middle way is the best solution. One example is the identification of a quantitative middle point, such as an average, that provide the most accurate solution if contributions are diverse. Another strategy is to find the middle way by developing a balanced representation of all sides like in collective argument mapping. In addition, the middle way can identify commonalities, like the online environment vTaiwan that let the crowd find consensual statements in political conflicts. A third approach scales up the number of contributions in the search for an unexpected solution. Many breakthrough ideas happen at the outskirts of a field. Online innovation contests aim to bring in creative outsiders or unknown others by inviting anyone to join. Furthermore, most of the contributions in CI-projects build on a modularization strategy that split a complex challenge into many smaller subtasks.

11.1 Background

What are collectively intelligent contributions? Obviously, the specific answer to this question will depend on the context. If we look back at the different CI projects in these chapters, we see that CI tasks and the production of content is organized in many different ways. For example, tasks can be done as separate contributions from scratch or in sequence by building on previous contributions. The importance of making separate independent contributions is prominent in many of the contest formats in innovation contests and when aggregating micro contributions in citizen science projects (see Section 2.3). The tasks will typically be performed within a relatively short period on equal terms. Originally, this approach was underlined by the wisdom of crowds literature (Surowiecki, 2005). However, the examples in this book show that dependent contributions that build on each other are equally important. The shift from independent contributions to dependent combined contributions is exemplified by the crowd peer review mentioned in the previous chapter. Traditional reviews will typically involve a few reviewers who do this work independently, but in crowd peer review a much larger group of reviewers build on each other’s work. By increasing the size of the group, the relative size of the individual contribution becomes much smaller, and the problem-solving time is drastically reduced. The “many eyes principle” helps improve the quality of the work.

In reality, successful CI projects may even include phases of both separate and combined contributions. For example, if we look to simple microtasks like the correct classification of images in Galaxy Zoo or in Snapshot Serengeti, this type of work is done in parallel with, but independently of, others. By letting several persons do the same task many times, the number of errors are reduced (Franzoni & Sauermann, 2014). In the next phase, scientists build on these contributions when they write up the research paper.

Another characteristic of many large CI projects is that they build on task modularization. Individuals can work on separate parts of the collective work without needing to have a complete overview. In Wikipedia, every article is organized as a module. Therefore, contributors can easily choose to work on one or a few articles separately even when there are millions of articles. In video platforms, every video can be regard as a separate module that is connected with other videos through user ratings and “smart recommendation” systems. One individual will only be capable of viewing a very limited amount of the available information.

This chapter will further examine the mechanisms that in different ways can make contributions intelligent. Four core mechanisms are highlighted as especially relevant in producing high-quality CI:

1. 1. Many different perspectives on the same work

2. 2. The golden middle way is the best solution

3. 3. Searching for the unexpected solution

4. 4. Modularizing the tasks

11.2 Many Different Perspectives on the Same Work

Several of the CI projects in this book combine contributions by bringing in different perspectives on the same work. The Polymath project illustrates how different individuals can contribute through collective problem-solving processes in time-centric asynchronous discussions. Prediction markets show how different contributions are aggregated through a numerical value that represents the current state of a solution. Crowd peer review brings in a new approach in this area, when dozens of reviewers engage in the review of a single article. It challenges previous models that invite a few reviewers to do more in-depth work (see Section 10.4). The problem-solving period varies, but all these examples aim to utilize CI through the “many eyes principle” (see Section 1.2.3).

Collective writing of a Wikipedia article is an especially good example of the value of letting many different eyes focus on the same work. The article format is supposed to be relatively short and provide a valid and readable overview of a topic. When many contributors work on the same article, studies have found that the quality improves because of the diversity of perspectives and the reduced likelihood of making errors (Giles, 2005). Its trustworthiness is reliant on the use of good secondary sources, and transparent storage of the different versions of the article.

The contributors will perform many different tasks, including simpler tasks like removing spelling errors, fixing hyperlinks or adjusting the content to encyclopedic guidelines. A large amount of the work on Wikipedia articles is done without any explicit discussions on the talk page. A new edit may automatically prompt a new action by another editor. In a stigmergic perspective, the article is an unfinished “text solution” that trigger others to continue to work on it. This is possible because everyone has access to the current “synthesized” version of the article. This work is collective, with the individual author disappearing into the background. All the contributions melt into one coherent solution, which aims to provide a more accurate, and detailed description of the topic.

The person who starts writing a Wikipedia article will often frame the work for new contributors. The first-mover advantage describes a pattern where the initial text on a page tends to survive longer. There are usually fewer modifications of this content than later contributions. It appears that the first person that creates an article generally sets the tone of content (Viégas, Wattenberg, & Dave, 2004). The risk is premature alignment. Because of positive feedback, early choices can be amplified even when they are not good. One such example from a different context is the incident when users on the Reddit site mistakenly identified the bombers of the Boston marathon in 2013 (Halavais, 2018: 84).

However, the ideal is to include “fair” representation of all perspectives in a Wikipedia article. The transparency of the knowledge production process aims to ensure that many perspectives can be integrated in an unbiased way. The talk page attached to each article is important in promoting an informal peer review that can integrate a diversity of perspectives over time.

Although anyone can contribute, a relevant contribution requires some level of background knowledge about the topic in an existing article. Still, the tasks are simple enough to recruit a large number of volunteers. Since there is no time limit for this work, the articles will need to be updated continuously. Because of this, it becomes even more important to create an online community that motivates sustained contributions.

A positive spinoff of this approach is that it can lead to the production of slightly different versions of the same work, which are modified according to personal interests or a local context. This is common in open source software development, but also in knowledge production processes in Wikipedia and open textbooks. For example, when an open textbook is translated into a new language, some of the content may be modified to better fit that specific culture.

Argument mapping is another example of how one can utilize many different eyes on the same work. The basic assumption is that deliberation should build on an informed and rational debate that include all relevant pro- and con- arguments. The interface in an argument map is designed so it can easily display a hierarchy of main arguments, sub arguments and explanations. Clusters of arguments are organized in a tree structure that provides a better overview of all the arguments. The final argument map aims to include all perspectives or positions in both a comprehensive and fair way.

In argument mapping, new contributors will add perspectives according to the existing structure, and they will have to adjust their arguments according to the missing parts in the map. Contributors will need to read other contributions in order to position their argument into an overall coherent structure. In contrast, online discussion forums are often overloaded with too large discussion treads and argument redundancy. Exposure to both pro and con arguments is assumed to strengthen the deliberative processes, which is important when dealing with a complex issue like public health or environmental issues. When the number of perspectives increases, it can potentially create better deliberations. An important design challenge is to be able to sustain the overview of the map when the number of contributions scale up (Klein, 2012).

The problem with a topic-centric technology like a wiki is that it aims to provide one single coherent answer, with the risk of oversimplifying the picture. It offers limited opportunities to deliberate on controversial topics. Since the goal is to include all relevant content on a given topic in one single article, this forces the authors to move towards the “least-common-denominator” consensus. In Wikipedia, the controversial discussions are found on the talk pages attached to the specific article, but this content may also become messy because of the time-centric organizations of the discussion thread (Klein, 2012).

A third example is online innovation teams, where team members have different eyes on the same work. One reason is that the problem requires a multidisciplinary team with different background, as one solver states:

The nature of the challenge was that there is a company out there that wants to get a broad overview of an area to see where they should invest in computational biotechnology. They want to know what should they be doing in the future and what are the general trends. In order to get a picture of that you need people with diverse backgrounds to really pull together this very big picture, and home in on the important things.

Team members with different backgrounds will be better able to provide a broad overview of the field. Another solver highlights how many different eyes help to further develop an idea:

But it’s not just the matter of having ideas. I mean you can have an idea, but if you don’t have a team to build it, it’s only an idea. From the first time you say “wow this is it,” there’s a long haul to making it a reality. You know, the original idea isn’t just “here’s the solution X,” it changes and become so much better for how the team molds and shapes it. The idea is enriched by the rest of the team.

The solver underlines how the team molds and shapes the solutions through elaboration, by continuously moving the idea work forward. However, there are challenges in sticking to one solution. Some teams have to submit several different solutions because they cannot agree on which solution should be further pursued. In contrast, in individual innovation contests, ideas cannot be further refined through elaborative collaborative problem solving.

Furthermore, the whole contest format in the Climate CoLab, described in Chapter 2, builds on the mobilization of many different eyes to help tackle climate change. For example, “integration contests” challenge contestants to combine previously proposed solutions that have been developed independently of each other. Because previous solutions are stored and published openly, this makes it possible to arrange new contests that build on these ideas. Both the proposals and the reviews are open and anyone can leave comments. The design of a transparent communication environment opens up the process to contributions from many different persons. By involving people from all over the world in collective problem-solving, the goal is to make people more aware of how such problems must be addressed from a local, but also systemic perspective.

All these examples illustrate how one type of intelligent contributions center on utilizing many different eyes on the same work.

11.3 The Golden Middle Way Is the Best Solution

If we examine different CI projects, many are similar in their attempt to find the best solution by balancing diverse contributions. It is assumed that the midpoint is the optimal outcome of the crowd opinion. This “golden middle way” or balancing of contributions can be achieved in several different ways.

11.3.1 Meeting at the Quantitative Middle Point

As mentioned in Chapter 4 on human swarm problem solving, many projects estimate a quantitative middle point as the best group contribution. The basic assumption is that a crowd can utilize more valuable information from a variety of sources compared with what a single individual can achieve. When this information is aggregated or combined in an effective way, “errors” or deviations from the optimal solution tend to cancel out if contributions are made independent of each other. If errors are randomly distributed and the group size is large, the law of large numbers ensures that the middle point provides an accurate answer (Surowiecki, 2005). Another example is a prediction market that use a market prize and individual betting to estimate the middle value of the crowd opinion (see Section 6.3). Deliberative Polling also illustrates that averaging can be used to aggregate crowd opinions in political decision-making (see Section 4.4).

An interesting paradox is that by maximizing a diversity of contributions, one increases the probability of aggregating the most accurate midpoint. For instance, one study that compared different crowdsourcing methods found that a crowd is wisest when it is maximally “diverse” in that its members are as negatively correlated with each other as possible (Davis-Stober et al., 2014). When adding a new member to a group, the best strategy is to select somebody who is maximally different from others. By adding a much less skilled, but more diverse member to the group, the group became more accurate compared with just adding a new member with higher but similar skills. This follows the logic that if all individuals provide very similar predictions, there is little extra value in aggregation. Especially if the crowd is small, it is more beneficial to add a new member who can bring in information that is more diverse. When the members become less correlated with one another, the wisdom of the crowd effect becomes stronger. This is even more important than ensuring that the individuals make their estimations independently of each other. The intuition follows the same logic as predictor variables in a multiple regression analyses that are very similar to each other. Adding a new predictor, which gives new information, helps the model even if it is poorly correlated with the outcome variable. In the presence of some skilled members, it becomes more important to add members with truly different perspectives, which also helps to avoid biased opinions (Davis-Stober et al., 2014). This finding supports the diversity prediction theorem, which underlines that crowd wisdom is maximized when judgements systematically differ as much as possible, not when judgements are independent and cancel each other out (L. Hong & Page, 2004).

11.3.2 Finding a Balanced Representation of All Sides

If we look at the content that gets attention today in the online setting, it is sensational rather than balanced content. According to Shifman (2014: 66–71), viral content is designed in a specific way to become popular. First, this type of content will often be funny, and it can be surprising, interesting, or practically useful. People tend to share content that arouses them emotionally. Positive stories can generate a feeling of being in contact with something greater than oneself, like natural wonders or people overcoming adversities. However, stories that evoke negative feelings of anger and anxiety can also become viral. The disadvantage with this trend is that rational arguments are left out because feelings are considered more important in making the content viral. Second, clear and simple news stories spread better than do complex ones. Jokes are more “sharable” because people can understand them quickly. Since the problem is simple, the solution offered is equally straightforward and easy to digest. However, the risk is that complex problems are oversimplified to increase the likelihood of making it viral. It is not necessarily the high-quality content that becomes most popular.

Still, there are some interesting examples of knowledge products that one could claim resemble a “golden middle way” in an attempt to include “all perspectives.” For instance, Wikipedia articles show that it is possible to successfully synthesize content into coherent articles with competing or opposing views. One explanation may be the shared consensus norms like the “neutral point of view” policy that explains that contributors are not supposed to write an “objective” text, but instead seek a fair representation of all sides. The focus on using reliable secondary sources transcends the debate from being a question of whether or not to include specific content to asking how content should be included and with what sources (Algan et al., 2013).

Likewise, in argument maps that promote deliberation, it is important that different political views are presented in a fair way. Argument maps can perhaps complement wiki technology, which is not able to capture knowledge about contentious topics in an efficient way. In simple decision-making, people will perhaps automatically “meet in the middle.” However, in complex decision-making where there is no simple correct answer, it may be more important to get an overview of the issues that divide people most deeply, or identify the best ideas for one specific issue and try to understand why it is like that. In a deliberative community, the goal will not only be to maximize the collective outcome, but also avoid that the minority feels alienated. This can undermine the decision and the future cohesion of the community itself. If discussions are to be experienced as fair and legitimate, the argument mapping tools must be designed to minimize regret rather than solely maximizing the majority outcomes (Klein, 2017). Still, one can question whether the organization of arguments maps into pros and cons risks polarizing the debate even when members have better access to all arguments.

11.3.3 Identifying Commonalities

Another interesting consensus-making approach is vTaiwan; it seeks to find the “golden middle way” by emphasizing consensual statements in the crowd. vTaiwan was established in the aftermaths of the Sunflower movement, a sudden three-week demonstration in 2014 by Taiwanese protesters who occupied parliament because of a trade bill that would bring their country closer to China. The protesters eventually backed down, but it raised another bigger question: how could Taiwan’s government listen better? To find a solution, Taiwan went to the civic hackers who had been part of the protest and asked for help. They wanted to avoid something like this from happening again (C. Miller, 2020).

These civic hackers were organized in leaderless collective called g0v (pronounced “gov zero.”). They believed in radical transparency, building on the values from the open source software philosophy, by which everyone should be included in the decision-making process. Not only were they invited to give advice, but one of its members, Audrey Tang, was appointed the country’s digital minister. A new group with a very different worldview was given political power in a way we have not seen before. The civic hackers saw the main problem to be a lack of communication and information between the citizens and the government. Elections were too infrequent to give government enough information about what the public wanted, while referenda and debates split society. Instead of measuring division, the group thought one should instead design systems that could improve the communication flow and invent new ways of constructing consensus (C. Miller, 2020).

Although the Internet could offer a solution, it was also part of the problem in Taiwan. Online politics in Taiwan was polarized and primarily made people angrier. There were no platforms that let citizens express their preferences to the government in a constructive way. Like in the rest of the world, social media like Twitter, Facebook, and YouTube had turned the political debate into a game focused on capturing attention. The algorithms prioritize information that gets the most clicks, independent of how crazy it is. These platforms are engineered to keep you on the site, leading to content that provokes the strongest emotions. This amplified the politics of division and outrage rather than nuanced discussions or attempts to compromise (C. Miller, 2019, 2020).

In an attempt to create a new type of democratic process that pulled people together rather than split them apart, the civic hackers invented vTaiwan (the “v” stands for virtual). The environment let citizens, politicians, and others discuss proposed laws in addition to joining face-to-face meetings and hackathons. The political goal is to help policymakers strengthen the legitimacy of their decisions by not only letting citizen vote on questions posed by the government, but also by letting them pose the questions. If it succeeds, it is expected to produce something that the government can turn into new laws (Horton, 2018; C. Miller, 2019).

vTaiwan builds on open source tools, and one of the key parts is the Pol.is platform. The platform lets anyone share their feelings openly with each other, and it is possible to agree and disagree with others. It is also possible to upvote or downvote other people’s comments. In this way, Pol.is resembles any other online forum. However, there are also some major differences, as many of the features are designed to bring the groups closer together. When the debate begins, Pol.is draws a map of the debate and shows everyone where they are positioned and where all the different knots of agreement and dissent are positioned. The upvotes and downvotes generate an opinion map of all the participants in the debate, clustering together people who have voted similarly. Even when there are hundreds or thousands of comments, it will be easy to identify like-minded groups rapidly in the map, and get an overview of where there are divides and consensus. The comment system is interesting because it can include a large population of several hundred persons and still stay coherent (Horton, 2018; C. Miller, 2019, 2020).

However, more importantly, the platform does not highlight the most divisive statements, but instead gives most visibility to the most consensual statements. Attention is given to the individual suggestions that find support across the different subgroups and not only in one cluster. After viewing the map, people will usually begin to draft comments they think can win votes from both sides of a divide, gradually bringing subgroups closer together. This specific design feature motivates a competition, bringing up the most nuanced statements that can win the vote of individuals across subgroups. Therefore, most of the participants will typically spend far more time discovering their commonalities rather than just discussing one particular sub-issue. Because of the visualization feature, people can also easily follow the crowd opinion as it unfolds. As such, the technological design in the platform builds on a conception of consensus rule and not majority rule (Horton, 2018; C. Miller, 2019).

Furthermore, because it is not possible to reply to comments, people lose interest in making divisive statements. This almost completely removes the problem of misbehavior. There are also fewer problems with redundancy in this method since only some statements will receive attention in the platform, and it is not necessary to include all statements like in an argument map (Horton, 2018; C. Miller, 2019).

Within a period of three to four weeks, most people will usually agree on most of the statements. This differs from politics when people often spend most of their time discussing their disagreements. By gamifying consensus, the platform is able to create a new type of unity in the process (C. Miller, 2020).

When people express their views, the online platform gives most visibility to those finding consensus across different subgroups. Groups become more aware of what they can agree upon, their hidden consensus. This is different from traditional social media, where algorithms often give primary attention to divisive statements or provocations that receive many comments (C. Miller, 2020). The case reports from Taiwan show that within a period of three to four weeks, most people will agree on most of the statements. While there might be half a dozen polarizing statements, there may be 20 or 30 statements that create broad unity. The success can be attributed to the fact that these commonalities are made more visible than the disagreements. The technology allows people to converge and form a polity. This is not done by resolving bitter disagreements in an online setting, but by pointing to a way forward by revealing the numerous areas most people can agree upon (C. Miller, 2019).

In one of the platform’s early successes, the political issue was how to regulate the company Uber, who were in conflict with the local taxi drivers. Within a few days, the platform voters had moved into two large groups, one pro-Uber and one anti-Uber. The online debate covered anything from calls to ban Uber or let the free market decide. Then something surprising happened when both groups were still trying to attract more supporters. Some members began posting statements that everyone could agree were important, such as rider safety and liability insurance. Gradually, these recommendations were refined as individuals tried to get more votes. Eventually, almost everyone had come up with seven recommendations they could agree upon, for example, that private passenger vehicles should be registered. Underneath an angry debate about Uber regulation, everyone realized that they just really cared about safety. The conflict between pro- and anti-Uber camps had been transformed into a consensus that described how they could both exist, but on specific terms. After the online deliberation, the recommendations were discussed in a face-to-face setting with Uber, the taxi drivers, and experts. The different stakeholders had already been drawn closer to each other as the online debate had identified several “consensus items” – statements that most people agreed with. The government followed the recommendations from vTaiwan and let Uber operate, but only with licensed drivers (Horton, 2018; C. Miller, 2020).

Another interesting example is a conflict over whether drunk drivers should be beaten with canes. More than 10,000 voted on a recent proposal that advocated caning as a punishment for drunk driving and sexual assault, but there was also fierce resistance against this kind of punishment. The government challenged vTaiwan to find consensus where none seemed to exist, with groups both supported and rejected caning with emotional intensity. Initially, opinion was divided into three camps: one group each for and against caning, and a third group argued that the punishment should be more serve. Surprisingly, as in the earlier example, the crowd in Pol.is transformed the discussion. The consensus opinion that emerged had nothing to do with caning, but focused on political strategies preventing such crimes. The crowd had found out that “To cane or not to cane?” was the wrong question to ask. Instead, the group began proposing legislation including alcohol locks and confiscating drunk drivers’ cars. This solution would not have emerged from a traditional online petition that only gave people the option of voting yes or no (Horton, 2018; C. Miller, 2019).

Still, there are several significant challenges in this new type of e-democracy. vTaiwan has mostly focused on digital issues and not yet on a national issue with entrenched polarization. Civil society needs to learn how to use such tools in cooperation with the government. A large part of the population is not comfortable with using such tools. Only 200,000 people have participated in the vTaiwan discussion. However, nearly five million of the country’s 23 million inhabitants have participated on the new platform Join, which builds on a similar method that attempts to create a new public service culture (Horton, 2018). It is problematic to implement such systems on a wider scale when there is a group of people who do not use this type of technology. Young people are usually more tech-savvy, even though all age groups are increasingly using social media. One will perhaps need to use lottocratic methods to ensure demographic representation on important issues.

Another challenge is to incorporate these decision-making structures such that they become a permanent part of the government. When Taiwan’s finance ministry decided to legalize online sales of alcohol, there was concern that online sales would make it easy for children to buy liquor. Alcohol merchants and social groups were just talking past each other, and in 2016, 450 citizens joined vTaiwan to deliberate on the issue. In just a few weeks, both sides discovered that they were actually willing to give the opposing side what it wanted, and they were able to formulate a set of recommendations together. Sales would be limited to a few e-commerce platforms and distributors, and purchases would be collected at convenience stores, making it very hard for children to collect them without arousing suspicion. One month later, the government incorporated the suggestions into a draft bill that it sent to parliament. However, because of a change in administration, the online alcohol sales bill was never implemented, showing the risk of “openwashing” – that such processes can end up only creating the pretense of transparency. Because the government can ignore the discussions, vTaiwan may eventually end up as a tiger without teeth. Many participatory governance projects around the world suffer from the same problem, thus making it difficult to gain credibility with citizens (Horton, 2018).

Public officials and politicians also need to regard online comments as something other than protests. They need to acknowledge the potential in mobilizing citizen expertise (Horton, 2018). Moreover, the experimentation continues. All government drafts of law are now subject to 60 days of public commentary that will be organized in a similar way as in vTaiwan. However, the survival of the platform still depends on the power of the ruling party (C. Miller, 2019).

The most important result of vTaiwan is that it shows that online deliberation is possible if the technology is designed in the right way. Mainstream social media have largely failed in creating a real political debate because they amplify polarizing content. In contrast, consensus platforms can include both citizens and politicians in more constructive ways. By clearing away the divisiveness, systems like vTaiwan can help the crowd agree and give advice to the government in making laws and regulations (C. Miller, 2019, 2020). In the political domain, it appears that intelligent contributions should highlight consensus elements. The system’s potential to reconnect people who are in conflict with each other provides evidence of a promising new approach that should be further examined in the future.

11.4 Searching for the Unexpected Solution

There are numerous examples in the history of science that shows us that scientists responsible for major scientific breakthroughs in a field tend to be marginal to that field. The marginality effect assumes that individuals in marginal positions have access to different knowledge than the actors who are at the center of the source problem field. This increases the likelihood of producing potentially novel solutions. The main reason is that they tend to ignore the prevailing core assumptions in the field of the focal problem. For instance, a study of “high impact” papers shows that they are different both in search scope, search depth, and atypical connections (Schilling & Green, 2011).

In the offline setting, this marginality effect will often not be present because only a limited group of persons have access to the problem. This all changes with the online setting that makes it much easier to recruit people. Several CI projects also aim to utilize this marginality effect by recruiting a large group of problem solvers to produce unexpected solution. For instance, online innovation intermediaries have even become a new business opportunity that seek to help problem seekers by hosting innovation contests. The open call for participation invites individuals from different scientific fields and with different backgrounds (age, institutional affiliation, educational pedigree) (Jeppesen & Lakhani, 2010). The expected quality depends on recruiting a large and diverse group that can produce enough unusual ideas, and increase the likelihood of solving the problem. In some cases, the solvers can even be amateurs with little formal education, as with the finalists in Climate CoLab (Malone, 2018: 181–182). Although most people will find it hard to solve a puzzle, many will still be able to recognize the solution when it is explained to them. The “aha-moment” can both occur when a solver reaches the solution, but also when the seeker is informed about the solution (e.g., “Aha I see it”). For instance, it’s easy to recognize when someone has written a good software program. What often drives the work is the gap between the difficulty of writing programs and the ease of evaluating it. It’s usually much easier to recognize the insight that solves the problem than actually reaching that insight (Nielsen, 2011: 74–75).

Regarding CI, it will be important to examine if contributions can be organized in ways that promote these unexpected “aha moments.” In one study of Innocentive, an online intermediary, 166 problems were broadcast to a potential solver population of up to 80,000 individuals; 993 individuals submitted solutions. In total, 59 were awarded, showing that the solvers managed to solve approximately 30 percent of the problems. (Jeppesen & Lakhani, 2010). The percentage of solutions may appear low, but these problems were all very difficult to solve. The solution seekers were large companies with their own research staff who had not yet been able to solve the problem. The most interesting and surprising finding was that a solver had a greater chance of winning if there was a wide distance between the solver’s field of expertise and the focal field of the problem. Successful solvers were often at the boundary or outside the expected field of expertise (Lakhani et al., 2007). The main reason is that the experts in the focal field had already failed to solve the problem. By announcing the problem as an open challenge, seekers were able to bring in individuals who would know the answer to similar problems in other domains.

If there already exists a solution in another field, a solver will not have to spend much time and effort in solving the problem. For instance, Innocentive arranged a contest for the Alaska-based Oil Spill Recovery Institute, asking for methods to deal with oil when it spills into frigid ocean waters. The problem was how to separate oil from water after they had frozen into a viscous mass. It was a chemist who came up with the solution. He had been working on a construction site, and realized that the same kind of vibrating devices that keep concrete from hardening prematurely could keep oil from congealing in cold water (Malone, 2018: 180–181). In another example, a firm’s research laboratory did not understand the toxicological significance of a particular pathology. The problem had also been discussed with top toxicologists, without any success. In the innovation contest, the solution was surprisingly solved by an expert in protein crystallography who had no previous experience with toxicology problems (Lakhani et al., 2007).

The assumption is that a person or team can solve the problem, but with a radically different approach to the problem. The broad outreach increases the likelihood of finding an outstanding contribution or thinking outside the box. The invitation to participate moves into many different fields and sectors that would not previously have been invited into the problem-solving process. In the recruitment phase, the persons in the relevant crowdsourcing environment will read the announcement and do a preliminary assessment of the probability of solving the problem. Only the persons who think they stand a chance of solving the problem will respond to the call. Although the solution is unexpected to the seeker, the solver may happen to reuse a solution that already exists:

The first thing we did was a general literature search to see if anyone had done part of the challenge. And there was a lot of information already publicly available. What we did then was had a discussion online about which avenue would be the most fruitful, what do we need to do to expand what is already out there, and integrate it with other things that have been out there. You know there was no need to reinvent the wheel, in that case.

Here, part of the work was about checking if anyone had already done the work before. The phrase “no need to reinvent the wheel” shows that the team could reuse solutions that were already publicly available. In recent years, it appears that an increasing number of challenges are less difficult to solve.

In general, the main weakness with online innovation contests is that ideas cannot be further synthesized or recombined after submission. The contest format hinders collaboration between competing individuals.

In comparison, science teams increasingly outperform individuals. They increase the probability of being extremely highly cited – in science and engineering, they are six times more likely to receive at least 1,000 citations than a solo-authored paper. These findings contradict a widespread belief that scientific, technological, and artistic breakthroughs originate from the minds of lone geniuses (Wuchty et al., 2007).

In recent years, team contests have also become more popular in online innovation contests. In the innovation model referred to in this book, a few multidisciplinary teams are invited to compete against each other. An important part of this process concerns the preselection of team members that are most likely to solve the concrete problem. Because of the online setting, it is much easier to put together a team of diverse multidisciplinary experts that cover a wide area of perspectives from all over the world. The teams that compete will work separately, and a facilitator is used to support the process in constructive ways (see also Chapter 8).

Some of the top solvers report that these group processes force them to move outside of their own focal field:

One of the challenges was pretty much in my field, although I think being in your field is difficult. That’s because most of the solutions require you to think outside the box. You may have good ideas but you have become too entrenched in the concepts, knowledge and ideas of your field. It’s harder then to think outside of the box which is really what new ideas require.

This solver claims that one can easily become “too entrenched” in the perspectives of your own field. Likewise, another solver emphasizes the importance of keeping the mind open and “keep thinking about the problem, as the solution might come from anywhere.”

In the previously mentioned Polymath project, comments from outsiders also stimulated top mathematicians to develop their ideas in new and unexpected directions, “something I found more striking than the opportunity for specialization of this kind was how often I found myself having thoughts that I would not have had without some chance remark of another contributor. I think it is mainly this that sped up the process so much” (Voytek, 2017). It shows that even chance remarks can trigger creativity. In this project, an unknown researcher who joined the project also brought in relevant competence from non-mathematical research (Nielsen, 2011).

Another solver emphasizes how an innovation team can do a longer ideation process:

Yes, I do both. Very often what I see is that as a standalone solver you basically come up with one or two ideas and go deep as quickly as you can, because you are alone. With a team, you can do a longer ideation process and I like it when people in a team very quickly list several ideas, even the craziest ideas. When you work in a team you truly think out of the box much more than when you are standalone. Secondly, when you work in a team you can go beyond just an idea because you have multiple expertise. You can really articulate much more because you are bringing multiple expertise and multiple thinking. You can really shape a solution which I think is much more attractive for a seeker.

Because the team is multidisciplinary, it can both provide more creative and “crazy” ideas. The trend is toward greater use of teams by innovation intermediaries. For example, Innocentive, who originally only offered challenges for single individuals, now also offer team challenges. When only a few teams participate in a competition, the winning chances increase. In addition, one avoids the typical “creativity overload” problem in individual competitions that leads to a burdensome review of all the ideas.

Furthermore, some CI projects illustrate a community approach to finding the unexpected creative solutions. The community performance in FoldIt is an interesting example of how a game environment can produce unexpected solutions of high quality. This is done by motivating many users to participate and compete, but also share ideas in a friendly manner (see Section 2.3). Both the IdeaRally (see Section 2.2) and the hackathon (see Section 4.4) illustrate how a large group is recruited to intermingle in an attempt to produce an optimal solution. Within participatory governance, the Better Reykjavík project is another example that shows how the local population can be involved in generating creative ideas that are relevant to the municipality. Together, these examples illustrate that the search for unexpected solutions cover both an individual level, a team level, and a community level.

11.5 Modularizing the Tasks

11.5.1 A Modularization Strategy

A key challenge in many CI projects is how to organize and combine a large number of contributions in an effective way. In general, a modularization strategy is the most common way of organizing the collective work. The complete work is split into many small subtasks that can be performed independently of each other. Collective work on open source software is a classic example, which make it possible to do separate subtasks that still depend on each other. It is easier for participants to organize their attention around single issues that can be separated from each other. It is not even necessary to understand the whole project. Individual work can be done separately from all the other ongoing work, making it much easier to contribute. It may take a lot of extra time to get an overview of all the content or the complete discussion. Another major advantage with modularization is that it builds on the principle that any contributions matters, even very small ones (Nielsen, 2011).

Many citizen science projects that involve analysis of huge amounts of data build on modularization. They are designed as simple well-structured tasks of low complexity (e.g., Galaxy Open Zoo). The problem is clearly defined in advance, it can easily be split into small task pieces, and the criteria for evaluating contributions are well understood (Franzoni & Sauermann, 2014; G. Graham et al., 2015). In one classical example from 2002, NASA published photos of the surface of Mars on an open website so anyone could volunteer to help mark and classify craters. During the first six months, 85,000 users visited the site. In total, volunteers made 1.9 million entries during the project period. Normally, this type of work would require many months of work by a scientist or graduate student. The maps of Mars were divided into many small segments with a simple marking tool. This made it possible to modularize the tasks and split them into smaller components (or modules), which could be worked on independently before they afterwards were put together again. One microtask usually only required a couple of minutes’ work. The users could therefore choose to either quickly mark one crater or work for hours with many craters. One study found that a small group of clickworkers did most of the work, while one-time contributors did 37 percent of the total amount of work. People contributed for the fun of it. Because the modules were independent of each other, contributors could choose when they wanted to contribute. This strengthened both user autonomy and flexibility in the project. The quality of the work was also high because the tasks were discrete and highly modularized. By averaging the coordinates of the user contributions, the results were assessed to be at approximately the same level as an expert scientist (Benkler, 2006; Malone et al., 2009). This example illustrates the potential in letting the crowd solve simple problems that have one correct solution. Benkler (2006) claims that the number of people who can participate in a project is inversely related to the size of the smallest-scale contribution necessary to produce a usable module. If the granularity of the modules is small and the required work effort is sufficiently low, there will be less need for extra “incentives” because individuals can more easily do it in their leisure time.

Another prominent example of modularization are the millions of articles in Wikipedia. In every article, contributions of any size matter. Thousands of persons will be working on thousands of separate articles in Wikipedia, but they do not need to know of each other. Still, articles will be linked together in the online environment. The size of the encyclopedia is much larger than printed encyclopedias, demonstrating how the complexity of solutions can scale in the global online setting. According to Nielsen (2011: 33,56), the ideal is to create a technological platform architecture that gives every participant an easy overview of how they can make the best contributions. This can broaden the range of expertise that can be used, making it easier for newcomers to join the project (reduce barriers to entry) and reduce the time needed to perform a task (Nielsen, 2011).

On the other hand, if projects become too monolithic, it is more difficult to get an overview of what is going on. This is why the Linux project put much effort into modularizing the collective work related to the development of an open source operating system. Many attempts to produce complex knowledge products have also failed because of insufficient modularization (Crowston et al., 2018). For example, although open textbooks can easily be reused and adapted, it has been very difficult to let a large group of people co-produce a textbook like in the Wikibooks project. Although the “big brother” Wikipedia is an enormous success, high-quality textbooks need to be both larger and more coherent, making the work more difficult for amateurs. Nor is it possible to have different writing styles, and one needs to follow local or national curriculum standards if the textbook is to become a part of the syllabus in colleges or schools. These guideline requirements have constrained the project’s granularity, making it more difficult for outsiders to contribute (Benkler, 2006).

In ill-structured tasks, the specific subtasks are not obvious, contributions cannot be easily evaluated, and the problem space will first emerge during the work. When contributions build on each other and are highly interconnected, it is much more difficult to modularize the tasks (Franzoni & Sauermann, 2014). For example, the Galaxy Zoo Quench project illustrates how difficult it is to involve amateurs in writing a research paper together. One reason is that the different parts of a paper, like the introduction, the review, and the data analysis need to be consistent with each other. They cannot be treated as separate modules. The production of a coherent paper requires additional work in planning the writing process and revising the parts so they fit with each other. Only a few tasks in writing, such as proofreading, can be compared with galaxy classification that is done without affecting other tasks. The voice and writing style of the different sections needs to be similar. In addition, problems at a conceptual level are more difficult to identify and resolve. The project failed because it was unable to decompose the analysis into specific subtasks (Crowston et al., 2018). One alternative strategy could perhaps have been to enforce stronger centralized control with a coordinator who organized and modularized the work in advance.

Attempts to let a large group of people write a novel together have also failed. One example is the Million Penguins project, which recruited 1,500 persons to write a novel together with wiki software. This project never became a success because people were not able to work effectively together. A major difference compared with writing an encyclopedia is that it is very difficult to modularize a novel. Every sentence in a novel is to a much larger degree connected with all the other sentences in the overall story. Different parts in the book will be connected with each other in intricate ways. Modularization may lead to dissonance and incoherence. Nor is the wiki software ideal at keeping an overview of longer pieces of writing and the relationships between them. On the contrary, it is a tool that is designed to work well for the collaborative writing of short, independent articles in a reference work (Nielsen, 2011: 53–54).

11.5.2 Modularization in Strongly Interconnected Content Structures

Depending on the complexity of solutions, the modularized task will either be part of a strongly interconnected content structure or a loosely connected content structure. Examples of strongly interconnected content structures are open database projects (e.g., Bird, OpenStreetMap) or collective argument maps. When the structure is predefined, contributor guidelines will usually be more precise. In open databases, contributors are given specific instructions on the type of information that is required. One recent example is participatory mapping in sustainability projects (Nicolosi, French, & Medina, 2020). In these modularized structures, each separate module is built from scratch, they will at the same time be part of a large collective knowledge construction, building on the others’ work. The goal is to produce a richness and diversity that is still easily accessible for others. In addition, it will be relatively easy to get an overview of the complete collective work in these structures.

In eBird, every uploaded bird observation is a module that becomes part of a large database with all observed bird activities. In comparison with an argument, a bird database will not to the same degree depend on filling the missing gaps because users can upload their information independently of others. These databases are collections of information, where individuals can upload information separately from each other. A large amount of data is effectively aggregated because there is no need to coordinate information between the contributors. However, the missing data spots on the map are visible and provide volunteers information about what areas need to be further explored. In this way, new contributions can also build on previous contributions in these databases. One example is the Gulf Coast Oil Spill Tracker, a data visualization tool within eBird. It was a mash-up of several datasets that included the locations of the ten bird species of conservation concern and the current forecast of the oil slick. It provided a valuable source of timely information. In the year following the disaster, more than 4,000 birdwatchers along the gulf submitted over 110,000 checklists. These data were used to estimate the number of birds that died but were never found. It illustrates that baseline monitoring from leisure birdwatchers can be valuable in many different ways (C. Cooper, 2016: 46–47).

An argument map is another example of a tightly interconnected content structure. Each argument will typically represent a module, and it will either be part of a number of pro-arguments or counter-arguments. These arguments are organized in a hierarchical structure. New contributors will need to build on others’ work by positioning a new argument within the framework of arguments that have already been published. The credibility of an argument depends on how it fits into a coherent map structure.

One challenge today is that most groups are better at producing more arguments that can reinforce their prior beliefs compared with finding counter-arguments. In the argument map, exposure to counter-arguments is assumed to lead to a possible change of confirmation bias. “Confirmation bias” is the empirically well-established tendency of individuals to seek out arguments that support a position they already hold. People tend to give more weight to the most striking pieces of information or simply to those pieces of information they already possess, instead of looking for relevant information that might be lacking. When group members disagree, they are most likely to find arguments for their own position, but this can be beneficial in an argument map as it provides more detail to specific arguments. Because people are usually competent at falsifying statements that oppose their views, this can be a useful skill that can be utilized in the map. If both parties participate in this process, it may contribute to the development of a more complete map with more in-depths explanations. However, if the argumentation map is skewed in one specific direction, there is a risk that deliberation can strengthen group polarization. Close attention must be paid to the how decisions are achieved when groups strongly disagree (Landemore, 2013).

11.5.3 Modularization in Loosely Connected Content Structures

Furthermore, some CI practices center on loosely connected content structures. Many knowledge products today, whether it is a text or a video, are published openly and become accessible for others. The work becomes a tiny module in an enormous network of interlinked information on the Internet. From this perspective, every video on YouTube is a separate module in a platform where the most realistic way of finding the video is by using its search engine.

However, one of the main challenges in the new global online environments is how to tackle the problem of information overload. While the traditional expert model built on the transmission of knowledge from a formal expert to many less knowledgeable others, the networked online environment illustrates how anyone with some skill or knowledge can share their knowledge openly with others. The problem is that both the number of experts available and the number of amateur contributions have become enormously large. Searching through hundreds of scientific papers that describe an issue is far too time-consuming for a single individual. While it is still possible to synthesize content in text documents, this is much harder to do with videos because of its multimodal properties. The most relevant videos will therefore need to be identified, and viewing many videos to select the most relevant is far too time consuming for an ordinary individual.

Today, search engines are considered the best option to solve this challenge of finding modularized information in a loosely connected content structure. These search engines require that citizens find and assess relevant information on their own. Although search engines like Google contain links to a large proportion of the web, only a tiny percentage is made easily accessible to users. Unlike television or even printing, it is easy for anyone to publish their opinion to a large, potentially global, audience, but this does not imply that anyone will pay attention to this “diversity of content.” The assumption that computer networks are more democratic, and necessarily provide a greater voice to everyone, is misguided (Halavais, 2018: 101).

While the distributed nature of computer networking makes it less likely that a small number of interests can control it, this does not imply that the web is a level, uniform network. Most likely, any given contribution on the web will be lost in the flood of similar efforts. While search engines make it easier to find answers to specific questions, this is done at the expense of the larger, diverse world of information and opinions. Of the millions of blogs in the blogosphere or videos on YouTube, most get viewed by only by one or two people, while a small number get millions of hits, this being far from equal access to the greater web audience (Halavais, 2018: 101).

Paradoxically, search engines that can retrieve enormous amounts of information are today being criticized because they oversimplify the available information when they only display a limited number of hits. A search for “staph infections” will generate a hit of about 1,200,000 pages, as health topics are popular on the Internet. The first three results, which most users will check, are from mainstream, relatively well-respected sites, but in total only 234 results are displayed. It shows that an enormous number of pages are left out. When most people use a search engine like Google, they typically only check the top ranked pages. On one side, it is necessary to select only a few results because our attention span is limited. The goal of using the search engine is to avoid the “junk” on the web and provide higher precision in the search results of the search engine. However, the hyperlinked structure tends to send the searches along the same path to the same informational sources. The result is that general-purpose search engines overrepresent the central tendency and reduce the diversity of the information when they operate in the hyperlinked structure of the web. Even though the search tools are not intentionally designed to amplify a few top selections, the ranking systems are conservative and reinforce existing orders of authority. It is not a question of whose power they conserve, but rather that they tend to enforce a “winner-take-all” structure that is difficult to break free from. The network structure on the Internet is organized in such a way that a lot of attention is given to a few sites, while many sites receive no attention at all. The risk is that search engines amplify a global groupthink monoculture and makes it more difficult to find local cultures and practices. There is a naturally tendency to move towards monopoly: “one search to rule them all.” This becomes a problem when large search engines are used by a very diverse set of users with different needs. It becomes more difficult to serve the interests of marginalized groups because a general relevance in search engine rankings does not necessarily match an individual query situated within a very particular aim (Halavais, 2018).

In the network, only a few links survive and are amplified, leading to concentrations of power and influence. This results in a fight for attention in the network structure. Virality itself is also highly persuasive; view counts inform viewers that many others find a message interesting, and this amplifies the spread of the message. If the author is already famous, it is more likely that people will share the work. In viral marketing, there are two types of preferred influencers – the “hubs,” people with a high number of connections to others, and “bridges,” people who connect between otherwise unconnected parts of the network. In marketing, it is much more important to get the attention of these highly connected individuals compared to just sending the message to “regular” users (Shifman, 2014).

11.6 Summary

If we compare the different value-producing mechanisms, they are all similar in their attempt to produce better solutions by integrating diverse contributions. Many CI projects are designed to include as many relevant contributions as possible in an attempt to capture the “complete complex picture” of a problem, either by combining all contributions or through selecting the single best contribution. However, the increase in informational diversity risks ruining the coherence by making the complexity overwhelming. Therefore, the intelligent contributions needs to be organized so they provide some kind of overview of both the processes and the products of the collective work.

Modularization of tasks is the most typical strategy to deal with the overview challenge. Another strategy is to remove the need for overview, like when independent contributions are harnessed in some “wisdom of crowd” approaches. Similarly, most innovation contests let individuals find optimal solutions independently of each other. The benefit is in the “stranger bonus” that is created when many proposals are generated. Here, the overview challenge will be how to effectively review all the proposed solutions afterwards.

If we also look at how knowledge products are modularized and accessed through search engines, it is evident that only a few “winner” solutions will get most of our attention. In an enormous, loosely connected network structures, it is very difficult, often impossible, for an individual to keep an overview of all the content. Algorithms do the work of selecting the best solution or narrowing down the individual choice to a few options. The disadvantage is that we know little of how the algorithms work, but convenience and time efficiency still make search engines the preferable alternative. However, there is a risk that popular hits are biased, and do not provide the best quality option, as many will fight for attention in the online setting.

Another way of coping with the challenge of information overload is to fuse all contributions together in the ongoing work, as when the complexity of a module in Wikipedia never moves beyond the size of one readable article. This makes it easy for an individual to have an overview of the content, and receive contributions over time. vTaiwan is another example of a decision-making technology (Pol.is) that enables individuals to easily keep an overview of discussions with several hundred participants. Improving the quality of deliberation in an online setting is a major challenge. The case stories from Taiwan illustrate that “consensus games” can transform disagreements. This is possible because all contributions are part of an emerging map structure that creates a visual overview of the debate, and the aggregated clusters of different positions.

In vTaiwan, the second phase is centered around an online competition between who can find the best “consensus solution”. Rapid negotiations are performed through up- and downvoting of different alternatives that resemble the honeybee dance in swarm problem solving. The crowd constantly chases the best consensus solution in its attempt to win votes. Part of the success appears to be how rapidly participants can change positions in the network structure, compromise, and move towards the “golden middle way.” In this process, the parties discover that they agree on more issues than they previously thought. The logic of communication is entirely different from the algorithms in social media platforms that are designed to maximize profit by reinforcing existing preferences.

In argument mapping, the CI design is very different in its emphasis on the construction of a comprehensive set of arguments. The hierarchical organization of the map assumes that all arguments can be linked together in a systematic and coherent way. New contributions should not be made separate from existing information, but rather adjust itself to the current state of the collective knowledge production. Previous contributions will also be checked and revised when many participants read the same information. Since there is no point in adding an argument or information that has already been made, this can potentially provide a better overview of a complex debate. The predefined structure in pro and con arguments aims to create a better overview of a complex debate and avoid that persons only stay on one side in the debate.

However, one can question whether the technological design overemphasizes a dichotomy between pro and con arguments, especially if the group needs to develop a solution that synthesizes or transforms the current debate. Still, the process of filling in gaps in an argument map may be educational and lead many individuals into new argumentative areas that they have not previously examined. This may result in an attitudinal change.

According to Landemore (2013), receiving complete information about political parties is not enough – there needs to be a deliberative discourse that builds on this informational diversity. The available information should create the foundation for a diversity of reasoning processes that include both pro and con arguments. This is important because individuals often fail to be self-critical towards their own arguments. In argument mapping, individuals are challenged to both find better support for their own beliefs and assess arguments advanced by others. If individuals want to convince others of a given proposition, they will be motivated to find good arguments that are likely to convince the listener. When listening to arguments, individuals will want to evaluate the soundness of the arguments before they accept the conclusions (Landemore, 2013: 126–127).

One must also remember that many find it challenging to engage in political debate with others with whom they disagree. Most people, when faced with disagreement, will prefer to retreat to like-minded peers or avoid political discourse at all. Disagreement threatens norms of politeness and interpersonal harmony (Landemore, 2020). While vTaiwan aims to transform the discussions through consensus building, argument mapping appeals to the rationality of individuals in providing arguments that are more informed to all parties. In addition to the verbal offline discussions, argument mapping is reliant on the production of written arguments as part of a comprehensive framework. It enables individuals to compare arguments before decisions are made. One advantage is that this tool makes it easier to bring forward arguments from minority groups (Klein, 2017).

However, the major challenge is how to summarize the complexity and bring forward the most interesting questions and arguments in an effective way. Depending on the problem, users can be challenged to synthesize the argument map or refine proposed solutions in the final stage. From one perspective, one could claim that the technological design builds on utopian rationality. It is assumed that all arguments can be integrated into a coherent and logical map. However, it is important to be aware that the argument map is usually not a goal in itself, but primarily a support for informed discussions in an offline setting. For example, the final decision may be a vote on a few proposals (Klein, 2017).

If we compare all these examples, transparency is important in letting any individual access all information. New digital technologies aim to provide relatively simple overview in different ways. Both the consensus platform and the argument map are very different from an echo chamber in that they provide information about every individual as a part of the whole group. While the pol.is platform partially expects the crowd to reach consensus on some issues before they meet offline, the argument map technologies assume that consensus must be achieved in an offline setting. In the offline setting, vTaiwan recommends both meetings and hackathons, which can help lawmakers implement decisions with a greater degree of legitimacy. In the political domain, it is important to find the right balance between intelligent contributions in the interplay between an offline and online setting. vTaiwan has been used in 26 cases, with 80 percent leading to “decisive government action” (Horton, 2018), but the government is still not required to pay attention to the outcomes of those debates. Institutionalizing CI-practices is key, but the pace of implementing new decision-making methods or argument maps is slow. Often, power structures in the existing system will need to change, and some may question whether it is a good idea to transfer more political power to a large crowd.