Recent advances in machine learning have enabled computers to converse with humans meaningfully. In this study, we propose using this technology to facilitate design conversations in large-scale urban development projects by creating chatbot systems that can automate and streamline information exchange between stakeholders and designers. To this end, we developed and evaluated a proof-of-concept chatbot system that can perform design conversations on a specific construction project and convert those conversations into a list of requirements. Next, in an experiment with 56 participants, we compared the chatbot system to a regular online survey, focusing on user satisfaction and the quality and quantity of collected information. The results revealed that, with regard to user satisfaction, the participants preferred the chatbot experience to a regular survey. However, we found that chatbot conversations produced more data than the survey, with a similar rate of novel ideas but fewer themes. Our findings provide robust evidence that chatbots can be effectively used for design discussions in large-scale design projects and offer a user-friendly experience that can help to engage people in the design process. Based on this evidence, by providing a space for meaningful conversations between stakeholders and expanding the reach of design projects, the use of chatbot systems in interactive design systems can potentially improve design processes and their outcomes.

Researchers have encountered many issues while studying rare illnesses such as lack of information, limited sample sizes, difficulty in diagnosis, and more. However, perhaps the biggest challenge is to recruit a large enough sample size for clinical studies; at the same time, obtaining chronological data for these patients is even more difficult. This has urged us to implement a decentralized crowdsourcing medical data sharing platform to obtain chronological rare data for certain diseases, providing both patients and other stakeholders an easier and more secure way of trading medical data by utilizing blockchain technology. This facilitates the obtention of the most elusive types of health data by dynamically allocating extra financial incentives depending on data scarcity. We also provide a novel framework for medical data cross-validation where the system checks the volunteer reviewer count. The review score depends on the count, and the more the reviewers, the bigger the final score. We also explain how differential privacy is used to protect the privacy of individual medical data while enabling data monetization.

Oxymorons combine two opposite terms in a paradoxical manner. They are closely intertwined with antonymy, since the union of antonymous items creates the paradoxical effect of the oxymoron and generates a new meaning. Compared to other forms of figurative language, oxymorons are largely underinvestigated. We explored what makes good oxymorons through a crowdsourcing task in which we asked participants to judge the acceptability, comprehensibility, effectiveness/aptness, commonness, pleasantness, and humoristic connotation of Italian adjective–noun oxymorons. We hypothesized that oxymorons featuring morphologically related antonyms (felice infelicità ‘happy unhappiness’) may be perceived to be better than oxymorons featuring morphologically unrelated antonyms (felice tristezza ‘happy sadness’) and that oxymorons constructed by complementaries (esatta inesattezza ‘exact inexactness’) may be perceived to be better than oxymorons constructed by contraries (bella bruttezza ‘beautiful ugliness’). The results confirmed only partially our hypotheses: oxymorons with complementaries were perceived as more acceptable, comprehensible, effective/apt, common, whereas no strong trend was found for the other two dimensions. Surprisingly, our analyses revealed that oxymoronic constructions containing morphologically unrelated words were perceived as more acceptable, comprehensible, effective/apt, common, pleasant, contradicting our initial expectations.

Le français est une langue parlée par plusieurs centaines de millions de locuteurs en Europe, en Afrique et en Amérique. Une telle dispersion favorise la variation, mais de grands corpus unifiés permettant de rendre compte de cette variation à l’échelle mondiale restent rares, et dans tous les cas nécessitent des efforts financiers et humains non négligeables, à l’instar du projet de Phonologie du Français Contemporain. Dans cet article, nous présentons une alternative possible : les données participatives. Pour ce faire, nous présentons Lingua Libre, la médiathèque linguistique participative de Wikimédia France, et l’utilisons pour décrire la variation sur une opposition phonémique entre deux voyelles ouvertes, /a/ et /ɑ/, dans de nombreuses variétés de français. Les données de 38 locuteurs provenant de 26 points d’enquête sont traitées automatiquement et comparées aux mesures présentées dans la littérature passée. Les résultats montrent que la plateforme a le potentiel de donner des résultats conformes à ceux des études de terrain professionnelles. L’article conclut sur les avantages et les limites de la plateforme, tout en proposant des pistes d’amélioration.

Although the individual has been the focus of most research into judgment and decision-making (JDM), important decisions in the real world are often made collectively rather than individually, a tendency that has increased in recent times with the opportunities for easy information exchange through the Internet. From this perspective, JDM research that factors in this social context has increased generalizability and mundane realism relative to that which ignores it. We delineate a problem-space for research within which we locate protocols that are used to study or support collective JDM, identify a common research question posed by all of these protocols—‘What are the factors leading to opinion change for the better (‘virtuous opinion change’) in individual JDM agents?’—and propose a modeling approach and research paradigm using structured groups (i.e., groups with some constraints on their interaction), for answering this question. This paradigm, based on that used in studies of judge-adviser systems, avoids the need for real interacting groups and their attendant logistical problems, lack of power, and poor experimental control. We report an experiment using our paradigm on the effects of group size and opinion diversity on judgmental forecasting performance to illustrate our approach. The study found a U-shaped effect of group size on the probability of opinion change, but no effect on the amount of virtuous opinion change. Implications of our approach for development of more externally valid empirical studies and theories of JDM, and for the design of structured-group techniques to support collective JDM, are discussed.

The availability of digital data in the fourth industrial revolution brings different trends with new opportunities and challenges for the engineering design community. As an opportunity, these trends would impact engineering design. However, the challenge is finding applications for these trends in engineering design. Crowdsourcing is one of the trends inspired by digital data. It is outsourcing an individually performed task to be mass-performed. This paper explores the application of crowdsourcing in identifying engineering design problems. Identifying an engineering design problem is an aspect of engineering design considered challenging but necessary for inventions. Secondary data from 63 invention-related cases and an interview with a renowned UK inventor are presented. The data contains scenarios on how the engineering design problems solved to qualify for a UK patent grant or application are identified. Lessons from the case studies are presented and discussed, especially regarding crowdsourcing engineering design problems. These seem to be promising ways of supporting the identification of new engineering design problems with inventive benefits once solved.

We examined the trade-off between the cost of response redundancy and the gain in output quality on the popular crowdsourcing platform Mechanical Turk, as a partial replication of Kosinski et al. (2012) who demonstrated a significant improvement in performance by aggregating multiple responses through majority vote. We submitted single items from a validated intelligence test as Human Intelligence Tasks (HITs) and aggregated the responses from “virtual groups” consisting of 1 to 24 workers. While the original study relied on resampling from a relatively small number of responses across a range of experimental conditions, we randomly and independently sampled from a large number of HITs, focusing only on the main effect of group size. We found that – on average – a group of six MTurkers has a collective IQ one standard deviation above the mean for the general population, thus demonstrating a “wisdom of the crowd” effect. The relationship between group size and collective IQ was characterised by diminishing returns, suggesting moderately sized groups provide the best return on investment. We also analysed performance of a smaller subset of workers who had each completed all 60 test items, allowing for a direct comparison between a group’s collective IQ and the individual IQ of its members. This demonstrated that randomly selected groups collectively equalled the performance of the best-performing individual within the group. Our findings support the idea that substantial intellectual capacity can be gained through crowdsourcing, contingent on moderate redundancy built into the task request.

As COVID-19 was declared a health emergency in March 2020, there was immense demand for information about the novel pathogen. This paper examines the clinician-reported impact of Project ECHO COVID-19 Clinical Rounds on clinician learning. Primary sources of study data were Continuing Medical Education (CME) Surveys for each session from the dates of March 24, 2020 to July 30, 2020 and impact surveys conducted in November 2020, which sought to understand participants’ overall assessment of sessions. Quantitative analyses included descriptive statistics and Mann-Whitney testing. Qualitative data were analyzed through inductive thematic analysis. Clinicians rated their knowledge after each session as significantly higher than before that session. 75.8% of clinicians reported they would ‘definitely’ or ‘probably’ use content gleaned from each attended session and clinicians reported specific clinical and operational changes made as a direct result of sessions. 94.6% of respondents reported that COVID-19 Clinical Rounds helped them provide better care to patients. 89% of respondents indicated they ‘strongly agree’ that they would join ECHO calls again.COVID-19 Clinical Rounds offers a promising model for the establishment of dynamic peer-to-peer tele-mentoring communities for low or no-notice response where scientifically tested or clinically verified practice evidence is limited.

The Portable Antiquities of the Netherlands (PAN) is an online system aimed at recording and documenting archaeological finds by the public. Since PAN launched in 2016, it has become an important data contributor to Dutch archaeology, amassing over 100,000 recorded finds. These data, mostly the result of metal detection, enable scholars to gain new insights and policy makers to make more informed decisions. This review describes the context in which PAN was established, along with its current structure and scope, before looking at its different components, including the underlying database and linked data reference collection. In a final section, the article briefly addresses some common issues inherent to public reporting programs and how PAN approaches these issues.

This article proposes the solver-aware system architecting framework for leveraging the combined strengths of experts, crowds and specialists to design innovative complex systems. Although system architecting theory has extensively explored the relationship between alternative architecture forms and performance under operational uncertainty, limited attention has been paid to differences due to who generates the solutions. The recent rise in alternative solving methods, from gig workers to crowdsourcing to novel contracting structures emphasises the need for deeper consideration of the link between architecting and solver-capability in the context of complex system innovation. We investigate these interactions through an abstract problem-solving simulation, representing alternative decompositions and solver archetypes of varying expertise, engaged through contractual structures that match their solving type. We find that the preferred architecture changes depending on which combinations of solvers are assigned. In addition, the best hybrid decomposition-solver combinations simultaneously improve performance and cost, while reducing expert reliance. To operationalise this new solver-aware framework, we induce two heuristics for decomposition-assignment pairs and demonstrate the scale of their value in the simulation. We also apply these two heuristics to reason about an example of a robotic manipulator design problem to demonstrate their relevance in realistic complex system settings.

Learning idiomatic expressions is seen as one of the most challenging stages in second-language learning because of their unpredictable meaning. A similar situation holds for their identification within natural language processing applications such as machine translation and parsing. The lack of high-quality usage samples exacerbates this challenge not only for humans but also for artificial intelligence systems. This article introduces a gamified crowdsourcing approach for collecting language learning materials for idiomatic expressions; a messaging bot is designed as an asynchronous multiplayer game for native speakers who compete with each other while providing idiomatic and nonidiomatic usage examples and rating other players’ entries. As opposed to classical crowd-processing annotation efforts in the field, for the first time in the literature, a crowd-creating & crowd-rating approach is implemented and tested for idiom corpora construction. The approach is language-independent and evaluated on two languages in comparison to traditional data preparation techniques in the field. The reaction of the crowd is monitored under different motivational means (namely, gamification affordances and monetary rewards). The results reveal that the proposed approach is powerful in collecting the targeted materials, and although being an explicit crowdsourcing approach, it is found entertaining and useful by the crowd. The approach has been shown to have the potential to speed up the construction of idiom corpora for different natural languages to be used as second-language learning material, training data for supervised idiom identification systems, or samples for lexicographic studies.

‘Openness’ is one of the key concepts brought forward by postindustrial narratives questioning the modern repartition of roles between industries and customers. In these narratives, citizen participation in design and intellectual property management based on open source principles are the promise of more sustainable production models. In this context, openness in product design and development has been the object of growing interest and experimentation from academia, businesses and grassroots communities. As a result, numerous concepts emerged that attempt to grasp the essence of this phenomenon, unfortunately leading to overlapping, conflicting or speculative depictions. In this article, we share the understanding we gained throughout 6 years of research on Open Design and Open Source Hardware and attempt to make the difference between myths and facts. We depict an enthusiastic but realistic picture of Open Design and Open Source Hardware practices as we could observe them and deliver a structured framework to situate concepts and their differences. From this, we share seven observations leading to seven corresponding research questions and establish a research agenda to stimulate further investigations into this socially relevant and potentially ground-breaking phenomenon.