One way to address this low sample efficiency of reinforcement learning (RL) is to employ human expert demonstrations to speed up the RL process (RL from demonstration or RLfD). The research so far has focused on demonstrations from a single expert. However, little attention has been given to the case where demonstrations are collected from multiple experts, whose expertise may vary on different aspects of the task. In such scenarios, it is likely that the demonstrations will contain conflicting advice in many parts of the state space. We propose a two-level Q-learning algorithm, in which the RL agent not only learns the policy of deciding on the optimal action but also learns to select the most trustworthy expert according to the current state. Thus, our approach removes the traditional assumption that demonstrations come from one single source and are mostly conflict-free. We evaluate our technique on three different domains and the results show that the state-of-the-art RLfD baseline fails to converge or performs similarly to conventional Q-learning. In contrast, the performance level of our novel algorithm increases with more experts being involved in the learning process and the proposed approach has the capability to handle demonstration conflicts well.

This paper holds the material presented at our keynote at the Intelligent Virtual Environments and Virtual Agents Conference, where we analyzed future trends in virtual agent and environment technologies, as applied to the fields of videogames and interactive multimedia. We begin by observing current tendencies in game development projects, both in the market and in the academia, outlining their respective strengths and weaknesses. We follow by describing the aspects of game development that will come to shape the next generation of games, and discuss the requirements for academic projects to achieve the level of quality required to be competitive in this field. In the rest of the paper we analyze the roles, importance, challenges, and research opportunities that a new generation of virtual characters and environments will bring to the world of interactive entertainment.

Among an array of techniques proposed to speed-up reinforcement learning (RL), learning from human demonstration has a proven record of success. A related technique, called Human-Agent Transfer, and its confidence-based derivatives have been successfully applied to single-agent RL. This article investigates their application to collaborative multi-agent RL problems. We show that a first-cut extension may leave room for improvement in some domains, and propose a new algorithm called coordination confidence (CC). CC analyzes the difference in perspectives between a human demonstrator (global view) and the learning agents (local view) and informs the agents’ action choices when the difference is critical and simply following the human demonstration can lead to miscoordination. We conduct experiments in three domains to investigate the performance of CC in comparison with relevant baselines.

One approach to solving planning problems is to compile them to other problems for which powerful off-the-shelf solvers are available; common targets include SAT, CSP, and MILP. Recently, a novel optimization technique has become available: quantum annealing (QA). QA takes as input problem instances of quadratic unconstrained binary optimization (QUBO) problem. Early quantum annealers are now available, though their constraints restrict the types of QUBOs they can take as input. Here, we introduce the planning community to the key steps in compiling planning problems to QA hardware: a hardware-independent step, mapping, and a hardware-dependent step, embedding. After describing two approaches to mapping general planning problems to QUBO, we describe preliminary results from running an early quantum annealer on a parametrized family of hard planning problems. The results show that different mappings can substantially affect performance, even when many features of the resulting instances are similar. We conclude with insights gained from this early study that suggest directions for future work.

This paper advocates a new science of intelligence, one that is holistic, multi-disciplinary, oriented to crucial values as health and well-being and able to contribute to the solution of real-world problems. As a starting point we study the interplay between two research disciplines that until now have been hardly related to each other: Ayurveda and multi-agent systems. We consider some possible results of the cross fertilisation like for instance the application of ayurvedic knowledge to improve the skills of practical reasoning agents.

The Knowledge Engineering Review is an outstanding journal in Computer Science. The guest editors and contributors to this Special Issue are economists. Why is this so? In recent years, there has been a growing dialogue between economists and computer scientists, to our mutual benefit. The Special Issue is devoted to nine papers in which economists survey aspects of the field of agent-based computational economics models, and in some cases report on new findings in several areas of application. As such, we hope it has something to offer both computer scientists and economists.

Video streaming over mobile wireless networks is getting popular in recent years. High video quality relies on large bandwidth provisioning, however, it decreases the number of supported users in wireless networks. Thus, effective bandwidth utilization becomes a crucial issue in wireless network as the bandwidth resource in wireless environment is precious and limited. The NGN quality of service mechanisms should be designed to reduce the impact of traffic burstiness on buffer management. For this reason, we propose an active dropping mechanism to deal with the effective bandwidth utilization in this paper. We use scalable video coding extension of H.264/AVC standard to provide different video quality for users of different levels. In the proposed dropping mechanism, when the network loading exceeds the threshold, the dropping mechanism starts to drop data of the enhancement layers for users of low service level. The dropping probability alters according to the change in network loading. With the dropping mechanism, the base station increases the system capability and users are able to obtain better service quality when the system is under heavy loading. We also design several methods to adjust the threshold value dynamically. By using the proposed mechanism, better quality can be provided when the network is in congestion.

We are interested in understanding how socially desirable traits like sympathy, reciprocity, and fairness can survive in environments that include aggressive and exploitative agents. Social scientists have long theorized about ingrained motivational factors as explanations for departures from self-seeking behaviors by human subjects. Some of these factors, namely reciprocity, have also been studied extensively in the context of agent systems as tools for promoting cooperation and improving social welfare in stable societies. In this paper, we evaluate how other factors like sympathy and parity can be used by agents to seek out cooperation possibilities while avoiding exploitation traps in more dynamic societies. We evaluate the relative effectiveness of agents influenced by different social considerations when they can change who they interact with in their environment using both an experimental framework and a predictive analysis. Such rewiring of social networks not only allows possibly vulnerable agents to avoid exploitation but also allows them to form gainful coalitions to leverage mutually beneficial cooperation, thereby significantly increasing social welfare.

The KADS methodology and its successor, CommonKADS, have gained a reputation for being useful approaches to building knowledge-based systems in a manner which is both systematic and well documented. However, these methods require considerable effort to use them completely. It has been suggested that automated support for KADS or CommonKADS users, in the form of “knowledge engineering workbenches”, could be very useful. These tools would provide computerised assistance to knowledge engineers in organising and representing knowledge, in a similar fashion to the support which CASE tools provide for software engineers. To provide support for KADS or CommonKADS, the workbenches should provide specific support for the modelling techniques recommended by these methods, which are very detailed in the representation and analysis stages of knowledge engineering. A good knowledge engineering workbench should also be easy to use, should be robust and reliable, and should generate output in a presentable format.

This paper reports on an evaluation of two commercially available workbenches for supporting the KADS approach: KADS Tool from ILOG and Open KADS Tool from Bull. This evaluation was carried out by AIAI as part of the CATALYST project, funded by the European Community's ESSI programme, which aimed to introduce CommonKADS to two technology-oriented companies. Information is also presented on two other workbenches: the CommonKADS workbench (which will soon become commercially available) and the VITAL workbench. The results show various strengths and weaknesses in each tool.

In the coming years, scheduling will be an increasingly important activity in the manufacturing and aerospace industries, especially at the US National Aeronautics and Space Administration (NASA). Expert systems have successfully been used to aid in the scheduling function. This paper surveys some of the conflict resolution approaches used in building expert scheduling systems, and then examines the feasibility of using an expert systems approach for scheduling activities and resolving conflicts for experimenters to use NASA Goddard-supported satellites. Recommendations for further reading in expert scheduling systems are given.

Traditional Route Search aims at finding the path that goes through geographical entities that are relevant to the provided search terms from the start point to the end point. Without constraints, traditional Route Search visiting multiple locations is unreliable because locations may close after a specified time. In this paper, time constraint (operating hours of each location) is drawn into Route Search query in order to make the query more realistic. Two methods are proposed in this paper, namely Route Search for fixed locations (RFix) and Route Search for flexible locations (RFlex). These two queries are different from the existing Route Search query because (1) the end point is not pre-defined and (2) time constraint is involved. Our two proposal queries consider whether the locations are specifically pre-defined by the user or only the location types are specified. In each method, two propositions are presented for pruning expansion branches, which highly improves the performance. Our experiments verified the applicability of RFix and RFlex to solve Route Search queries with time constraint queries.

The First International Conference on Autonomous Agents brought together researchers concerned with implementing systems that perceive and act in dynamic, unpredictable environments, that coordinate interoperation among complementary component capabilities, and that perform significant jobs with a high degree of autonomy.

This paper presents a new ‘Language Independent Recommender Agent’ (LIRA), using information distributed over any text-source pair on the Web about candidate items. While existing review-based recommendation systems learn the features of candidate items and users’ preferences, they do not handle varying perspectives of users on those features. LIRA constructs agents for each user, which run regression algorithms on texts from different sources and builds trust relations. The key advantages of LIRA can be listed as: LIRA does not require reviews from target users, LIRA calculates trust values based on prediction accuracy instead of social connections or rating similarity, LIRA does not require the reviews to come from the same community or peer user group. Since ratings of the reviewers are not necessary for LIRA, we can collect and use reviews from different sources (web pages, professional critiques), as long as we know the corresponding item and source of that text. Since LIRA does not combine text from different sources, texts from different sources are not required to be in the same language. LIRA can utilize text from multiple languages, as long as sources are consistent with their language usage.

We propose an approach to knowledge-based decision-model construction. The domain knowledge is represented in the form of multilevel influence diagrams. At each level, a regular influence diagram representing a decision model may be generated and the influence diagrams between the levels are related through a series of operations. We defined eight such operations and show that these are sufficient at the structural level to construct any target influence diagram in any practical situation. We provide a series of examples drawn from different domains on the use of multilevel influence diagrams for model construction. We also propose suggestions on providing users with guidance that can direct the model-construction procedure towards the best target model.

The development of semi-automated and automated ontology alignment techniques is an important part of realizing the potential of the Semantic Web. Until very recently, most existing work in this area was focused on finding simple (1:1) equivalence correspondences between two ontologies. However, many real-world ontology pairs involve correspondences that contain multiple entities from each ontology. These ‘complex’ alignments pose a challenge for existing evaluation approaches, which hinders the development of new systems capable of finding such correspondences. This position paper surveys and analyzes the requirements for effective evaluation of complex ontology alignments and assesses the degree to which these requirements are met by existing approaches. It also provides a roadmap for future work on this topic taking into consideration emerging community initiatives and major challenges that need to be addressed.

The design and assessment of safety critical systems often involves broad and distributed teams of designers, suppliers and analysts who represent diverse areas of expertise and motivations. Accurate and effective communication between these groups is therefore an issue of primary importance. The formalisation of specifications and arguments of safety can be of significant benefit in ensuring the consistency of evidence in such cases, when it must be presented across many domains. However, a formal description of a safety critical system may be unconvincing unless it is presented in a form which is (or forms which are) accessible to the broad range of users and assessors of safety cases. This raises issues of human communication which include the tailoring of information to particular communicative tasks; the efficacy of differing media for communication and the cognitive impact that such differing media have. This paper draws together work in fields of knowledge engineering, knowledge based systems and human communication in an effort to address, from a sound theoretical basis, these and other communication issues raised by the use of formal descriptions in safety critical systems. Further, this paper argues that a primary role for knowledge based systems techniques in safety critical systems is in supporting the communication of information.

This paper introduces the Special Issue of The Knowledge Engineering Review devoted to reports arising from selected Technical Forum Groups at the First and Second AgentLink III Agent Technical Forums, held in 2004 and 2005.

Web-based expert systems (WBESs) provide the benefits of both expert system technology and web technology. The use of web services to deliver functionalities of WBESs allows the integration of these systems in web-portals. WBESs are used in a diversity of areas like engineering, management, medicine, agriculture, education, tourism, finance etc. A study on the various features of WBESs like knowledge-representation, reasoning, languages, implementation tools, use of various web services-related processes such as discovery, selection, composition etc. can result into some interesting conclusions. Presented work tries to fulfill the same purpose. The paper presents various WBESs classified according to their use of domain. The comparisons, observations, and assessments of these systems are presented by emphasizing the above-mentioned features of WBESs. A discussion on different web services-related processes and some representative techniques for performing each has also been provided to clarify their use in the WBESs. On the basis of assessments and surveys from different perspectives, some remarkable conclusions are drawn.

Knowledge-based systems and their ontologies evolve due to different reasons. Ontology evolution is the adaptation of an ontology and the propagation of these changes to dependent artifacts such as queries and other ontologies. Besides identifying basic/simple changes, it is imperative to identify complex changes between two versions of the same ontology to make this adaptation possible. There are many definitions of complex changes applied to ontologies in the literature. However, their specifications across works vary both in formalization and textual description. Some works also use different terminologies to refer to a change, while others use the same vocabulary to refer to distinct changes. Therefore, there is a lack of a unified list of complex changes. The main goals of this paper are: (i) present the primary documents that identify complex changes; (ii) provide critical analyses about the set of the complex changes proposed in the literature and the documents mentioning them; (iii) provide a unified list of complex changes mapping different sets of complex changes proposed by several authors; (iv) present a classification for those complex changes; and (v) describe some open directions of the area. The mappings between the complex changes provide a mechanism to relate and compare different proposals. The unified list is thus a reference for the complex changes published in the literature. It may assist the development of tools to identify changes between two versions of the same ontology and enable the adaptation of artifacts that depend on the evolved ontology.