The problem of traffic congestion incurs numerous social and economical repercussions and has thus become a central issue in every major city in the world. For this work we look at the transportation domain from a multiagent system perspective, where every driver can be seen as an autonomous decision-making agent. We explore how learning approaches can help achieve an efficient outcome, even when agents interact in a competitive environment for sharing common resources. To this end, we consider the route choice problem, where self-interested drivers need to independently learn which routes minimise their expected travel costs. Such a selfish behaviour results in the so-called user equilibrium, which is inefficient from the system’s perspective. In order to mitigate the impact of selfishness, we present Toll-based Q-learning (TQ-learning, for short). TQ-learning employs the idea of marginal-cost tolling (MCT), where each driver is charged according to the cost it imposes on others. The use of MCT leads agents to behave in a socially desirable way such that the is attainable. In contrast to previous works, however, our tolling scheme is distributed (i.e., each agent can compute its own toll), is charged a posteriori (i.e., at the end of each trip), and is fairer (i.e., agents pay exactly their marginal costs). Additionally, we provide a general formulation of the toll values for univariate, homogeneous polynomial cost functions. We present a theoretical analysis of TQ-learning, proving that it converges to a system-efficient equilibrium (i.e., an equilibrium aligned to the system optimum) in the limit. Furthermore, we perform an extensive empirical evaluation on realistic road networks to support our theoretical findings, showing that TQ-learning indeed converges to the optimum, which translates into a reduction of the congestion levels by 9.1%, on average.

Ambient intelligence (AmI) proposes pervasive information systems composed of autonomous agents embedded within the environment who, in orchestration, complement human activity in an intelligent manner. As such, it is an interesting and challenging application area for many computer science fields and approaches. A critical issue in such application scenarios is that the agents must be able to acquire, exchange, and evaluate knowledge about the environment, its users, and their activities. Knowledge populated between the agents in such systems may be contextually dependent, ambiguous, and incomplete. Conflicts may thus naturally arise, that need to be dealt with by the agents in an autonomous way. In this survey, we relate AmI to the area of knowledge representation and reasoning (KR), where conflict resolution has been studied for a long time. We take a look at a number of KR approaches that may be applied: context modelling, multi-context systems, belief revision, ontology evolution and debugging, argumentation, preferences, and paraconsistent reasoning. Our main goal is to describe the state of the art in these fields, and to draw attention of researchers to important theoretical issues and practical challenges that still need to be resolved, in order to reuse the results from KR in AmI systems or similar complex and demanding applications.

This paper presents a parameterized gait generator based on linear inverted pendulum model (LIPM) theory, which allows users to generate a natural gait pattern with desired step sizes. Five types of zero moment point (ZMP) components are proposed for formulating a natural ZMP reference, where ZMP moves continuously during single support phases instead of staying at a fixed point in the sagittal and lateral plane. The corresponding center of mass (CoM) trajectories for these components are derived by LIPM theory. To generate a parameterized gait pattern with user-defined parameters, a gait planning algorithm is proposed, which determines related coefficients and boundary conditions of the CoM trajectory for each step. The proposed parameterized gait generator also provides a concept for users to generate gait patterns with self-defined ZMP references by using different components. Finally, the feasibility of the proposed method is validated by the experimental results with a teen-sized humanoid robot, David, which won first place in the sprint event at the 20th Federation of International Robot-soccer Association (FIRA) RoboWorld Cup.

The concept of holonic systems has its roots in the desire to understand the structure of natural systems (e.g. living organisms and social organisations) and in particular their ability to behave in a stable yet flexible manner in the face of change. It is not surprising that the lessons learned from these natural systems could help with the design and control of complex man-made systems. However, a key issue is, how can one translate holonic concepts to real industrial environments? For example, one of the key holonic concepts, the holon, can be described as a self-contained autonomous and cooperative entity; when deciding how to implement holons, software agents appear to be the logical choice. In this paper, we summarise the presentations and discussions from a workshop held at the recent International Conference on Autonomous Agents that focused on this issue and brought together researchers from both the holonic systems and the multi-agents systems communities.

Mechanical engineering design is a broad subject area covering many topics and bas influences upon many other engineering disciplines and activities. Computer support for mechanical engineering design activity has been in draughting Systems and analysis packages, but there has been little in conceptual design assistance. This paper presents a number of areas of work in which AI techniques and developments are being used, sometimes in conjunction with traditional methods, to improve the support of design. The approaches to design and design Systems are covered, along with some techniques that are used. Specifie design Systems illustrate progress, and integration issues and simultaneous engineering Systems indicate the way research is moving. Finally, discussion of the trends and future topics indicates where and how effort may be applied in the future.

This paper explores the implications of research results in behavioural decision theory on knowledge engineering. Behavioural decision theory, with its performance (versus process) orientation, can tell us a great deal about the validity of human expert knowledge, and when it should be modelled. A brief history of behavioural decision theory is provided. Implications for knowledge elicitation and representation are discussed. An approach to knowledge engineering is proposed that takes into account these implications.

Expert critiquing systems present an interesting and important complement to conventional expert systems. The basic idea is to emphasize the support for the user's own decision making rather than to let the system independently suggest a solution to a given problem. Thus it is assumed that the user initially proposes a decision or course of action. The system then reviews this suggestion relative to known circumstance, tries to evaluate the proposed solution, provides suggestions for improvements, draws attention to possible risks, indicates alternatives, and evaluates their merits, etc. A critiquing system is characterized both by its approach to problem solving and by its style of interaction with the user, as will be detailed below.

Traditional approaches to plan representation have focused on the generation of a sequence of actions and orderings. Knowledge rich models, which incorporate plan rationale, provide benefits to the planning process in a number of ways. The use of rationale in planning is reviewed in terms of causality, dependencies, and decisions. Each dimension addresses practical issues in the planning process, and adds value to the resultant plan. The contribution of this paper is to explore this categorisation, and to motivate the need to explicitly record and represent rationale knowledge for situated, mixed-initiative planning systems.

Recent progress in reinforcement learning (RL) using self-play has shown remarkable performance with several board games (e.g., Chess and Go) and video games (e.g., Atari games and Dota2). It is plausible to hypothesize that RL, starting from zero knowledge, might be able to gradually approach a winning strategy after a certain amount of training. In this paper, we explore neural Monte Carlo Tree Search (neural MCTS), an RL algorithm that has been applied successfully by DeepMind to play Go and Chess at a superhuman level. We try to leverage the computational power of neural MCTS to solve a class of combinatorial optimization problems. Following the idea of Hintikka’s Game-Theoretical Semantics, we propose the Zermelo Gamification to transform specific combinatorial optimization problems into Zermelo games whose winning strategies correspond to the solutions of the original optimization problems. A specially designed neural MCTS algorithm is then introduced to train Zermelo game agents. We use a prototype problem for which the ground-truth policy is efficiently computable to demonstrate that neural MCTS is promising.

The scope of the Technical Forum Group (TFG) on Agents in Bioinformatics (BIOAGENTS) was to inspire collaboration between the agent and bioinformatics communities with the aim of creating an opportunity to propose a different (agent-based) approach to the development of computational frameworks both for data analysis in bioinformatics and for system modelling in computational biology. During the day, the participants examined the future of research on agents in bioinformatics primarily through 12 invited talks selected to cover the most relevant topics. From the discussions, it became clear that there are many perspectives to the field, ranging from bio-conceptual languages for agent-based simulation, to the definition of bio-ontology-based declarative languages for use by information agents, and to the use of Grid agents, each of which requires further exploration. The interactions between participants encouraged the development of applications that describe a way of creating agent-based simulation models of biological systems, starting from an hypothesis and inferring new knowledge (or relations) by mining and analysing the huge amount of public biological data. In this report we summarize and reflect on the presentations and discussions.

Automated modelling is a young research field and is attracting increasingly more attention. It is a cross-disciplinary field involving simulation, modelling, qualitative reasoning, bond graphs and systems dynamics. It is an investigation of the modelling process with the purpose of developing computer tools which will automatically follow modelling principles. In addition, these tools will take into account the details of an application and generate the most appropriate model for the application. Its objective is to develop computer modelling tools which will have perception of model correctness, completeness and appropriateness and can perform modelling automatically. One way to achieve this objective is to introduce well-defined models and automate the process of assembling submodels into models to create well-defined models. This paper reviews the motivation and background behind this new field, its theory and current state of the art, compares existing approaches and discusses the underlying issues. It is hoped that more researchers will become aware of this field and be encouraged to work in it.

Openness is a challenging property that may characterize multi-agent systems (MAS). It refers to their ability to deal with entities leaving and joining agent society over time. This property makes the MAS behaviour complex and difficult to study and analyze, hence the need for a representative model allowing its understanding. In this context, many models were defined in the literature and we propose to classify them into three categories: structural models, functional models and interactional models. The existing models were proposed either for representing structural openness or for modelling functional or interactional ones independently. But, none of them was oriented to represent MAS openness in a global way while considering its three aspects at once. Besides, each one was defined in order to realize a specific objective and in a particular domain of application. In this paper, we propose an evolving KAGR graph. The latter provides a common understanding of openness and unifies its structural, functional and interactional aspects in a generic way. Our model is finally tested and validated on a multi-agent rescue simulator.

Conventional reinforcement learning focuses on problems with single objective. However, many problems have multiple objectives or criteria that may be independent, related, or contradictory. In such cases, multi-objective reinforcement learning is used to propose a compromise among the solutions to balance the objectives. TOPSIS is a multi-criteria decision method that selects the alternative with minimum distance from the positive ideal solution and the maximum distance from the negative ideal solution, so it can be used effectively in the decision-making process to select the next action. In this research a single-policy algorithm called TOPSIS Q-Learning is provided with focus on its performance in online mode. Unlike all single-policy methods, in the first version of the algorithm, there is no need for the user to specify the weights of the objectives. The user’s preferences may not be completely definite, so all weight preferences are combined together as decision criteria and a solution is generated by considering all these preferences at once and user can model the uncertainty and weight changes of objectives around their specified preferences of objectives. If the user only wants to apply the algorithm for a specific set of weights the second version of the algorithm efficiently accomplishes that.

An appropriately designed architecture of a neural network is essential to many realistic pattern-recognition tasks. A choice of just the right number of neurons, and their interconnections, can cut learning costs by orders of magnitude, and still warrant high classification accuracy. Surprisingly, textbooks often neglect this issue. A specialist seeking systematic information will soon realize that relevant material is scattered over diverse sources, each with a different perspective, terminology and goals. This brief survey attempts to rectify the situation by explaining the involved aspects, and by describing some of the fundamental techniques.

Nakamoto proposed a new solution to transact value via the internet. And since 2009, blockchain technology has expanded and diversified. It has, however, proven to be inefficient in the way it achieves its outcomes, especially through the proof of work protocol. Other developers are promoting alternative methods but, as yet, none has superseded proof of work. The competing protocols illuminate a key feature of the blockchain community, namely, its inability to create consensus in a decentralized community. Because of this lack of consensus, the formation of standards is particularly difficult to achieve. At best standards are contested sites, and we examine three such sites where some form of agreement over standards will be essential if blockchain is to evolve successfully. These three sites are blockchain governance, smart contracts, and interoperability of blockchains. We argue that because standards’ formation is a contested and contingent process, the blockchain community will persist in creating difficulties and barriers for itself until it is able to resolve internal conflicts.

This paper presents a methodology that permits to automate binary classification using the minimum possible number of attributes. In this methodology, the success of the binary prediction does not lie in the accuracy of an algorithm but in the evaluation metrics, which give information about the goodness of fit; which is an important factor when the data batch is unbalanced. The proposed methodology assesses the possible biases in identifying one algorithm as the best performer when considering the goodness of fit of an algorithm through evaluation metrics. The dimension of data has been reduced through the cumulative explained variance. Then, the performance of six machine learning classification models has been compared through Matthew correlation coefficient (MCC), area under curve – receiver operating characteristic (ROC-AUC), and area under curve – precision-recall (AUC-PR). The results show graphically and numerically how the evaluation metrics interfere with the most optimal outcome of an algorithm. The algorithms with the best performance in terms of evaluation metrics have been random forest and gradient boosting. In the imbalanced datasets, MCC has provided better prediction results than ROC-AUC or AUC-PR. The proposed methodology is adapted to the case of bankruptcy prediction.

Designing realistic multi-agent systems is a complex process, which involves specifying not only the functionality of individual agents, but also the authority relationships and lines of communication existing among them. In other words, designing a multi-agent system refers to designing an agent organisation. Existing methodologies follow a wide variety of approaches to designing agent organisations, but they do not provide adequate support for the decisions involved in moving from analysis to design. Instead, they require designers to make ad hoc design decisions while working at a low level of abstraction.

We have developed RAMASD (Role Algebraic Multi-Agent System Design), a method for semi-automatic design of agent organisations based on the concept of role models as first-class design constructs. Role models represent agent behaviour, and the design of the agent system is done by systematically allocating roles to agents. The core of this method is a formal model of basic relations between roles, which we call role algebra. The semantics of this role-relationships model are formally defined using a two-sorted algebra.

In this paper, we review existing agent system design methodologies to highlight areas where further work is required, describe how our method can address some of the outstanding issues and demonstrate its application to a case study involving telephone repair service teams.

This paper deals with the implications of showing the storyteller’s perspective when telling stories in virtual environments. The paper proposes an open and reusable architecture for the construction of virtual guides who tell stories about the worlds they inhabit from their own perspective. The system consists of two main components: the structure of a knowledge base to code the narrative knowledge about the worlds and a novel hybrid algorithm for the generation of narratives showing the storyteller’s viewpoint. The complete architecture has been developed. Furthermore, we carried out a study with users in order to both validate the implemented storytelling system and test whether the users consider it significantly better or worse (in terms of several aspects related to believability) adding the virtual guide perspective (as both text and animations showing emotions) to the guide discourse. The interesting results are discussed.

Repeated games are an important mathematical formalism to model and study long-term economic interactions between multiple self-interested parties (individuals or groups of individuals). They open attractive perspectives in modeling long-term multiagent interactions. This overview paper discusses the most important results that actually exist for repeated games. These results arise from both economics and computer science. Contrary to a number of existing surveys of repeated games, most of which originated from the economic research community, we are first to pay a special attention to a number of important distinctive features proper to artificial agents. More precisely, artificial agents, as opposed to the human agents mainly aimed by the economic research, are usually bounded whether in terms of memory or performance. Therefore, their decisions have to be based on the strategies defined using finite representations. Furthermore, these strategies have to be efficiently computed or approximated using a limited computational resource usually available to artificial agents.

Potential-based reward shaping is a commonly used approach in reinforcement learning to direct exploration based on prior knowledge. Both in single and multi-agent settings this technique speeds up learning without losing any theoretical convergence guarantees. However, if speed ups through reward shaping are to be achieved in multi-agent environments, a different shaping signal should be used for each context in which agents have a different subgoal or when agents are involved in a different interaction situation.

This paper describes the use of context-aware potential functions in a multi-agent system in which the interactions between agents are sparse. This means that, unknown to the agents a priori, the interactions between the agents only occur sporadically in certain regions of the state space. During these interactions, agents need to coordinate in order to reach the global optimal solution.

We demonstrate how different reward shaping functions can be used on top of Future Coordinating Q-learning (FCQ-learning); an algorithm capable of automatically detecting when agents should take each other into consideration. Using FCQ-learning, coordination problems can even be anticipated before the actual problems occur, allowing the problems to be solved timely. We evaluate our approach on a range of gridworld problems, as well as a simulation of air traffic control.