In the framework of fully cooperative multi-agent systems, independent (non-communicative) agents that learn by reinforcement must overcome several difficulties to manage to coordinate. This paper identifies several challenges responsible for the non-coordination of independent agents: Pareto-selection, non-stationarity, stochasticity, alter-exploration and shadowed equilibria. A selection of multi-agent domains is classified according to those challenges: matrix games, Boutilier's coordination game, predators pursuit domains and a special multi-state game. Moreover, the performance of a range of algorithms for independent reinforcement learners is evaluated empirically. Those algorithms are Q-learning variants: decentralized Q-learning, distributed Q-learning, hysteretic Q-learning, recursive frequency maximum Q-value and win-or-learn fast policy hill climbing. An overview of the learning algorithms’ strengths and weaknesses against each challenge concludes the paper and can serve as a basis for choosing the appropriate algorithm for a new domain. Furthermore, the distilled challenges may assist in the design of new learning algorithms that overcome these problems and achieve higher performance in multi-agent applications.

Nowadays, there are a lot of techniques and tools for addressing the ontology matching problem; however, the complex nature of this problem means that the existing solutions are unsatisfactory. This work intends to shed some light on a more flexible way of matching ontologies using ontology meta-matching. This emerging technique selects appropriate algorithms and their associated weights and thresholds in scenarios where accurate ontology matching is necessary. We think that an overview of the problem and an analysis of the existing state-of-the-art solutions will help researchers and practitioners to identify the most appropriate specific features and global strategies in order to build more accurate and dynamic systems following this paradigm.

The Knowledge Engineering Review is an outstanding journal in Computer Science. The guest editors and contributors to this Special Issue are economists. Why is this so? In recent years, there has been a growing dialogue between economists and computer scientists, to our mutual benefit. The Special Issue is devoted to nine papers in which economists survey aspects of the field of agent-based computational economics models, and in some cases report on new findings in several areas of application. As such, we hope it has something to offer both computer scientists and economists.

As acknowledged by the SESAR (Single European Sky ATM (Air Traffic Management) Research) program, current Air Traffic Control (ATC) systems must be drastically improved to accommodate the predicted traffic growth in Europe. In this context, the Episode 3 project aims at assessing the performance of new ATM concepts, like 4D-trajectory planning and strategic deconfliction.

One of the bottlenecks impeding ATC performances is the hourly capacity constraints defined on each en-route ATC sector to limit the rate of aircraft. Previous works were mainly focused on optimizing the current ground holding slot allocation process devised to satisfy these constraints. We propose to estimate the cost of directly solving all conflicts in the upper airspace with ground holding, provided that aircraft were able to follow their trajectories accurately.

We present a Constraint Programming model of this large-scale combinatorial optimization problem and the results obtained with the FaCiLe (Functional Constraint Library). We study the effect of uncertainties on the departure time and estimate the cost of improving the robustness of our solutions with the Complete Air Traffic Simulator (CATS). Encouraging results were obtained without uncertainty but the costs of robust solutions are prohibitive. Our approach may however be improved, for example, with a prior flight level allocation and the dynamic resolution of remaining conflicts with one of CATS’ modules.

Although they flow from a common source, the uses of multi-agent systems (or ‘agent-based computational systems’––ACE) vary between the social sciences and computer science. The distinction can be broadly summarized as analysis versus synthesis, or explanation versus design. I compare and contrast these uses, and discuss sufficiency and necessity in simulations in general and in multi-agent systems in particular, with a computer science audience in mind.

In recent years, digital content of three-dimensional (3D) scientific objects has become widespread and is made available in a plethora of on-line scientific repositories. A systematic and formal approach becomes necessary to represent the knowledge/information related to these objects, in order to facilitate their search, retrieval and reuse. In this paper, we employ semantic interoperability to manage 3D scientific object content in a large-scale framework of scientific applications that consists of 3D shape models, associated tools and resources. We use an advanced ontological organization for their metadata, powerful reasoning engines for their search and retrieval and we elucidate several crucial issues in the design and implementation of 3D knowledge-based management systems.

This work provides a general overview of the statistical machine translation (SMT) scientific field, which is a subfield of machine translation (MT). Specifically, this paper focuses on one of the most popular SMT approaches, that is, the phrase-based system.

The phrase-based translation units are typically extracted using statistical criteria, and they are weighted using different models. These models are log-linearly combined in the decoding, which is in charge of choosing the most probable translation. Significant quality improvements have been produced from original phrase-based SMT systems. Among others, the main challenges are reordering, domain adaptation and evaluation.

In this paper we provide a brief overview of the main characteristics of agent-based computational economics. We discuss its points of strength, with respect to analytical models, and its weaknesses. The latter are mainly related to how the results of a simulation model can be interpreted, and how the structural parameters of the model can be estimated. We then show how these problems can be dealt with.

This paper describes and compares the most relevant approaches for associating tags with semantics in order to make explicit the meaning of those tags. We identify a common set of steps that are usually considered across all these approaches and frame our descriptions according to them, providing a unified view of how each approach tackles the different problems that appear during the semantic association process. Furthermore, we provide some recommendations on (a) how and when to use each of the approaches according to the characteristics of the data source, and (b) how to improve results by leveraging the strengths of the different approaches.

Recent discoveries in automated planning are broadening the scope of planners, from toy problems to real applications. However, applying automated planners to real-world problems is far from simple. On the one hand, the definition of accurate action models for planning is still a bottleneck. On the other hand, off-the-shelf planners fail to scale-up and to provide good solutions in many domains. In these problematic domains, planners can exploit domain-specific control knowledge to improve their performance in terms of both speed and quality of the solutions. However, manual definition of control knowledge is quite difficult. This paper reviews recent techniques in machine learning for the automatic definition of planning knowledge. It has been organized according to the target of the learning process: automatic definition of planning action models and automatic definition of planning control knowledge. In addition, the paper reviews the advances in the related field of reinforcement learning.

This paper considers the problem of generating conflict-free movement schedules for a set of vehicles that are operating simultaneously in a common airspace. In both civilian air traffic management and military air campaign planning contexts, it is crucial that the movements of different vehicles be coordinated so as to avoid collisions and near misses. Our approach starts from a view of airspace management as a 4D resource allocation problem, where the space in which vehicles must maneuver is itself managed as a capacitated resource. We introduce a linear octree representation of airspace capacity to index vector-based vehicle routes and efficiently detect regions of potential conflict. Generalizing the notion of contention-based search heuristics, we next define a scheduling algorithm that proceeds by first solving a relaxed version of the problem to construct a spatial capacity profile (represented as an octree), and then using spatio-temporal regions where demand exceeds capacity to make conflict-avoiding vehicle routing and scheduling decisions. We illustrate the utility of this basic representation and search algorithm in two ways. First, to demonstrate the overall viability of the approach, we present experimental results using data representing a realistically sized air campaign planning domain. Second, we define a more abstract notion of ‘encounter set’, which tolerates some amount of conflict on the assumption that on-board deconfliction processes can take appropriate avoidance maneuvers at execution time, and show that generation of this more abstract form of predictive guidance can be obtained without loss in computational efficiency.

The European air traffic flow management problem poses particular challenges on optimization technology as it requires detailed modelling and rapid online optimization capabilities. Constraint programming proved successful in addressing these challenges for departure time slot allocation by offering fine-grained modelling of resource constraints and fast allocation through heuristic-repair strategies.

In this paper, we model knowledge dynamics in agents’ belief bases in a collaborative multi-agent system (MAS). Four change operators are introduced: expansion, contraction, prioritized revision, and non-prioritized revision. For all of them, both constructive definitions and an axiomatic characterization by representation theorems are given. We formally justify minimal change, consistency maintenance, and non-prioritization principles. These operators are based on an epistemic model for multi-source belief revision in which a rational way to weigh the beliefs using a credibility order among agents is developed. The defined operators can be seen as skills added to the agents improving the collective reasoning of a MAS.

Agent-based models are often described as bottom-up because macro-level phenomena emerge from the micro-level interactions of agents. These macro-level phenomena include fixed points, cycles, dynamic patterns, and long transients. In this paper, I explore the link between micro-level characteristics—learning rules, diversity, network structure, and externalities—and the macro-level patterns they produce. I focus on why we need agent-level modeling, on how these models produce emergent phenomenon, and on how agent-based models help understand outcomes of social systems in a way that differs from the analytic, equilibrium approach.

Today's organizations require techniques for automated transformation of their large data volumes into operational knowledge. This requirement may be addressed by using event recognition systems that detect events/activities of special significance within an organization, given streams of ‘low-level’ information that is very difficult to be utilized by humans. Consider, for example, the recognition of attacks on nodes of a computer network given the Transmission Control Protocol/Internet Protocol messages, the recognition of suspicious trader behaviour given the transactions in a financial market and the recognition of whale songs given a symbolic representation of whale sounds. Various event recognition systems have been proposed in the literature. Recognition systems with a logic-based representation of event structures, in particular, have been attracting considerable attention, because, among others, they exhibit a formal, declarative semantics, they have proven to be efficient and scalable and they are supported by machine learning tools automating the construction and refinement of event structures. In this paper, we review representative approaches of logic-based event recognition and discuss open research issues of this field. We illustrate the reviewed approaches with the use of a real-world case study: event recognition for city transport management.

In recent years, a number of contributions have argued that monetary—and, more generally, economic—policy is finally becoming more of a science. According to these authors, policy rules implemented by central banks are nowadays well supported by a theoretical framework (the New Neoclassical Synthesis) upon which a general consensus has emerged in the economic profession. In other words, scientific discussion on economic policy seems to be ultimately confined to either fine-tuning this ‘consensus’ model, or assessing the extent to which ‘elements of art’ still exist in the conduct of monetary policy. In this paper, we present a substantially opposite view, rooted in a critical discussion of the theoretical, empirical, and political-economy pitfalls of the neoclassical approach to policy analysis. Our discussion indicates that we are still far from building a science of economic policy. We suggest that a more fruitful research avenue to pursue is to explore alternative theoretical paradigms, which can escape the strong theoretical requirements of neoclassical models (e.g. equilibrium, rationality, etc.). We briefly introduce one of the most successful alternative research projects—known in the literature as agent-based computational economics (ACE)—and we present the way it has been applied to policy analysis issues. We conclude by discussing the methodological status of ACE, as well as the (many) problems it raises.

Safety-critical software must be thoroughly verified before being exploited in commercial applications. In particular, any TCAS (Traffic Alert and Collision Avoidance System) implementation must be verified against safety properties extracted from the anti-collision theory that regulates the controlled airspace. This verification step is currently realized with manual code reviews and testing. In our work, we explore the capabilities of Constraint Programming for automated software verification and testing. We built a dedicated constraint solving procedure that combines constraint propagation with Linear Programming to solve conditional disjunctive constraint systems over bounded integers extracted from computer programs and safety properties. An experience we made on verifying a publicly available TCAS component implementation against a set of safety-critical properties showed that this approach is viable and efficient.