Learning and Adaption in Multi-Agent Systems: First International Workshop, LAMAS 2005, Utrecht, The Netherlands, July 25, 2005, Revised Selected Papers (Lecture Notes in Computer Science)

This book constitutes the thoroughly refereed post-proceedings of the First International Workshop on Learning and Adaption in Multi-Agent Systems, LAMAS 2005, held in Utrecht, The Netherlands, in July 2005, as an associated event of AAMAS 2005. The 13 revised papers presented together with two invited talks were carefully reviewed and selected from the lectures given at the workshop. The papers increase awareness and interest in adaptive agent research, encourage collaboration between machine learning experts and agent system experts, and give a representative overview of current research in the area of adaptive agents. Front matter......Page 1 Introduction......Page 9 Setting......Page 11 Types of Games......Page 12 Repeated Play......Page 14 Cooperative MASs......Page 17 Team Learning......Page 18 Concurrent Learning......Page 19 Credit Assignment......Page 20 The Dynamics of Learning......Page 21 Learning and Communication......Page 22 Preamble......Page 23 Design of Adaptive Software Agents for Market-Based Multi-agent Games......Page 24 MARL......Page 29 Contributions of This Book......Page 34 Open Research Issues......Page 37 Introductory Notions from (Evolutionary) Game Theory......Page 49 Strategic Games......Page 50 Minimax and Maximin......Page 51 Evolutionary Stable Strategies......Page 52 Population Dynamics......Page 53 Introduction......Page 55 Related Work......Page 56 Continuous Area Sweeping Formulation......Page 58 Learning the Expected Reward......Page 59 Choosing Actions......Page 61 Correctness of the Approach......Page 63 Multi-robot Continuous Area Sweeping......Page 64 Single Robot Results......Page 67 Single-Robot Configuration $I$ with a Real Robot......Page 68 Single-Robot Configuration $II$ with a Real Robot......Page 69 Single-Robot Simulation......Page 70 Multi-robot Configuration $I$ in Simulation......Page 73 Multi-robot Configuration $II$ in Simulation......Page 75 Conclusions......Page 77 Introduction......Page 79 Basic LA Schemes......Page 80 Illustration of LA Behavior......Page 82 Some Convergence Properties More Formally......Page 84 Automata Games Games: Single Stage, Multi-agent......Page 85 The Solution Concept......Page 86 Update Rule for Interconnected LA......Page 87 Interconnected LA and Their Relationship to Ant Colony Optimization......Page 88 Markov Games: Multi-stage, Multi Agent......Page 90 Conclusion......Page 92 Introduction......Page 94 Related Work......Page 97 Game Protocol and Learners......Page 98 Learners......Page 99 Results on the Testbed......Page 101 Results on Randomly Generated Matrices......Page 105 Conclusion and Future Work......Page 106 Introduction......Page 108 Multiagent Reinforcement Learning......Page 109 Related Work......Page 110 Our Approach: ReDVaLeR with Variable $ igma$......Page 111 Analysis of RV_$ igma(t)$......Page 112 Convergence Against Eventually Stationary Opponents......Page 113 No-Regret Property......Page 115 Convergence in Self-play......Page 116 Conclusion......Page 121 Introduction......Page 123 Social Attachments and Networks......Page 124 Description of the Information Sharing Scenarios......Page 125 Scenarios Without Recommendation......Page 128 Scenarios with Recommendation......Page 132 Final Remarks About All Scenarios......Page 134 Conclusions and Future Work......Page 135 Introduction......Page 137 Desired Properties......Page 138 The Reservation System......Page 139 Communication Protocol......Page 140 Learning Opportunities for the Intersection Manager......Page 141 Vehicles with Priorities......Page 142 Learning Opportunities for the Driver Agent......Page 143 Making Better Reservations......Page 144 Conclusion......Page 145 Introduction......Page 147 Decision Making in a Stochastic World......Page 148 Q-Learning......Page 149 The Agent......Page 150 The Learning Algorithm......Page 154 Cooperative Learning in Environments with Errors......Page 155 Related Work......Page 159 Conclusions and Future Work......Page 160 Introduction......Page 163 Related Work......Page 164 Tags in Prisoner's Dilemma and Related Games......Page 165 Effect of Tags Where Cooperation $\neq$ Mimicry......Page 169 Conclusion......Page 171 Introduction......Page 173 The Firm Model......Page 174 XCS and Adaptive Firms......Page 175 Exploration Techniques......Page 176 Wilson Techniques......Page 177 Experimental Results......Page 178 Exploration Techniques......Page 179 Exploration-Exploitation Techniques......Page 180 Conclusion......Page 183 Introduction......Page 185 Rover Reward Function Properties......Page 186 Difference Reward Functions......Page 187 Continuous Rover Problem......Page 188 Rover Capabilities......Page 189 Rover Policy Selection......Page 190 Learning Control Strategies in a Collective......Page 191 Results......Page 192 Learning Rates......Page 193 Scaling Properties......Page 194 Learning with Communication Limitations......Page 195 Leaning in Changing Episodic Environment......Page 197 Discussion......Page 198 Introduction......Page 200 Reinforcement Learning......Page 202 Relational Multi-agent Reinforcement Learning......Page 203 Complexity Factors......Page 204 Settings......Page 205 Experimental Setup......Page 208 Experimental Results......Page 210 Conclusions......Page 212 Backup Trees......Page 215 Ant Colony Optimization......Page 217 Multi-type ACO......Page 218 Our Algorithm......Page 219 Experimental Results......Page 220 The Complete Algorithm......Page 221 Conclusion......Page 222 Back matter......Page 224 An Overview Of Cooperative And Competitive Multiagent Learning / Pieter Jan't Hoen [and Others] -- Multi-robot Learning For Continuous Area Sweeping / Mazda Ahmadi And Peter Stone -- Learning Automata As A Basis For Multi Agent Reinforcement Learning / Ann Nowe, Katja Verbeeck And Maarten Peeters -- Learning Pareto-optimal Solutions In 2x2 Conflict Games / Stephane Airiau And Sandip Sen -- Unifying Convergence And No-regret In Multiagent Learning / Bikramjit Banerjee And Jing Peng -- Implicit Coordination In A Network Of Social Drivers : The Role Of Information In A Commuting Scenario / Ana L.c. Bazzan, Manuel Fehler And Franziska Klugl -- Multiagent Traffic Management: Opportunities For Multiagent Learning / Kurt Dresner And Peter Stone -- Dealing With Errors In A Cooperative Multi-agent Learning System / Constanca Oliveira E Sousa And Luis Custodio -- The Success And Failure Of Tag-mediated Evolution Of Cooperation / Austin Mcdonald And Sandip Sen -- An Adaptive Approach For The Exploration-exploitation Dilemma And Its Application To Economic Systems / Lilia Rejeb, Zahia Guessoum And Rym M'hallah -- Efficient Reward Functions For Adaptive Multi-rover Systems / Kagan Tumer And Adrian Agogino -- Multi-agent Relational Reinforcement Learning Explorations In Multi-state Coordination Tasks / Tom Croonenborghs, Karl Tuyls, Jan Ramon And Maurice Bruynooghe -- Multi-type Aco For Light Path Protection / Peter Vrancx, Ann Nowe And Kris Steenhaut. Karl Tuyls ... [et Al.] (eds.). This Book Contains Selected And Revised Papers Of The International Workshop On Learning And Adaptation In Multi-agent Systems (lamas 2005), Held At The Aamas 2005 Conference--pref. Includes Bibliographical References And Index. This book contains selected and revised papers of the International Workshop on Lea- ing and Adaptation in Multi-Agent Systems (LAMAS 2005), held at the AAMAS 2005 Conference in Utrecht, The Netherlands, July 26. An important aspect in multi-agent systems (MASs) is that the environment evolves over time, not only due to external environmental changes but also due to agent int- actions. For this reason it is important that an agent can learn, based on experience, and adapt its knowledge to make rational decisions and act in this changing environment autonomously. Machine learning techniques for single-agent frameworks are well established. Agents operate in uncertain environments and must be able to learn and act - tonomously. This task is, however, more complex when the agent interacts with other agents that have potentially different capabilities and goals. The single-agent case is structurally different from the multi-agent case due to the added dimension of dynamic interactions between the adaptive agents. Multi-agent learning, i.e., the ability of the agents to learn how to cooperate and compete, becomes crucial in many domains. Autonomous agents and multi-agent systems (AAMAS) is an emerging multi-disciplinary area encompassing computer science, software engineering, biology, as well as cognitive and social sciences. A t- oretical framework, in which rationality of learning and interacting agents can be - derstood, is still under development in MASs, although there have been promising?rst results.