Evolving Evil: Optimizing Flocking Strategies through Genetic Algorithms for the Ghost Team in the Game of Ms. Pac-Man

by Federico Liberatore, Antonio Mora, Pedro Castillo, Juan Julián Merelo in EvoGAMES
Flocking strategies are sets of behavior rules for the interaction of agents that allow to devise controllers with reduced complexity that generate emerging behavior. In this paper, we present an application of genetic algorithms and flocking strategies to control the Ghost Team in the game Ms. Pac-Man. In particular, we define flocking strategies for the Ghost Team and optimize them for robustness with respect to the stochastic elements of the game and effectivity against different possible opponents by means of genetic algorithm. 
The performance of the methodology proposed is tested and compared with that of other standard controllers belonging to the framework of the Ms. Pac-Man versus Ghosts Competition. The results show that flocking strategies are capable of modelling complex behaviors and produce effective and challenging agents. 

The presentation is:

 

You can also see a brief demo here (we are the ghosts :D):

Enjoy it!

(And cite us, of course :D)

Hackathon in Videogames at EVO* 2014

Imagen

Hi to all,

Finally, the EVOHackathon will be held in the Oficina de Software Libre on Tuesday 22 April (one day before EVOGames conference).

There are 5 projects confirmed right now,namely:

  • Creating autonomous agents for Super Mario Bros. game
  • Creating an AI engine for the game Wetland (Greyman Studios)
  • Creating bots for 1 vs 1 combats in the RTS Planet Wars
  • Procedural generation of stages for a new game (Greyman Studios)
  • Progamer: Code visualization tool based in Super Mario Bros. levels

As you can see, two of them are proposed and will be directed by a videogames company.

We invite you to join us. It is free! :D

Sistemas Clasificadores

Los sistemas clasificadores son una fusión entre los algoritmos evolutivos, el aprendizaje por refuerzo y el supervisado. Se conocen como Learning Classifier Systems. El viernes pasado aproveché la reunión del grupo para presentar una breve revisión histórica y dar detalles sobre quizá el algoritmo más importante introducido en este campo, el eXtended Classifier System o XCS de Wilson.

Básicamente, el algoritmo busca mediante evolución genética y aprendizaje un conjunto de reglas que modelen la solución a un problema donde existe recompensa. Las reglas se componen de una condición y una acción. La población de reglas representa para cualquier condición dada, cual será la mejor acción. Esto se consigue asociando al espacio de entrada una predicción de la mejor recompensa futura obtenida para cada acción posible.

Entonces, dado un estado que representa el entorno, se buscan las reglas cuya condición coincide, y de ellas se toma la acción que ofrece mejor recompensa futura.

La tarea no es fácil, los algoritmos formales de aprendizaje por refuerzo, necesitan a priori un conocimiento determinista de las posibles entradas y las transiciones resultantes de las acciones, dejando poco o nada para la búsqueda y aplicación de generalización.

Con XCS este problema se resuelve introduciendo algunos ajustes a la componente genética. La idea general es básicamente repartir los recursos (reglas) para que representen todo el espacio con la mayor precisión y generalización posible. Como no es algo que se pueda resumir en unas pocas líneas, aquí os dejo la presentación:

Modelando el conocimiento de un experto en Unreal Tournament (CEDI 2013)

En concreto, hemos presentado el artículo “Modelling Human Expert Behaviour in an Unreal Tournament 2004 Bot” dentro del Primer Simposio Español en Entretenimiento Digital, incluido dentro del CEDI 2013.

Y vosotros diréis, ¿por qué un artículo en inglés en un congreso español?. Pues porque los artículos en inglés que sean seleccionados podrán enviarse a un número especial de la revista Entertainment Computing (Elsevier). A ver si hay suerte. :D

El trabajo presenta el diseño de un bot (jugador autónomo) para jugar a Unreal Tournament 2004 (UT2K4). Dicho bot ha sido creado por Francisco Aisa y Ricardo Caballero, modelando el conocimiento y comportamiento de un jugador experto en dicho juego (el primero de ellos ;D).

La presentación podéis verla en:

Que la disfrutéis (y nos citéis, claro). :D

Saludos.

Unreal Expert Bots at IWANN 2013

Last week there was held IWANN 2013 at Tenerife, an international conference mainly devoted to researches inside the neural networks scope. In it, Antonio Fernández Leiva, Raúl Lara and Me organized the Special Session on Artificial Intelligence and Games.

There were five works in the session, one of them “Designing and Evolving an Unreal Tournament— 2004 Expert Bot“.

It describes the designing and improvement, through off-line (not during the game) evolution, of an autonomous agent (or bot) for playing the game Unreal Tournament 2004. This was created by means of a finite state machine which models the expert behaviour of a human player in 1 vs 1 deathmatch mode, following the rules of the international competition.

Then, the bot was improved by means of a Genetic Algorithm, yielding an agent that is, in turn a very hard opponent for the medium-level human player and which can (easily) beat the default bots in the game, even in the maximum difficulty level.

The presentation can be seen at:

Moreover, you can watch one example of the evolution in the following video:

Finally, the Unreal Expert and Genetic bot’s source code are available at https://github.com/franaisa/ExpertAgent

Enjoy them. ;)

Super Mario autonomous agents at LION 2013

Recently, inside the last LION 7 (2013) conference (Special Session on Games and Computational Intelligence) there was presented the paper entitled “FSM-Based Agents for Playing Super Mario Game”.

It describes the implementation and test of an autonomous agent which can play Super Mario game better than an expert user can do (in some trained levels).
It is build starting from a Finite State Machine and applying an Evolutionary Algorithm.

The presentation is:

You can watch one example of the obtained agent playing a game here:

Enjoy it. ;)

Science and Videogames Tutorial

Last 16th November, inside the GAME-ON 2012 Conference, I presented (with Antonio Fernández Leiva) a tutorial entitled “Computational Intelligence applied to videogames; past, present and future”.

It was a two parts tutorial, being the first one (mine) devoted to introduce the relationship between science and videogames, describing the integration of Computational Intelligence in this scope.

My part presentation can be seen here:

Enjoy it! :D