Master of Evolution! Using Genetic Algorithms to generate decks for the game HearthStone

This September we attended to the IEEE CIG 2017 Conference in Santorini, Greece, to present the paper “Evolutionary Deckbuilding in HearthStone”. This paper was written in collaboration with our colleagues Alberto Tonda and Giovanni Squillero.

The story of this paper started a (not so) long time ago while me and Alberto were discussing about how awesome HearthStone is. Suddenly, we thought about how easy it would be to create the constraints for the uGP framework, and that there were some open source simulators of the game. In a while, we already had the constraints, the simulator adapted to accept individuals from uGP, and some experiments running.

And we finished the paper after, of course.

You can download the paper draft from here (the electronic original version is not available yet).

And here is the presentation:

The abstract:

—One of the most notable features of collectible card games is deckbuilding, that is, defining a personalized deck before the real game. Deckbuilding is a challenge that involves a big and rugged search space, with different and unpredictable behaviour
after simple card changes and even hidden information. In this paper, we explore the possibility of automated deckbuilding: a genetic algorithm is applied to the task, with the evaluation delegated to a game simulator that tests every potential deck against a varied and representative range of human-made decks.
In these preliminary experiments, the approach has proven able to create quite effective decks, a promising result that proves that, even in this challenging environment, evolutionary algorithms can find good solutions.

Starcraft GP nominated to the HUMIES award

This year we participated in the HUMIES awards with our paper “Towards Automatic StarCraft Strategy Generation Using Genetic Programming“, accepted at CIG2015, wrote in collaboration with Politecnico di Torino and INRA. Our paper was elected from 28 candidates to be part of the 8 finalists, so we can consider it a great achievement. Although we didn’t won, because the astounding quality of the other works, we are thrilled about our nomination :)

Here is the presentation. It even includes a reference to Starship Troopers!

Towards automatic StarCraft strategy generation using genetic programming

I forgot to mention that we published our paper “Towards automatic StarCraft strategy generation using genetic programming” in CIG 2015 conference, held in Taiwan. This was a work made in collaboration with Alberto Tonda (INRA) and Giovanni Squillero (Politecnico di Torino), starting a new research line using this game (and also, starting other nice collaborations that are still a secret!)

The abstract:

Among Real-Time Strategy games few titles have enjoyed the continued success of StarCraft. Many research lines aimed at developing Artificial Intelligences, or “bots”, capable of challenging human players, use StarCraft as a platform. Several characteristics make this game particularly appealing for researchers, such as: asymmetric balanced factions, considerable complexity of the technology trees, large number of units with unique features, and potential for optimization both at the strategical and tactical level. In literature, various works exploit evolutionary computation to optimize particular aspects of the game, from squad formation to map exploration; but so far, no evolutionary approach has been applied to the development of a complete strategy from scratch. In this paper, we present the preliminary results of StarCraftGP, a framework able to evolve a complete strategy for StarCraft, from the building plan, to the composition of squads, up to the set of rules that define the bot’s behavior during the game. The proposed approach generates strategies as C++ classes, that are then compiled and executed inside the OpprimoBot open-source framework. In a first set of runs, we demonstrate that StarCraftGP ultimately generates a competitive strategy for a Zerg bot, able to defeat several human-designed bots.

Do you want to know more? Download the paper draft or electronic version in IEEE web.

Evostar 2015 mandatory post

We can never skip the chance to assist the Evostar conference, and aside learn the latest trends in Evolutionary Computation and present our results, we also have a good time with our other colleagues.

This time the conference was held in Copenhagen (Denmark), and because Antonio and me were part of the organization we didn’t have much time to go sightseeing, but we went to Tivoli Gardens and ride the flying chairs (and screamed like babies).

On the scientific part, we presented two papers to EvoGames track, related with our research lines on content generation for videogames and AI optimization. The first paper, How the World Was MADE: Parametrization of Evolved Agent-Based Models for Backstory Generation, presents a study on parametrization of the values that define a virtual world to facilitate the emergence of archetypes, and be able to generate interesting backstories (for videogames, for example). See the poster here:

The poster

Also, as we are commited to open science and open software, you can download the MADE environment from its web. The abstract:

Generating fiction environments for a multi-agent system optimized by genetic algorithms (with some specific requirements related to the desirable plots), presents two main problems: first it is impossible to know in advance the optimal value for the particular designed fitness function, and at the same time, it creates a vast search space for the parameters that it needs. The purpose of this paper is to define a methodology to find the best parameter values for both, the evolutionary algorithm, and the own fictional world configuration. This design includes running, to completion, a world simulation represented as a chromosome, and assigning a fitness to it, thus composing a very complex fitness landscape.
In order to optimize the resources allocated to evolution and to have some guarantees that the final result will be close to the optimum, we systematically analyze a set of possible values of the most relevant parameters, obtaining a set of generic rules. These rules, when applied to the plot requisites, and thus, to the fitness function, will lead to a reduced range of parameter values that will help the storyteller to create optimal worlds with a reduced computation budget.

Evostar 2015 - Copenhagen(That’s me with the IKEA rat plushies I used to describe our system)

Our other paper, It’s Time to Stop: A Comparison of Termination Conditions in the Evolution of Game Bots, describes a methodology to compare different termination conditions in noisy environments such as the RTS games. The abstract:

Evolutionary Algorithms (EAs) are frequently used as a mechanism for the optimization of autonomous agents in games (bots), but knowing when to stop the evolution, when the bots are good enough, is not as easy as it would a priori seem. The first issue is that optimal bots are either unknown (and thus unusable as termination condition) or unreachable. In most EAs trying to find optimal bots fitness is evaluated through game playing. Many times it is found to be noisy, making its use as a termination condition also complicated. A fixed amount of evaluations or, in the case of games, a certain level of victories does not guarantee an optimal result. Thus the main objective of this paper is to test several termination conditions in order to find the one that yields optimal solutions within a restricted amount of time, and that allows researchers to compare different EAs as fairly as possible. To achieve this we will examine several ways of finishing an EA who is finding an optimal bot design process for a particular game, Planet Wars in this case, with the characteristics described above, determining the capabilities of every one of them and, eventually, selecting one for future designs.

Evostar 2015 - Copenhagen(Here’s Antonio presenting the paper)

You can see the rest of the Evostar photos in their flickr account.

CoSECiVi 2015 ya está en marcha

La segunda edición del Congreso de la Sociedad Española para las Ciencias del Videojuego ya está en marcha.

La web del mismo es:

Al igual que el año anterior, este congreso pretende ser un foro científico para el intercambio de ideas y resultados sobre el diseño, la ingeniería y la teoría de la tecnología aplicada al entretenimiento en España. Por tanto, en él se abordarán temas de inteligencia artificial, informática gráfica, e ingeniería del software entre otros, así como su aplicación en cuestiones de diseño y en otras ramas como la enseñanza, la medicina, la comunicación o el arte, por ejemplo.

Al igual que en la edición anterior , se ha llegado a un acuerdo con la organización del GameLab 2015 para que el congreso se celebre en su mismo entorno (habrá inscripción conjunta), y se está trabajando para que el CoSECiVi se integre totalmente dentro del GameLab, siendo una parte parte de las actividades del mismo y celebrándose en su misma sede. Esto dará un gran impulso al congreso y lo acercará aún más a la industria del videojuego en nuestro país. :D

Al igual que el año pasado habrá cuatro posibles categorías para los artículos a enviar, que podrán escribirse en español o en inglés:

  1. Trabajos Regulares: Trabajos que muestren resultados de investigación. Éstos pueden tener una extensión máxima de 12 páginas en formato LNCS.
  2. Trabajos Emergentes: Nuevas ideas de investigación que pueden ser enriquecidas a través de la discusión con la comunidad SECiVi. Estos trabajos deberán tener una extensión máxima de 10 páginas en formato LNCS.
  3. Demostración de herramientas: Esta opción está orientada a la presentación de herramientas que soporten de forma práctica las propuestas teóricas. Las propuestas de demostraciones de herramientas podrán tener una extensión máxima de 6 páginas en formato LNCS.
  4. Divulgación de Trabajos Relevantes ya Publicados: Este apartado busca dar a conocer trabajos de investigación con altos índices de calidad y de especial relevancia para la comunidad SECiVi. Estas propuestas, en su versión definitiva (la que se publicará en la actas del congreso), podrán tener una extensión máxima de 6 páginas en formato LNCS, resumiendo el contenido del trabajo que pretende darse a conocer.

Como colofón, se hará una selección de los mejores trabajos (escritos en inglés) para su posible publicación en un número especial de la revista Entertainment Computing de Elsevier

¡Esperamos vuestros trabajos! ;D

[Paper] My life as a sim: evolving unique and engaging life stories using virtual worlds

Our latest publication My life as a sim: evolving unique and engaging life stories using virtual worlds, using our framework MADE (created by @rubenhek), has been published in the ALIFE 2014 conference. The abstract:

Stories are not only painfully weaved by crafty writers in the solitude of their studios; they also have to be produced massively for non-player characters in the video game industry or tailored to particular tastes in personalized stories. However, the creation of fictional stories is a very complex task that usually implies a creative process where the author has to combine characters, conflicts and backstories to create an engaging narrative. This work describes a general methodology to generate cohesive and coherent backstories where desired archetypes (universally accepted literary symbols) can emerge in complex stochastic systems. This methodology supports the modeling and parametrization of the agents, the environment where they will live and the desired literary setting. The use of a Genetic Algorithm (GA) is proposed to establish the parameter configuration that will lead to backstories that best fit the setting. Information extracted from a simulation can then be used to create the literary work. To demonstrate the adequacy of the methodology, we perform an implementation using a specific multi-agent system and evaluate the results, testing with three different literary settings.

Check out the presentation by @jjmerelo at You can download the proceedings of the conference (CC license), or download the paper draft.

More information is available on the project page.

Tenemos el segundo bot más humano DEL MUNDO…

…jugando a Unreal Tournament 2004. :D

José L. Jiménez, estudiante de informática de la Universidad de Málaga, dirigido por Antonio Fernández Leiva y por mí ha creado un bot, llamado NizorBot, a partir del ExpertBot que hicieron Francisco Aisa y Ricardo Caballero, que ha obtenido el segundo puesto (considerando la humanidad (humanness) del mismo) en el BotPrize 2014, celebrado dentro del CIG 2014.

Human or Bot competition (BotPrize)

Resultados de la competición BotPrize 2014 (primeros puestos)

La competición se basa en un Test de Turing que los bots deben pasar frente a jueces humanos. Éstos se enfrentarán a los bots en combates (Deathmatch) dentro del juego Unreal Tournament 2004. Durante dichos combates (en varias tandas), los jueces marcarán a cada uno de sus rivales como humano o bot en base a su criterio sobre el comportamiento que observen en el jugador.

Además, este año se ha añadido la presencia de un gran número de jueces off-line, es decir, ‘fuera del juego’, que visualizan los enfrentamientos y deciden también qué jugadores son humanos y cuáles no.

Esto le da más valor al resultado que otros años, ya que estos jueces cuentan con un punto de vista ‘menos limitado’ que los participantes en el juego.

El bot ha sido implementado mediante algoritmos evolutivos interactivos, en los que un controlador humano interviene cada cierto tiempo en dos sentidos:

  • Ajustando parámetros del algoritmo (controlador experto en el algoritmo)
  • Respondiendo a cuestiones sobre el jugador y el juego (controlador experto en el juego).

En ambos casos, el desarrollo del algoritmo (su ejecución) se ve afectado y dirigido por este controlador, lo que hace que el bot evolucione de una forma más acorde con lo que éste espera. En este caso, para comportarse de la manera más humana posible.

José está en proceso de escritura de su Proyecto Fin de Carrera y, a continuación, los tres trabajaremos en un artículo describiendo todo el proceso.

¡Estad atentos! :D