New research line on µRTS

This new research line was started one year ago together with PhD student Abdessamed Ouessai and professor Mohammed Salem both from the University of Mascara (Algeria).

The objective is focused on the improvemet of the decision process of an autonomous agent for playing a simple Real-Time Strategy Game, named µRTS (microRTS). See an illustrative image below of this game/simulator created by Proffessor Santiago Ontañón mainly for research purposes.

For the moment three papers have been published:

1. Online Adversarial Planning in μRTS: A Survey. Presented at 2019 International Conference on Theoretical and Applicative Aspects of Computer Science (ICTAACS) on December 2019, and selected as Best paper of the conference.
2. Improving the Performance of MCTS-Based µRTS Agents Through Move Pruning. Published and presented at IEEE Conference on Games 2020 last August 2020.
3. Parametric Action Pre-Selection for MCTS in Real-Time Strategy Games. Presented at CoSECiVi 2020 yesterday.

Here you can see the slides and the video presentation of the paper at CoG:

Moreover an agent named USMBot was created and participed in the last µRTS AI Competition, reaching rank 4!!!

The bot can be found in Github

We hope you like it, as usual. :D

Ants are already in 5G

Today we have presented our first international paper on the scope of 5G. I have used for that my favourite metaheuristic: Ant Colony Optimization, which has been adapted to solve a problem of network service composition, i.e., the so-called Service Function Chaining.

The paper is titled «Applying Ant Colony Optimization for Service Function Chaining in a 5G Network» paper with the same title presented today (22 October 2019) in Granada, on the «6th IEEE International Conference on Internet of Things: Systems, Management and Security (IOTSMS 2019)«, and in this, inside the «International Workshop on Efficient and Smart 5G Technologies for IoT (ES5TI)«.

The abstract of the paper is:

The growth of data traffic and the demand for new services are two of the main challenges to take into account in the design of next-generation networks. Service Function Chaining (SFC) is a technique that allows the execution of advanced services, routing network traffic through an ordered list of virtual functions. This mechanism is getting great relevance due to the rise of Software-defined Networks (SDNs) and the use of Network Function Virtualization (NFV), as well as the offered possibilities in terms of flexibility and automation. Given the existing need for operators to offer low latency services in 5G networks, the composition of this chain is a critical process that affects the performance of these services. Inside this context, this paper presents the design and implementation of an Ant Colony Optimization algorithm (ACO) for the minimization of the routing cost of service chain composition. ACO is a specially designed metaheuristic to work with weighted graphs, also considering restrictions, as is the case of the addressed problem. To test the value of the implemented algorithm, two different instances have been solved. The first one (with only 6 nodes) is a proof of concept, which easily allows to analyze the obtained solutions. The second one (19 nodes) models a medium-size 5G network, and tries to show the performance of this method in a wider graph. The results show that the proposed algorithm can lead to optimal solutions in many cases, even in a short time (less than 0.5 seconds) in the largest instance, so we consider this method as a very promising solution in this field.

The presentation can be see from Slidehare:

Enjoy it!

(and cite us :D)

 

A new lap in the race to improve our evolutionary fuzzy-controllers for TORCS

Last week we presented at the IEEE Conference on Game 2019, held in London (UK), our new paper titled «Beating uncertainty in racing bot evolution through enhanced exploration and pole position selection«.

The abstract of the work is:

One of the main problems in the design through optimization of car racing bots is the inherent noise in the optimization process: besides the fact that the fitness is a heuristic
based on what we think are the keys to success and as such just a surrogate for the ultimate objective, winning races, fitness itself is uncertain due to the stochastic behavior of racing conditions and the rest of the (simulated) racers. The fuzzy-based genetic controller for the car racing simulator TORCS that we have defined in previous works is based on two fuzzy subcontrollers, one for deciding on the wheel steering angle and another to set the car target speed at the next simulation tick. They are both optimized by means of an Evolutionary Algorithm, which considers an already tested fitness function focused on the maximization of the average speed during the race and the minimization of the car damage. The noisy environment asks for keeping diversity high during evolution, that is why we have added a Blend Crossover (BLX-alpha) operator, which is, besides, able to exploit current results at the same time it explores. Additionally, we try to address uncertainty in selection by introducing a novel selection policy of parents based in races, where the individuals are grouped and compete against others in several races, so just the firsts ranked will remain in the population as parents. Several experiments have been conducted, testing the value of the different controllers. The results show that the combination of a dynamic BLX-alpha crossover operator plus the pole position selection policy clearly beats the rest of approaches. Moreover, in the comparison of this controller with one of the participants of the prestigious international Simulated Car Racing Championship, our autonomous driver obtains much better results than the opponent.

The presentation can be seen below:

As usual, enjoy it and…cite us! :D

Making birds less angry by evolving pig supporting structures in the game

After our poster in EvoStar, GECCO 2019 saw another poster on evolution of Angry Birds structures, in this ocassion focused on the inclusion of the Box2D Physics simulation engine into the evolutionary algorithm to save using Science Birds, which improved evaluation of the structures that needed it by 100x.
The poster is minimalistic, with the intention of making it awesome.
//embedr.flickr.com/assets/client-code.js
Get data, code and the paper itself from our repository.

Angry Birds meet EAs at EVO* 2019

Last 24 of April we presented the work «Free Form Evolution for Angry Birds Level Generation» at EVOApplications 2019 (EvoGAMES) a conference part of EVO* 2019, held in Leipzig (Germany).

The abstract of the work is:

This paper presents an original approach for building structures that are stable under gravity for the physics-based puzzle game Angry Birds, with the ultimate objective of creating fun and aesthetically pleasing Angry Birds levels with the minimum number of constraints. This approach consists of a search-based procedural level generation method that uses evolutionary algorithms. In order to evaluate the stability of the levels, they are executed in an adaptation of an open source version of the game called Science Birds. In the same way, an open source evolutionary computation framework has been implemented to fit the requirements of the problem. The main challenge has been to design a fitness function that, first, avoids if possible the actual execution of the simulator, which is time consuming, and, then, to take into account the different ways in which a structure is not structurally sound and consider them in different ways to provide a smooth landscape that eventually achieves that soundness. Different representations and operators have been considered and studied. In order to test the method four experiments have been carried out, obtaining a variety of stable structures, which is the first path for the generation of levels that are aesthetically pleasing as well as playable.

@amorag did a short presentation and later ‘defended’ a poster during the reception act. The presentation is a description of the poster:

Actually the poster was selected as the second best of the conference by the attendants. :D

Those interested can found the paper at Springer web: https://link.springer.com/chapter/10.1007/978-3-030-16692-2_9

Enjoy it… and cite us! ;D

Stateless evolutionary algorithms

Most algorithms keep some kind of state: global variable that holds the optimum, a counter of the number of evaluations, some context every piece algorithm must be aware of. However, this might not be the best when we want to create cloud-native algorithms, and it’s not in the case of cloudy evolutionary algorithms. There was a bit of that in GECCO, but as long as I was attending the Perl Conference in Glasgow, and I was using Perl, I kind of switched focus from the evolutionary part (but there was a bit of that too) to the language-design part and talked about evolutionary algorithms in Perl 6. The presentation is linked from the talk description.
Main problem is that you have to create dataflows that allow the algorithm to progress, as well as work efficiently in that kind of concurrent architecture, which is similar to the serverless architecture that is our eventual target.
We’ll be continuing this research in the workshop on engineering applications in Medellín, where my keynote will deal with this same topic.

//embedr.flickr.com/assets/client-code.js

A data-mining based process to early identify breast cancer from metabolomic data

Abstract of our work presented at EURO 2018, the largest and most important conference for Operational Research, co-authored by Víctor M. Rivas Santos, jointly with researchers of Complejo Hospitalario de Jaén and Fundación Medina.

This paper was presented last 9-July-2018 at Valencia, as part of the stream Data Mining and Statistics.

A data-mining based process to early identify breast cancer from metabolomic data

Abstract

We present the results yielded by our multidisciplinary group in the task of discriminating blood samples coming from breast cancer patients and healthy people. Models used to classify samples have been built using data mining techniques; data have been collected by means of liquid chromatography-mass spectrometry, a technique that detects and quantifies the metabolites present in blood samples.

Different algorithms have been tested under 10-CV and 75/25 scenarios. Our experiments showed that IBk, and J48 and Logistic Model Trees yielded rates greater than 90% only for healthy people. Naive Bayes and Random Forest enhanced the previous results in the 10-CV approach, but they did not yield more than 85% of true positives for patients in the 75/25 one. Finally, Bayesian network resulted to be the best algorithm as rates greater than 90% were yielded for both patients and rest of the people.

Many statistics have been computed as well as confusion matrices, showing that the model built by Bayesian network can effectively be used to solve this problem. Currently, the metabolites used to do built the model are being identified by biochemists. This last step will be definitive in order to consider them as a valid biomarker for breast cancer.

Creating Hearthstone decks by using Genetic Algorithms

I’m glad you’re here, friend! There’s a chill outside, so pull up a chair by the hearth of our inn and prepare to learn how the Ancient Gods use the power of the secret and ancient branch of the Evolution to generate Hearthstone decks by means of the magic and mistery!!

Several months ago, my colleague Alberto Tonda and I were discussing about our latest adventures playing the Digital Collectible Card Game Hearthstone, when one of us said «Uhm, Genetic Algorithms usually work well with combinatorial problems, and solutions are usually a vector of elements. Elements such as cards. Such as cards of Hearthstone, the game we are playing right now while we are talking. Are you thinking what I’m thinking?»

Five minutes later we found an open-source Hearthstone simulator and started to think how to address the possibility of automatically evolve decks of Hearthstone.

The idea is quite simple: Hearthstone is played using a deck of 30 cards (from a pool of thousands available), so it is easy to model the candidate solution. With the simulator, we can perform several matches using different enemy decks, and obtain the number of victories. Therefore, we have a number that can be used to model the performance (fitness) of the deck.

Soooo, it’s easy to see one and one makes two, two and one makes three, and it was destiny, that we created a genetic algorithm that generates deck for Hearthstone for free.

Our preliminary results where discussed here, but we wanted to continue testing our method, so we tested using all available classes of the game, with the help of JJ, Giovanny and Antonio. All the best human-made decks were outperformed by our approach! And not only that, we applied a new operator called Smart Mutation that it is based in what we do when we test new decks in Hearthstone: we remove a card, and place another instead, but with +/-1 mana crystals, and not one completely random from the pool. The results were even better. Neat!

Maybe you prefer to read the abstract, that it is written in a more formal way than this post. You know, using the language of the science.

Collectible card games have been among the most popular and profitable products of the entertainment industry since the early days of Magic: The Gathering in the nineties. Digital versions have also appeared, with HearthStone: Heroes of WarCraft being one of the most popular. In Hearthstone, every player can play as a hero, from a set of nine, and build his/her deck before the game from a big pool of available cards, including both neutral and hero-specific cards.
This kind of games offers several challenges for researchers in artificial intelligence since they involve hidden information, unpredictable behaviour, and a large and rugged search space. Besides, an important part of player engagement in such games is a periodical input of new cards in the system, which mainly opens the door to new strategies for the players. Playtesting is the method used to check the new card sets for possible design flaws, and it is usually performed manually or via exhaustive search; in the case of Hearthstone, such test plays must take into account the chosen hero, with its specific kind of cards.
In this paper, we present a novel idea to improve and accelerate the playtesting process, systematically exploring the space of possible decks using an Evolutionary Algorithm (EA). This EA creates HearthStone decks which are then played by an AI versus established human-designed decks. Since the space of possible combinations that are play-tested is huge, search through the space of possible decks has been shortened via a new heuristic mutation operator, which is based on the behaviour of human players modifying their decks.
Results show the viability of our method for exploring the space of possible decks and automating the play-testing phase of game design. The resulting decks, that have been examined for balancedness by an expert player, outperform human-made ones when played by the AI; the introduction of the new heuristic operator helps to improve the obtained solutions, and basing the study on the whole set of heroes shows its validity through the whole range of decks.

You can download the complete paper from the Knowledge-based Systems Journal https://www.sciencedirect.com/science/article/pii/S0950705118301953

See you in future adventures!!!

A better TORCS driving controller presented in EvoStar 2018

Last year, we presented along with Mohammed Salem, from the university of Mascara, in Algeria, our TORCS driving controller. This controller effectively drives a simulated vehicle, considering input from its sensors, and deciding on a target speed and how to turn the steering wheel.

This year, in Evostar 2018 in Parma, we had again our paper accepted for the poster session, which took place in the incredible corridor to the right of these words. The poster included interactive elements, such as a small car used for demonstration on how the driver worked.

And it works really well, or at least better than the previous versions. The key element was the design of a new fitness function that includes damages, and also terms related to speed. Still some way to go; in the near future we will be posting our new results in this area.

The book of proceedings can be downloaded from Springer. Our paper is in page 342 and you can also download just the paper from here, but we do open science, so you can follow our writing process and download the paper from this GitHub repository too

 

 

Early prediction of the outcome of Starcraft Games

As a result of Antonio Álvarez Caballero master’s thesis, we’ll be presenting tomorrow at the IJCCI 2017 conference a poster on the early prediction of Starcraft games.
The basic idea behind this line of research is to try and find a model of the game so that we can do fast fitness evaluation of strategies without playing the whole game, which can take up to 60 minutes. That way, we can optimize those strategies in an evolutionary algorithm and find the best ones.
In our usual open science style, paper and data are available in a repository.
Our conclusions say that we might be able to pull that off, using k-nearest neighbor algorithm. But we might have to investigate a bit further if we really want to find a model that gives us some insight about what makes a strategy a winner.

Dark clouds allow early prediction of heavy rain in Funchal, near where IJCCI is taking place