

The most successful benchmarks existing today are probably those used in robot competitions.” įinally, when one knows that the RoboCup official goal on the long view ( ”By 2050, develop a team of fully autonomous humanoid robots that can win against the human world champion team in soccer.”), one can still expect a lot of surprises in the robotic field. Generally speaking, cognitive benchmarks stimulate the research.Ībout the life span of a benchmark, the EURON website says: ”A benchmark can only be considered successful if the target community accepts it and uses it extensively in publications, conferences and reports as a way of measuring and comparing results. Thanks to this kind of event, pros and cons of a technique can be observed in comparison with another one. Indeed, some techniques advocated by the researchers in scientific publication lack comparison with other related research results. In robotic research, the cognitive benchmarks are useful mainly for comparing the research results. Moreover, this cognitive benchmark is an hybrid benchmark, i.e., completely different entities (for example, humans and computer programs) can participate. In a tournament where each player meets all the others, the number of points describes precisely the chess level of each participant. If a player wins a game he gets three points, if he loses he doesn't get any point, and if the two players tie each of them get one point. The rules of this game are limited and unambiguous.

Thanks to this competition spirit, at the end of the contest, the participants achieve often to very good solutions for the given problem.Ī chess game is given as example of a cognitive benchmark. To increase the motivation of the participants, the cognitive benchmarks are often shaped as contests. The rules of the benchmark must be unambiguous. This comparison is performed on a measurable criterion as the time to solve a problem or the victory in a round of a game. In robotics, cognitive benchmarks are used to compare the quality of the artificial intelligence of several robot controllers. An animal ranking can be done according to this criterion. If the animal belongs to the last class, it is self-awareness (the inverse is not necessary right). The animals can be sorted in three classes: the animal doesn't take interest in the mark, the animal takes interest in the mark of its reflection, or the animal takes interest in its own mark. Then, the animal is placed in front of a mirror. An odorless mark is placed on the animal without him noticing it. For example, the mirror test measures the self-awareness. Similarly, cognitive benchmark is a method for quantifying the intelligence (and not only the artificial intelligence). Once every CPU performance is quantified, it's easy to rank them according to this metric. Quicker the CPU finishes this computation, more efficient it is. A typical CPU benchmark consists to solve a long computation. For example, in the computer field, if one wants to establish a list of CPUs sorted by their performance, a CPU benchmark is performed on each CPU.

Generally speaking, a benchmark is a method for quantifying a specific attribute. The last part is dedicated to other active cognitive benchmark. To avoid redundancy with Rat's Life official web site we provide only a quick description of the contest and some clues to start quickly with this benchmark. Then, it introduces one cognitive benchmark in more details: Rat's Life. This chapter provides first a quick introduction about this topic. The cognitive benchmark topic was already introduced in section Enjoy Robot Competition. 3.2 The Federation of International Robot-soccer Association (FIRA) - Kheperasot.2.2.2 Vision-based Navigation - Simultaneous Localization and Mapping (SLAM).2.2.1 Visual-System - Pattern Recognition.This is because after a goal is scored, the robots are also relocated. Typical lack of progress situations are when the ball is stuck between robots, when there is no change in ball and robot’s positions, or when the ball is beyond detection or reach capability of all robots on the field.”īy detecting a lack of progress, a team is able to stop or alter it’s movement to prevent a wheel from moving while being transported.Īnother way to prevent wheels from falling off from being transported is by stopping once our robot scores. In the RCJ Soccer Simulation Demonstration rules, a Lack of Progress is defined as when “there is no progress in the gameplay for a reasonable period of time. These methods ensure that the motors never have to change speeds rapidly, resulting in wheels falling off. Another method is in ensuring that the motor speeds never change rapidly by comparing the previous speed with the current speed. One way of preventing a robot from moving from fast to slow is by creating a dead zone that prevents the motors from going back and forth over a large distance.
