Each one of us deals constantly with time. For example, we might wake up at 8 o’clock, spend 30 minutes getting ready, reach the office by 9 o’clock, work for 8 hours, and so on. Time becomes even more important when we are interacting with other humans: in order to communicate, collaborate and make arrangements, we need to understand time in the same way.

Project team member working with robot to perform cooking tasks

But what about robots?

Robots, and machines in general, can count and track time very accurately using their internal clocks, but they do not have the capacity to perceive and manage time. In other words, today’s robots do not have a “sense of time” that allows them to seamlessly interact and collaborate with humans and be smoothly integrated into human societies. As robots become more and more a part of our everyday lives, we need to find ways for them to understand time.

Time for human-robot interaction

At the Computational Vision and Robotics Lab at FORTH (the Foundation for Research and Technology – Hellas), we have built up substantial experience in both robotics and human-robot interaction (HRI), and for several years we have investigated the importance and impact of time perception on HRI. We soon realised that only by bringing together research labs with complementary areas of expertise could we shed light on this hard problem and significantly advance relevant aspects of robotics technology. Accordingly, we gathered a consortium of European institutes with notable expertise in scientific domains where Europe is a key player, such as robotics, psychology and neurosciences, to investigate the concept of time and its perception, for both humans and robots.

TimeStorm’s first steps

Obtaining funding from the EU’s FET (Future and Emerging Technologies) programme allowed us to launch the TimeStorm project. The project put forward the innovative concept that human-like time perception should be an innate capacity of artificial systems such as robots.

During the early stages of the project, time perception was investigated from the perspective of humans. An extensive series of behavioural, developmental and neuroimaging experiments was conducted in order to define and model temporal perception, along with aspects of temporal distortions, noise, duration estimation, timely action planning and time-aware memory management. The results of these experiments were used to produce a unified theoretical framework of how time perception works and also build computational models of it.

project team with robot

Delivery… on time!

TimeStorm’s biggest breakthrough was during the later stages of the project, when the application of the time perception framework in robotics was demonstrated in a realistic “breakfast preparation” scenario. Two robots and a human had to collaborate in order to prepare and serve breakfast. Although the robots cannot yet cook food, in this scenario they managed to fluently coordinate their actions and fulfill their task on time! A video of the demonstration can be found here.

Now what?

The members of the TimeStorm consortium are proud to have transformed the notion of time perception in the context of artificial intelligence from a shapeless and immature research topic to a promising research strand. We have clearly demonstrated the proof of concept and the necessity of time perception for the execution of complex, collaborative tasks by humans and robots.

Despite the fact that there are numerous aspects of time yet to be explored and revealed, it is our strong belief that TimeStorm can serve as the basis for natural, time-aware, human-robot symbiosis, even though the topic is so complex that further work is definitely needed to deliver mature and robust technology for the next generation of robots. We roughly estimate that it will be at least one decade before robots are routinely designed with the capacity to perceive time – but the market potential of these technologies is enormous.