The EnTimeMent project aims to enable a radical change of paradigm and technology in human movement analysis. The goal is to develop innovative software technologies supporting everyday life of disabled persons, as well as sensory-motor entertainment and commitment in sport, dance and music.

Picture of a dancer dancing on a stage together with virtual dancer, an avatar displayed on the wall by a cloud of points

Antonio Camurri, professor of Human Computer Interaction at the University of Genova, is the coordinator of the EnTimeMent. He started working on this ambitious project in 2019.

The creation of realistic animations of virtual characters in videogames and movies, or the analysis of human movement in sport trainings and in therapies for rehabilitation are just a few examples of the many applications of motion capture technology. Professor Antonio Camurri has helped us to understand the challenges of this fascinating research field.

Antonio, what specific issues the EnTimeMent project is addressing?

In 1973, the Swedish psychologist Gunnar Johansson and his colleagues showed that the human visual system can perceive the movement of a human body from a limited number of moving points. This landmark study grounded the scientific basis of current motion capture technologies.

However, the state of the art of motion capture technology only considers low-level temporal and spatial information, limited to computation of kinematic measures whose time-frame is usually too short for an effective perception and prediction of complex phenomena. While a lot of effort is being spent improving such technologies in the direction of more accurate and more portable systems (e.g., wearable and wireless), such developments are incremental with respect to a conceptual and technological paradigm that remains unchanged.

The Future and Emerging Technologies (FET) project EnTimeMent proposes a radical change of paradigm and technology in human movement analysis. Under this new perspective, the time-frame for analysis is grounded on novel neuroscientific, biomechanical, psychological, and computational evidence, inspired by the arts, and dynamically adapted to the human time governing the phenomena under investigation, the multiple temporal scales concurrently interacting in the brain.

What are the objectives of the project?

EnTimeMent includes measurable scientific, technological, and community-building objectives.

Scientific objectives include (i) the definition and empirical validation of neuro-cognitive models of the multiple and mutually interactive time scales that contribute to human perception of gesture qualities and action prediction, (ii) the development of computational models grounded on such neuro-cognitive models, (iii) the investigation of music and dance performers movements synchronised at different temporal scales, and a broader set of controlled as well as ecological experiments.

Technological objectives consist of the development of computational methods and software modules integrated in the project platform, to support experiments and proof-of-concepts in three different real-world scenarios focusing on healing and support of everyday life in disabled persons, sensory-motor entrainment and commitment in sport, dance and music.

Community-building objectives include the building of a novel scientific interdisciplinary community leading the transformation of the current generation of motion capture systems into a novel generation of time-aware motion perception and prediction technologies.      

How do models and technologies developed by EnTimeMent fit in such a diverse range of potential applications (from healthcare, to performing arts, to sports)?

The multi-timescale EnTimeMent approach faces different scenarios by always sharing a common approach to automated movement analysis, inspired to the human capabilities to understand and predict movement qualities as well as non-verbal social signals such as empathy, emotional contagion, and leadership.

Last September, you presented the activities and objectives of EnTimeMent at the Future Tech Week 2019. What were the reactions of the stakeholders from industry and institutions?

The first EnTimeMent public event A Tempo! was designed to collect useful feedbacks from a number of selected stakeholders. The event was structured in two parts: a presentation of the project by means of live demonstrations explaining the main core concepts of the project. And a round table discussing the feedbacks from the participants. The audience was selected among representatives from industry and institutions. In the first part, concepts such as “multiple temporal scales” and “motor signature” were introduced and concretely demonstrated applied to healing, rehabilitation, music, and dance.

One of the demos presented at the workshop consisted of an improvisation of a real dancer with a virtual dancer (an avatar displayed by a cloud of points): the two dancers start to move with very different movement qualities (different individual motor signatures, emphasized by two very different interactive sonifications) and they slowly converge to a single organism, characterized by a common group motor signature.

The presentations and the discussion were very productive. Several novel ideas and suggestions emerged and cultural heritage emerged as an important sector where EnTimeMent research results may play an important role. For example, on the application of project multi-timescale sensitive technologies to support the active experience of museum visitors, towards a deeper and more intense experience and awareness of cultural content.

In sport and in particular in paralympic sports, EnTimeMent technology emerged as a promising support to enhance the training practices of athletes. It could also be used in collective sports as a way to analyse and predict the movements of individual players and the interactions between them.

Lastly, as a result of the A Tempo! event, EnTimeMent has also been offered the possibility to take part in the Disability Innovation Summit 2020 which will be held in Tokyo on the eve of the Paralympic Games.

What has been your experience with FET programme so far? How did FET contribute to your research?

Personally, FET contributed significantly to my scientific growth and career.

Moreover, results obtained by my previous research in the FET SIEMPRE project were also an important premise for exploitations of new contracts at an international level with industry and institutions.

More in general, the possibility enabled by FET to participate in concrete and at the same time visionary research programs is of paramount importance for achieving scientific and technological results often unexpected and outstanding. FET projects are ideal ecosystems to enable the creation of trans-disciplinary research communities: for example, in EnTimeMent a community at the intersection of cognitive neuroscience, computer engineering, and humanities and the arts is taking shape, with the involvement of young researchers, PhD students, artists, as well as industry and startup incubators: I deem this is a very important premise for a successful and strong impact at societal and industry levels.

Background

FET-Open and FET Proactive are now part of the Enhanced European Innovation Council (EIC) Pilot (specifically of the Pathfinder), the new home for deep-tech research and innovation in Horizon 2020.