Marina Carbone and Nadia Cattari from the University of Pisa in Italy talk about Vostars, an augmented reality project that aims to revolutionise the future of surgery

What is the Vostars project about?

The Vostars (Video and Optical See-Through Augmented Reality Surgical Systems) project aims to develop a breakthrough augmented reality visor to help surgeons with difficult surgery techniques. The aim is to bring ‘surgical navigation’ directly in front of the surgeon’s eyes. It is a wearable device that fuses the two main elements of augmented reality - video see-through and and optical see-through - to give the surgeon the best of both technologies while operating.

What do you mean by ‘augmented reality’?

Augmented reality is not the same as virtual reality. When VR is used in games, for example, it involves building an entirely new virtual world through the software; with AR you see the real world around you with additional information ‘virtually’ projected onto a display worn by the user. So you see reality, but augmented with virtual information.

As far as Vostars is concerned, we are augmenting the surgeon’s reality, adding all the information they might need to improve the surgery and guiding the surgeon as they work. For example, starting from data from MRI or CT scans of the patient, we can build a 3D model of the patient, showing the surgeon the exact pathology of the patient undergoing the surgery. This helps the surgeon to plan the surgery in advance, outside the operating room, and they can then see this plan projected onto the patient during the surgery itself. The majority of image-guided procedures at the moment project for example a CT image onto a remote display, meaning the surgeon’s attention is not wholly on the patient.

Using our visor, the surgeon can view the patient’s internal anatomy while looking directly at the body. They will be able to see parts of the body that are hidden when viewed normally, which can be extremely helpful when deciding where to place an implant or any other element needed to complete the surgery successfully. Using a wearable device and having all the information in front of their eyes is better for both the surgeon and the patient.

It is not just us saying this, by the way - it is the doctors and surgeons themselves. Any medical research and innovation should always take into account the needs of the doctors. We work very closely with the medical profession and all the technical specifications of our projects start from a series of brainstorming sessions directly with surgeons. This helps ensure a high level of acceptance of the technology by surgeons themselves, and makes sure that we are focusing our research on meeting a real surgical need.

What is your involvement in this project?

Vostars is a Horizon 2020 project with 11 partners around Europe; the University of Pisa is project leader, responsible for software development, some mechanical development and designing of surgical needs in terms of clinical trials. There will be three such trials (one in Italy, two in Germany). We here at the University of Pisa are responsible for dealing with all the ethical issues relating to these clinical trials.

We also have academic partners from Italy and Germany in our consortium, as well as medical centres and private businesses (who for example work on the AR displays) from across Europe. The Vostars project is funded by the European Union (with €3.8m from the Horizon 2020 research budget).

Although the project will end in June 2020, this does not mean the end of our work. The prototype we are working on is not very far from what the final product is likely to be, as our aim is to deliver a product at technology readiness level 7 (operational prototype); in two to three years, with the right investments, we believe it will be possible to push this to TRL 9 (fully operational system) and bring the project to market. We know that to do this we will need business partners, and the companies in the consortium are already working on an investment strategy, trying to assess how much is needed and whether there is sufficient funding available from the consortium members already or whether additional investment will be needed to bring this project to the market.

Are you confident you can create a product that is accessible to everyone?

AR is not needed for a lot of surgical procedures, but some crucial ones can certainly benefit from it - by allowing the procedure to be carried out more quickly for example, or by ensuring that the exact spot is reached. We are focusing on three surgical specialities at the moment: maxillofacial surgery, neurosurgery (in particular paediatric neurosurgery) and ear, nose and throat surgery.

The visor itself and all the electronics are not particularly expensive but the final product, which will need a lot of certification and regulatory approvals, will be a full surgical navigation system and as such will need a certain level of investment on the part of hospitals. But this is already the case for existing surgical navigation systems using remote imaging, and our aim is of course to develop a system that can be sold commercially and can be compared favourably with what is already available.

Does anything else need to change at the EU level to help you get your product to market?

In fact, the EU is pretty up-to-date when it comes to adapting to the rapid advances in technology. The rules on the approval of new medical devices have been updated relatively recently (in May 2017), and the EU is investing a lot in innovation and in surgical innovation. Because we already know what rules we have to follow when it comes to the development of new surgical devices, this makes it a lot easier to develop the technology, and thus improve its chances of approval and take-up.

Video

Euronews recently made a video, part of the Futuris series, about Vostars.

About the team

Marina Carbone was born in 1983 in Messina, Italy. She has an MSc in Biomedical Engineering from the University of Pisa, Italy, and a PhD in surgical navigation from the Biorobotics Institute at the Scuola Superiore Sant’Anna (Pisa). She has worked at EndoCAS Centre for Computer Assisted Surgery (University of Pisa) since 2009. Her main research interests are in the field of biomedical imaging, robotics and computer-assisted surgery.   

Nadia Cattari was born in 1992 in Sassari, Italy. She is a Biomedical Engineer and a PhD student in the Clinical and Translational Science programme at the University of Pisa. She has been a researcher at the EndoCAS Centre for Computer Assisted Surgery (University of Pisa) since 2018. Her main research activities span vision augmentation, stereoscopic 3D displays, computer graphics and analysis of perceptual issues in the interaction with augmented environments.