Counting carbs from smartphone photos of food
Have you ever taken a photo of your meal in a restaurant? Aside from sharing what you are eating with friends on social media, this habit could significantly improve your quality of life if you suffer from diabetes, thanks to an innovative smartphone app developed by an EU-funded project.
© Diabetes Technology Research, ARTORG Center, University of Bern
The novel app, designed and tested in the GoCARB initiative, allows people to easily count the carbohydrate content in each portion of food on a plate an essential requirement for diabetes patients to keep the disease in check. The patented system, which is set to be made commercially available, is more accurate than other everyday methods for calculating carbohydrate content, including patients trying to guess carbohydrate levels themselves.
“First, you place a credit-card-sized reference object next to the dish, and then take two photographs from different points of view. The photos are used to automatically detect, segment and recognise the food items. A 3D model is then generated to estimate the size of the portions, which is compared to a nutritional database to calculate carbohydrate content,” explains project coordinator Stavroula Mougiakakou at the University of Bern’s ARTORG Center for Biomedical Engineering Research.
By harnessing advances in smartphone technology and incorporating computer vision and artificial intelligence, the GoCARB app is the first of its kind to be technically and clinically validated for monitoring carbohydrate levels under real-life conditions.
“Treating diabetes involves accurately counting carbohydrates, both to reduce intake and calculate how much insulin a patient needs to control their blood-glucose levels, but this has not been an easy task. Even well-trained diabetic patients find it difficult to estimate carbohydrates precisely,” Mougiakakou says.
Trials conducted by the GoCARB researchers at Bern University Hospital’s Department of Diabetes, Endocrinology, Clinical Nutrition and Metabolism highlighted the seriousness of the challenge and underscored the advantages of using the new smartphone technology.
Accurate and easy-to-use glucose control
In a preclinical study, the GoCARB team asked people with type 1 diabetes to estimate the carbohydrate content of a number of dishes and then use the app for the same purpose. Comparison of the results showed that the estimates provided by the GoCARB app had a significantly lower margin of error at 12.28 grams of carbohydrate per meal than the estimate by the average participant, which was wrong by around 27.89 grams per meal.
A follow-up randomised crossover study, in which half the participants used the app for a week and the rest were given an app with simple logging functionalities, demonstrated that it is easier to keep glucose levels within the normal range when GoCARB is used.
According to patients’ feedback, GoCARB was easy to understand and use: 80 % of trial participants would be ready to use the software or a similar system in their daily life, and 100 % would recommend the app to others.
“We plan to test GoCARB in a more extended clinical trial to obtain more detailed and statistically significant results on the system’s clinical impact,” the project coordinator says. “We have reached an agreement on further development of the system with our partner Roche Diabetes Care, and the current prototype will be used and improved in several ways.”
Mougiakakou points out that daily use of the GoCARB system could result in significant improvements in quality of life for millions of people with diabetes by offering them an easy, cheap and accurate way to control their blood-glucose levels.
Diabetes affects one in 11 adults globally, and by 2040 it is estimated that more that 642 million adults worldwide will suffer from it.
The system could also be expanded to meet the needs of people with other conditions, such as obesity, kidney and cardiovascular diseases, or simply to provide lifestyle advice to the general public.
“Diet monitoring and assessment is an issue that should concern us all. The promotion of healthy eating behaviour would lead to a variety of benefits, not only for individuals but also for health services and society in general,” Mougiakakou concludes.