Using Computer Vision To Support Gut Disorder Patients

Painful gut disorders, including Crohn’s disease and Irritable Bowel Syndrome (IBS), affect between 60 and 70 million people in the United States. Such conditions can cause embarrassment and seriously impact sufferers’ personal and professional lives. Monitoring patient’s stool, among other symptoms, is essential in managing these conditions and treatments, yet such data is hard to collect reliably outside of clinics.

Now, a team of graduates from Cornell Tech and The Massachusetts Institute of Technology (MIT) are building a platform aiming to dramatically improve the way patients experience chronic gut disorders. Notably, this technology features computer vision to analyze patients’ stool in the privacy of their own homes. Their pioneering solution has been awarded the Cornell Tech Startup Award, which includes $80,000 in pre-seed funding and $20,000 of co-working space in the Tata Innovation Center. 

Auggi, which stands for “augmented gastroenterology”, uses technology built by David Hachuel, Technion-Cornell Dual Master’s Degrees in Health Tech ’19. During his specialization research project, a requirement for all students in the Jacobs Technion-Cornell Institute programs, Hachuel chose to develop a healthcare solution that draws on readily available hardware, such as smartphones, to monitor health signals from home. 

“I started asking myself, ‘Where can we find health data that is not currently being measured, nor captured, and that is actually clinically relevant?” he said.

Hachuel realized that the bathroom ticked all these boxes. “All this data that happens outside of the clinic, i.e. your bowel movements, is not being leveraged to really understand your symptoms and treatment.”

Introducing augmented gastroenterology

Tracking bowel movements allows physicians to make an accurate diagnosis, understand symptoms, and monitor treatments, said Hachuel. “Currently there is no good way of measuring characteristics of bowel movements because it’s subjective and patients forget. Patients don’t do it. Patients often don’t know how to report it,” he said.

Auggi solves this problem. Patients can take photos of their stool and document other key symptoms on their phones, then computer-vision algorithms and deep-learning techniques are deployed to extract clinical insights from this data. This information becomes the enabler of an improved patient-provider experience.

The algorithms themselves are trained by specialists. Crowd-sourced stool images are analyzed and annotated by a team of physicians, said Hachuel. “This team of physicians provides unbiased and robust training for our algorithms.”

Hachuel’s business partners are Alfonso Martinez and Cherry Gao. Last year, Hachuel and Martinez took raw versions of Auggi’s algorithms to a 48-hour hackathon at MIT, where Martinez is currently studying for an MBA, and they won. The win gave them access to part-time incubator program at MIT and allowed them to collaborate with physicians at Massachusetts General Hospital in Boston. “That’s when we got really excited and realized this could be a company, not just a random idea,” said Hachuel. Gao, a Ph.D student in biological engineering at MIT, joined in early 2019, completing the Auggi team. 

Auggi was further refined in Cornell Tech’s Startup Studio, where students evaluate and develop startup ideas under the guidance of experienced entrepreneurs and venture capitalists. The team has also been selected for delta v, MIT’s highly competitive summer accelerator program.

Testing the technology and crowd-sourcing the data

Going forward, Hachuel, Martinez, and Gao will launch a 100-patient pilot to rigorously test and validate their technology. The team will also build data sets of crowd-sourced images collected via a new app they have developed, “Believe it or not, there is no database of stool images out there that we can use to train our algorithm,” said Hachuel.

The final product will include what Hachuel describes as “an empathetic conversational interface.” Aimed at reducing the isolation and stigma associated with gut disorders, it will predict events like when patients are likely to have bowel movements (allowing them to plan daily lives more easily) and provide remote access to vetted specialists.

Hachuel believes that Auggi is a timely technology with a bright future. “Stool images present a new opportunity for healthcare research and delivery,” he said. Additionally, with other gut-related fields such as microbiome research in their infancy, Auggi looks like the right tech at the right time.