In groundbreaking research at the University of Hawaiʻi at Hilo, computer scientist Travis Mandel and his interdisciplinary team have garnered national acclaim for their innovative use of artificial intelligence in revolutionizing reef fish tracking. The team’s FISHTRAC software, a creation of collaboration among students, alumni and faculty, employs AI to meticulously follow individual fish through video footage and photographs, providing a transformative solution to the time-consuming task of image review in the realm of marine research.

Mandel and his team published their research paper in Pattern Recognition this past March and recently learned they won the editor’s choice award.

“It certainly works better than all the other algorithms we compared it to in the paper, which was a very large number,” said Mandel who is an associate professor of computer science and director of UH Hilo’s interdisciplinary data science program.

It’s still not perfect, he explained, but the reality is the AI allows for much less human effort and time in terms of someone sitting and watching a hundred videos and then drawing boxes around every frame, “because that takes forever.”

FISHTRAC software

The AI-based video identification would serve as an alternative to catch-and-release tagging research methods, a process that can be invasive to the fish. The central question the research team hoped to address: How can AI and machine learning systems work with humans to solve real problems?

Initially, Mandel was called on by environmental scientists at UH Hilo to help with computer vision issues, or the ability of software to recognize objects consistently in photographs or videos. The process of teaching an AI engine to learn is complex, and projects such as these are on the cutting edge of computer science and environmental science today.

Although Mandel’s training was not in computer vision, the pressing need for research within this field quickly presented itself.

“A lot of people started reaching out to me, faculty members and grad students in different disciplines, saying, ‘Hey, can you help us with our computer vision problems?’” said Mandel.

Mandel co-authored the paper with UH Hilo alumni Mark Jimenez (computer science), Emily Risley (computer science), Taishi Nammoto (physics) and Rebekka Williams (mathematics). UH Hilo students Meynard Ballesteros (computer science) and Bobbie Suarez (tropical conservation biology and environmental science) also served as co-authors alongside Max Panoff, a doctoral student in electrical and computer engineering at the University of Florida.

For more go to UH Hilo Stories.

—By Evangeline Lemieux, who is double majoring in English and medical anthropology at UH Hilo.