Researchers at University of Hawai’i at Mānoa, University of Illinois at Chicago, and Virginia Tech were awarded a $5 million National Science Foundation grant to synergize two complementary technologies — large-scale data visualization and Artificial Intelligence (AI) — to create the Smart Amplified Group Environment (SAGE3) open-source software that allows AI to be a co-pilot in the scientific discovery process.

Principal investigator Jason Leigh, director of the Laboratory for Advanced Visualization & Applications, and the Hawaii Data Science Institute, at the University of Hawai’i at Mānoa,  invented SAGE in 2004 as software to enable teams of collaborators to work together with data in the form of data visualizations. It soon became the most widely used platform in the world for collaborative data visualization.

SAGE3’s novel interactive artificial intelligence methods will learn from human analytic activity, and use the knowledge gained to assist people in synthesizing relevant data connections that lead to new hypotheses and findings, explained Chris North, a professor of computer science at Virginia Tech and a co-principal investigator. Scientists analyzing their data in SAGE3 will collaborate with each other and with artificial intelligence through large interactive visualization spaces, such as multi-monitor workstations, tiled display walls, and virtual reality headsets. 

Leigh said the applications for this technology are endless. “It will be tremendously useful to enable evidence-based response during natural disasters and events like the current COVID-19 pandemic,” Leigh said. 

He created SAGE in collaboration with researchers from the University of Illinois at Chicago, 16 years ago. They’ve since further developed SAGE and SAGE2 with 15 years of research funding by the National Science Foundation. Whereas SAGE pioneered the ability for users to collaborate over great distances to visualize data, SAGE2 moved that capability into the Cloud to allow users to take advantage of computing resources being rapidly dispersed on the Internet. SAGE3 will merge AI into SAGE2.

“This new funding provides us with an opportunity to take SAGE to the next level,” said Maxine Brown, director of the Electronic Visualization Laboratory at the University of Illinois at Chicago. Brown has been involved with SAGE since its inception. 

“As a ‘co-pilot’ to individuals or teams who are trying to examine large datasets, SAGE3 will provide a discovery pathway from data sources to human insights and knowledge, which is currently a gap in the e-Science pipeline that the National Science Foundation has called the ‘missing middle,” citedAssistant Professor Mahdi Belcaid, Data Scientist and Artificial Intelligence lead on the project.

According to the researchers, SAGE3 will make artificial intelligence technologies broadly accessible, not just a privilege for the technically savvy. North said the project will conduct user-centered design research studies to ensure usability of this novel combination of data visualization and artificial intelligence in SAGE3.  

Through seed funds provided by Virginia Tech’s Institute for Creativity, Arts, and Technology and the Hawai‘i Data Science Institute, North had the opportunity to work side-by-side with Leigh in the Laboratory for Advanced Visualization & Applications at the University of Hawai’i during the spring 2020 semester. This in-person exchange was essential to helping plan and initiate the five-year SAGE3 collaboration, noted North. SAGE3 will serve the Institute’s missions of bringing together computer scientists, domain scientists, engineers, artists, and designers to tackle some of the world’s most complex challenges. 

The user base goes well beyond scientists. Leigh said he sees SAGE3 helping teachers use and share artificial intelligence concepts to future generations of students. And for those who work in creative media, such as filmmakers and video game designers, Leigh envisions it could be a highly visual tool to brainstorm new ideas and analyze all forms of digital media to see how they relate to each other, which ones were most popular and why. Film producer Chris Lee, founder of University of Hawaiʻi’s Academy for Creative Media System has long been a supporter of SAGE2, helping Leigh construct SAGE2-based tiled display walls, called CyberCANOEs (Cyber-enabled Collaboration Analysis Navigation and Observation Environments) throughout Hawaiʻi.

SAGE3 is funded under National Science Foundation awards 2004014, 2003800 (Collaborative Research: CSSI Frameworks: SAGE3: Smart Amplified Group Environment for Harnessing the Data Revolution).