Unpacking UC San Diego’s use of LLMs to boost access to knowledge

With the rise of commercial AI solutions, UCSD chancellor Pradeep Khosla turned to the university’s IT team to explore innovative alternatives. Tapping into the capabilities of the university’s supercomputer center (SDSC), which is located on campus and is home to some of the field’s top researchers, its team was tasked with developing a cost-effective, locally hosted language model that could boost efficiency for staff. The goal was to use gen AI to streamline administrative workflows and reduce friction in accessing institutional knowledge, explains Brett Pollak, project lead and executive director for AI, data, and digital transformation at UCSD.

When the tool was being developed, UCSD had just launched a community of practice, where different team members who were experimenting with gen AI in their own time could share and showcase their work. “It was through this group that we found an open source orchestration platform, from a company called Onyx, that allowed us to integrate our institutional data through connectors into large language models,” says Pollak. “This use case allowed the LLM to draw information from our knowledge base, called Confluence, to answer service desk questions.”

UCSD’s service desk is traditionally staffed by students who undergo extensive training to assist with user queries. But unsurprisingly, being able to answer every question in real-time was impossible. “By connecting our knowledge base with large language models, our students were able to serve customers better,” he adds.

Developing TritonGPT

This was the inspiration to expand their use of LLMs to other knowledge repositories, and what ultimately became TritonGPT, a collection of UCSD-specific AI assistants capable of providing information across a wide range of topics.

Development of TritonGPT, which earned UCSD a CIO 100 Award this year, started in June 2023, and a pilot was launched with about 400 administrative staff by October. Based on user feedback, they decided to roll out an AI essentials course to ensure everyone had basic AI literacy and could make proper use of TritonGPT when it officially launched in the spring of 2024.

“When folks first started using it, they treated it like a search engine,” he says. This is why training was so important to teach people to prompt and ask questions in the same way you would if you were chatting to a regular person,” Pollak says. TritonGPT incorporates general AI assistance and more customized assistants that can help with things like writing job descriptions, for example, which, he adds, was a major pain point for hiring managers. According to Pollak, this tool helped them improve their recruiting processes by automating tasks such as compiling interview questions and posting job information.

Another valuable use case saw TritonGPT being used to speed up contract reviews. “As an institution, we have to sign non-disclosure agreements based on the research we’re doing for outside entities, like other institutions or third parties,” he adds. “The process of reading and reviewing each contract is exceptionally time-consuming, but by incorporating AI into this process, UCSD has reduced the number of hours spent on contract reviews by 60%.”

An ever-evolving solution

While the platform was originally only powered by Meta’s Llama models, Pollak and his team quickly realized the value of incorporating other commercial models into TritonGPT. “Over time, we’ve found that folks have gotten accustomed to working with different commercial solutions,” he continues. “Because we want our enterprise data to stay within UCSD, we knew if we wanted to get people to use our solution, we’d have to give them the choice to toggle between different models through an API.”

It’s all about finding ways to work smarter, especially in the current budget climate, which is forcing universities across the country to reduce spending and, in some cases, even lay off staff. One way to increase overall efficiency is by partnering with experts to identify use cases for automation and help different departments streamline their processes. “Unfortunately, some are worried about bringing in these tools because they fear they’ll be replaced by technology, which is why we had to do a lot of education around what we were trying to achieve,” Pollak says. “Slowly, people are realizing they’ll need to adapt if they don’t want to be left behind.”

Finding fresh use cases

TritonGPT is now in the hands of 38,000 faculty and employees at the university. They’ve also opened up the platform to other universities, including San Diego State University, the California State University system office, UC Berkeley, and the San Diego Community College District, to expand access to college students across the San Diego area.

“Right now, we’re poised to give students access and roll out student-facing assistance,” says Pollak. “We’re already piloting this with a dozen faculty members signed on to use instructional bots that would essentially act as virtual teaching assistants.” And as people become more accustomed to using AI tools, he hopes to democratize the ability to create assistants and agents in the system. “We want to enable users through no code tools to incorporate their own local content or context to create bespoke assistants based on specific individual, team, or department use cases,” he adds.