Going ‘AI native’ with in-house ChatGPT the MITRE way

As a nonprofit R&D center for the US government, MITRE is no stranger to AI. Its researchers have long been working with IBM’s Watson AI technology, and so it would come as little surprise that — when OpenAI released ChatGPT based on GPT 3.5 in late November 2022 — MITRE would be among the first organizations looking to capitalize on the technology, launching MITREChatGPT a month later.

MITREChatGPT, a secure, internally developed version of Microsoft’s OpenAI GPT 4, stands out as the organization’s first major generative AI tool. Released in May 2023, the project — which garnered MITRE a 2024 CIO 100 Award for IT leadership and innovation — is integrated with MITRE’s 65-year-old knowledge base and tools, and has been put into production by more than 60% of its 10,000-strong workforce.

The tool, which complements the R&D’s close partnership with Microsoft, has been enhanced to fulfill requests for information from six federally funded US agencies, including the Department of Defense, Federal Aviation Administration, and Department of Homeland Security. And it enables research teams to analyze legislation and policy documents in record time, delivering plans for proposed changes to these critical agencies in a day rather than weeks.

Some of MITRE’s most prominent projects include the development of the FAA air traffic control system and the MITRE ATT&CK Framework collection of cybercriminal attack techniques.

Most recently, MITRE’s investment in an Nvidia DGX SuperPod in Virginia will   accelerate its research into climate science, healthcare, and cybersecurity. The AI data center pod will also be used to power MITRE’s federal AI sandbox and testbed experimentation with AI-enabled applications and large language models (LLMs).

Toward ‘AI-native’ operations

MITRE CIO Deborah Youmans and Michal Cenkl, director of innovation and experimentation at MITRE, aim to use this evolving platform to transform the McLean, Va.-based research organization into an “AI-native organization” that provides the most efficient, intelligent, and critical data for government agencies.

“AI is a huge part of our future,” says Youmans, who was named CIO in August 2023 after serving 17 years at Booz Allen Hamilton, most recently as deputy CIO. “We have a co-led strategy between the Office of CIO and our CTO around our AI strategy and policy this year and our focus moving forward is to make MITRE an AI-native organization.”

As part of that effort, MITRE will leverage LLMs across its massive data intranet to create differentiating value, says Cenkl, a longtime MITRE IT leader. Since MITREChatGPT’s initial release, the nonprofit has bolstered its capabilities by integrating the tool across its knowledge and project repositories and by establishing a companywide user group for training and sharing of use cases.

By June 2024, MITREChatGPT offered document analysis and reasoning on thousands of documents, provided an enterprise prompt library, and made GPT 3.5 API available to projects, Cenkl says. MITRE has since deployed capabilities using GPT-4 and retrieval-augmented generation (RAG) for very large documents, he adds.

“We took a risk. We were very early in securely deploying generative AI across the enterprise,” Cenkl says, explaining that MITRE’s experience with AI-based systems, such as IBM’s Watson, to answer questions from information generated by its labs and research centers enabled them to quickly deploy OpenAI’s GPT-4. This means researchers don’t have to start from scratch on every request for information from federal agencies, which fund MITRE.

Youmans says MITRE’s aggressive embrace of ChatGPT will not only aid employees with their workloads and help produce reports faster, but it will also elevate the level of analysis due to MITRE’s significant hardware investment for data mining the nonprofit’s massive data repositories.

“Since we announced our acquisition of the NVIDIA DGX SuperPod for MITRE’s Federal AI Sandbox, there have been many requests for information from our federal sponsors,” Youmans says. “Integrating MITREChatGPT into our intranet with access to our knowledge assets helps connect all our employees with relevant data at MITRE.”

Must be secure

Given the sensitive nature of the data MITRE has on hand, step one of the project was to ensure MITRE’s version of ChatGPT was secure and did not provide access to classified information.

To that end, MITRE, which uses the OpenAI service in Azure, negotiated with Microsoft to decrease logging and add security controls to meet its stringent security requirements, the CIO says.

“Having the instance of OpenAI’s ChatGPT in our secure Microsoft Azure tenant that we manage versus relying on the general public version so the information remains private is vital to ensuring security,” Youmans says. “We have guidelines in terms of what type of information can be shared in this environment.”

In addition to saving time researching and writing reports, MITREChatGPT offers coding assistance, idea generation, and problem-solving capabilities, as well as quality assurance improvements that reduce errors, according to the company. On average, the time to complete key tasks has decreased by between 10% and 30%, according to MITRE, which also operates a major R&D center in Bedford, Mass.

MITRE hosts monthly AI workshops with its employees to instruct on effective chat prompting and to help its employees understand the nuances of prompt engineering. Additionally, Youmans says, MITRE recently held its second technical exchange meeting across the company to enable sharing of MITREChatGPT innovations and to reinforce guidelines about security requirements.

MITRE CISO Bill Hill, who reports to Youmans, has worked to ensure MITREChatGPT passes muster with the organization’s infosec team.

“Our CISO has partnered hip to hip with the rest of the organization, and we’re going through a large transformation to make sure we are following [CISA’s] Secure by Design principles, from the beginning [of MITRE’s genAI activity],” Youmans says. “We have security from the beginning, connected with our development team. We are developing security from the ground layer. “

Demands on MITRE’s genAI platform, robust AI hardware, and vast repositories will likely increase, says Chirag Dekate, a vice president and AI analyst at Gartner.

“Government agencies are looking to thoughtfully leverage generative AI as a force multiplier with all guardrails built in. Across our engagements we are now seeing government agencies use gen AI innovation, including multimodal technologies from innovators like Google and OpenAI, to build transformative pilots designed to improve productivity and effectiveness of frontline agents and simplify complex and intricate processes,” Dekate says. “MITRE’s genAI innovations are a core part of the toolset that government agencies can explore in their journey to build value-centered solutions.”