IT leaders go small for purpose-built AI

When adopting AI, sometimes the best direction is to go small. That’s what a number of IT leaders are learning of late, as the AI market and enterprise AI strategies continue to evolve.

During the new AI revolution of the past year and a half, many companies have experimented with and developed solutions with large language models (LLMs) such as GPT-4 via Azure OpenAI, while weighing the merits of digital assistants like Microsoft Copilot. But purpose-built small language models (SLMs) and other AI technologies also have their place, IT leaders are finding, with benefits such as fewer hallucinations and a lower cost to deploy.

Microsoft and Apple are seeing the potential for small AIs, with Microsoft rolling out its Phi-3 small language models in April, and Apple releasing eight small language models, for use on handheld devices, in the same month.

SLMs and other traditional non-LLM AI technologies have many applications, particularly for organizations with specialized needs, says Dave Bullock, CTO at UJET, a contact-center-as-a-service provider experimenting with small language model AIs. SLMs can be trained to serve a specific function with a limited data set, giving organizations complete control over how the data is used.

Low barriers to entry

Better yet, the cost to try a small language model AI is close to zero, as opposed to monthly licensing costs for an LLM or spending millions of dollars to build your own, Bullock says.

Hugging Face offers dozens of open-source and free-to-use AIs that companies can tune for their specific needs, using GPUs they already have or renting GPU power from a provider. While AI expertise in LLMs is still rare, most software engineers can use readily available resources to train or tune their own small language models, he says.

“You might already have a GPU in your video game machine, or you want to just spin up some GPUs in the cloud, and just have them long enough to train,” he says. “It could be a very, very low barrier to entry.”

Insight Enterprises, a technology solutions integrator, sees about 90% of its clients using LLMs for their AI projects, but a trend toward smaller, more specialized models is coming, says Carm Taglienti, CTO and chief data officer at the company.

Taglienti recommends LLMs to clients that want to experiment with AI, but in some cases, he recommends classic AI tools for specific tasks. LLMs are good for tasks such as summarizing documents or creating marketing material but are often more difficult and expensive to tune for niche use cases than small AIs, he says.

“If you’re using AI for a very targeted set of tasks, you can test to ensure that those tasks are executed properly, and then you don’t really worry too much about the fact that it can’t do something like create a recipe for souffle,” he says.

Sometimes, ML is all you need 

A small AI approach has worked for Dayforce, a human capital management software vendor, says David Lloyd, chief data and AI officer at the company.

Dayforce uses AI and related technologies for several functions, with machine learning helping to match employees at client companies to career coaches. Dayforce also uses traditional machine learning to identify employees at client companies who may be thinking about leaving their jobs, so that the clients can intervene to keep them.

Not only are smaller models easier to train, but they also give Dayforce a high level of control over the data they use, a critical need when dealing with employee information, Lloyd says.

When looking at the risk of an employee quitting, for example, the machine learning tools developed by Dayforce look at factors such as the employee’s performance over time and the number of performance increases received.

“When modeling that across your entire employee base, looking at the movement of employees, that doesn’t require generative AI, in fact, generative would fail miserably,” he says. “At that point you’re really looking at things like a recurrent neural network, where you’re looking at the history over time.”

A generative AI may be good for screening resumes, but once the recruiting process starts, a traditional machine learning model works better to assist recruiters, Lloyd adds. Dayforce uses a human-reinforced ML process to assist recruiters.

“This concept of bigger is better is, in my view, false,” he says. “When you look at the smaller models for the generative side, you have very good specialty models. You can look at some that are good for language translation, others that are very strong on math, and ours, which is very strong on human capital management.”

Building AI for your needs

HomeZada, provider of digital home management tools, is another convert to a purpose-built approach to AI. The company has licensed an LLM, but since June, it has also built seven proprietary AI functions to help homeowners manage costs and other issues associated with their properties.

HomeZada’s Homeowner AI functionality is integrated with the larger digital home management platform, says John Bodrozic, co-founder and CIO at the company. HomeZada uses retrieval augmented generation (RAG) alongside external, proprietary, and user data to improve the accuracy and reliability of its licensed LLM.

Using an LLM without any tweaks results in generic answers about the value of a home or the cost of a bathroom remodeling project, Bodrozic says. “By itself, it does not provide a deep personalization for every unique homeowner on the platform, thus it is not specific enough to provide real value,” he says. “Consumers demand expertise specificity that considers their home and location.”

For example, Homeowner AI creates budgets for home improvement projects, based on location, materials used, and other factors. The AI tool enables homeowners to document home and personal asset inventories using photographs, and it can diagnose repair and home improvement issues in real time. Homeowner AI can also send users weather alerts based on their locations, and it can assess climate disaster risk.

Bodrozic considers RAG as a happy midpoint between building or training a small AI and using an LLM by itself. An LLM may provide an answer to any of a million prompts in milliseconds, but the RAG-enhanced Homeowner AI doesn’t need to be as fast, nor does it need to be an expert in all things.

“We’re not big enough, nor do we need to build our own AI tool for a homeowner, because it doesn’t need to be real time like that,” he says. “Does the user need the response over how much my bathroom remodel is going to cost in milliseconds? No, they can wait 30 seconds.”

The right tool for the job

CIOs and chief data officers at companies trying to decide what size of AI they need should ask themselves several questions before jumping in, Bodrozic says. Response time, cost, data privacy, and specialized needs are some considerations.

“You really need to sort of figure out the context of domain of who is going to use your AI, where are you are going to use the AI,” he adds. “Is there a unique set of data versus a massive set of data?”

He suggests that CIOs and CDOs run short experiments with an AI to see whether it fits their needs. Too often, companies launch a six-month AI project and spend significant time and resources on something that ultimately doesn’t work.

“To start, you need to run a test for one day,” he says. “Instead of having a 50-person committee all trying to have input on this thing, create a five- or 10-person committee that can do rapid tests over the course of three weeks.”

With the current AI craze, UJET’s Lloyd sees a rush to adopt AI when it may not be the right solution. CIOs first need to identify a problem that AI can fix.

“I don’t think companies actually ask themselves, when they look at the problems they’re trying to solve, whether AI is even applicable,” he says. “I can open a bottle with a wrench, but that’s not necessarily the best approach.”