Open-weight AI models are changing the equation. Here’s what CIOs should experiment with next

For years, enterprise AI strategy has been dependent on a small set of proprietary, closed model providers, a limitation that has carried cost, lock-in and opacity. Now, open-weight models from Meta, Google and other global competitors are challenging that default. As of July 2026, open-weight models accounted for 36% of Vercel’s AI Gateway token volume, up from 11% in April, while share of spend more than doubled to 8.6%.

Floyd DCosta and Allan Dabre have made compelling cases for greater enterprise control over AI, particularly for strategic and security reasons. I agree, but making the case is easier than moving an organization wholesale to open-weight models. My advice for technology leaders is to stop debating the theory and start experimenting.

From personal to team experimentation

My hands-on experience with open-weight models started with a mentor recommending OpenClaw, an open-source AI agent designed to be a personal assistant. So, I tried running it with my ChatGPT subscription and burned through my quota so fast I could barely use ChatGPT for development, let alone regular chats. Then I found Ollama, which lets you run open-weight models locally and in the cloud. For the same monthly fee, I got significantly more OpenClaw usage because of the lower token cost.

Today, that setup has evolved to Hermes powered by GLM 5.3 Flash on Ollama’s cloud offering.   This configuration powers a personal assistant that helps manage my calendar, coordinate family logistics and surface relevant articles. If it went away, I wouldn’t be doomed, but being able to play with this tool has been unexpectedly useful.

My main takeaway from these experiments is that the performance gap between open-weight and frontier models is narrowing faster than many realize. That honestly surprised me.

So, I’ve now expanded my experiment by encouraging my team to use Gemma, an open-weight model from Google that offers enterprise-grade compliance. Gemma runs locally on a MacBook or in the cloud alongside Gemini models, offering flexibility and customization without full retraining.

Using Gemma for standup reports, issue triage and log monitoring gives my team hands-on experience with how these tools fit into our workflows and collaboration patterns without substantial infrastructure or token costs.

Bringing open-weight AI into the leadership conversation

Preliminary experiments like mine provide helpful insights: what tasks the models handle well, where they fall short, what the cost comparison looks like. That’s enough to start a conversation with your leadership team about where open-weight models fit into your AI strategy.

But how you frame that conversation matters. Don’t assume everyone in the room can parse the differences between open-source, open-weight and closed models. I’ve found it more productive to start with where the organization sits today. Our production environment and internal employee tools run on closed models, and for good reason: compliance coverage, enterprise support and predictable performance.

So, when my team started experimenting with Gemma alongside that stack, I wasn’t trying to replace a closed model vendor. I was trying to answer a specific question: where could an open-weight model take on tasks at lower cost without creating new compliance work? Based on my success streamlining my personal admin burden, having my team do the same made a lot of sense. Now I hope we can scale similar use cases across our organization to help teams spend more time on high-value work, and less time on administrative tasks.

What enterprise agreements bring to the equation

Depending on your business, an enterprise agreement provides more than access to a model;  it provides the infrastructure around it. Whether I’m using Google’s closed model Gemini or open-weight Gemma, I rely on enterprise agreements for compliance. Because my organization handles sensitive data, I’m cautious about using frontier research-tier models that require data retention.

I align my engineering organization to ISO 42001, the international standard for AI management systems. Using Google models off the shelf means Google’s compliance regime drops into my security program.  I’d hesitate to modify an open-weight model for production, even one I trust like Gemma, because my team would then have to manage compliance for the modified model.  If customization was a core concern, adding a RAG database to an agent or an organic workflow may deliver results similar to fine-tuning without modifying the model.

No matter how open or closed the model is, my organization carries the risk that an enterprise contract can help mitigate. I’d need to see outstanding results to justify the hidden costs of customizing models: legal review, audits, documentation, bias assessment — responsibilities that fall outside my engineering team’s core competencies. Those compliance constraints shaped how I think about governance at the organizational level, but the instinct started earlier.

How personal boundaries can inform healthy enterprise governance

When I built out my agent, I had to decide what I was comfortable with letting it access. ChatGPT and Claude can browse the internet by default, while with open-weight models, that’s a capability you have to enable. My open-weight-powered personal assistant wasn’t going to be useful without internet access — so I turned it on. I didn’t give the agent access to my email, but I did let it read my calendar and our family to-do list. Each decision came down to the same question: What’s the worst thing this agent could do with this access, and can I live with that? Internet browsing was worth the risk because the agent is far less useful without it, even though a malicious website could try to feed it bad instructions. Calendar access felt low risk; the worst outcome was a bad reminder. Email was where I drew the line. An agent that can send messages on my behalf can do real damage, and I’m not ready to accept that risk.

That same boundary applies to outputs. Just as I restrict what an agent can do with my data, I restrict what it can generate on my behalf. The model acts as an advisor and a critic, but the final work remains mine. As watermarking advances and scrutiny of synthetic content grows, today’s work will be auditable tomorrow. The inevitable backlash will fall on those who outsourced their critical thinking rather than using the tool to sharpen it. Keeping the model in an advisory role protects both personal integrity and work quality.

Where you should start

The realities of working with open-weight models — the compliance costs, the access decisions, the gap between experimentation and production  — are complex. For most organizations, wholesale switching isn’t realistic,at least not yet. But that doesn’t mean open-weight models aren’t worth exploring for specific use cases.

That’s why I advocate for a measured approach. Carve out some cycles for hands-on experimentation, but do it with eyes wide open about the practical limitations.

Here’s a process I recommend following.

1. Identify a specific use case or problem area where open-weight models could provide value.
2. Start small with a focused experimentation project, perhaps within a single team or department.
3. Engage with your compliance and legal teams early to understand the risks and requirements.
4. Document what you learn and share widely with your organization.
5. Use hands-on insights to inform your broader AI strategy and roadmap.

Open-weight models offer substantial architectural freedom, but that freedom is hollow without explicit operational guardrails. Before expanding these tools across the enterprise, establish clear lines around what data they touch, what they are permitted to generate and who carries the compliance burden when systems fail. Treating the model as an auditable advisor rather than an autonomous actor protects your organization from unnecessary legal exposure while capturing real productivity gains.