Beyond the hype: Do you really need an LLM for your data?

As someone deeply involved in shaping data strategy, governance and analytics for organizations, I’m constantly working on everything from defining data vision to building high-performing data teams. My work centers around enabling businesses to leverage data for better decision-making and driving impactful change. Recently, my involvement with IASA and SustainableIT.org has given me a new lens through which to view these projects: sustainability. Thanks to some thought-provoking conversations, I’m now looking at everything with a more environmentally conscious mindset. This article reflects some of what I’ve learned. 

The hype around large language models (LLMs) is undeniable. They promise to revolutionize how we interact with data, generating human-quality text, understanding natural language and transforming data in ways we never thought possible. From automating tedious tasks to unlocking insights from unstructured data, the potential seems limitless. But here’s the question I keep asking myself: do we really need this immense power for most of our analytics?  

Think about it: LLMs like GPT-3 are incredibly complex deep learning models trained on massive datasets. We’ve all seen the demos of ChatGPT, Google Gemini and Microsoft Copilot. They’re impressive, no doubt. In analytics, LLMs can create natural language query interfaces, allowing us to ask questions in plain English. They can also automate report generation and interpret data nuances that traditional methods might miss. Imagine generating complex narratives from data visualizations or using conversational BI tools that respond to your queries in real time. In life sciences, LLMs can analyze mountains of research papers to accelerate drug discovery. In retail, they can personalize recommendations and optimize marketing campaigns. These potential applications are truly transformative.  

LLMs offer compelling capabilities in natural language processing, automation and complex data interpretation 

But let’s get real. What do most organizations actually need from analytics? In my experience, particularly during my time at Parexel and even working with various clients at Cleartelligence, it often boils down to core needs like  

• Clear data visualization  
• Solid descriptive analytics (trends, KPIs) 
• Reliable predictive analytics (forecasts)  
• Easy-to-use dashboards 

While at Parexel, the focus was often on analyzing clinical trial data to identify trends in patient outcomes, site selection based on past performance and predict the success of future trials. Working with clients at Cleartelligence, the needs have been quite diverse. For example, at a company providing manufacturing technology services, the priority was predicting sales opportunities, while at a company that designs and manufactures automatic test equipment (ATE), it was developing a platform for equipment production automation that relied heavily on forecasting. Despite the different contexts, the underlying need for reliable, actionable insights remained constant. 

And guess what? We already have excellent tools for these tasks. Tableau, Qlik and Power BI can handle interactive dashboards and visualizations. SQL can crunch numbers and identify top-selling products. Even basic predictive modeling can be done with lightweight machine learning in Python or R. In life sciences, simple statistical software can analyze patient data. In retail, basic database queries can track inventory. You get the picture. These traditional tools are often more than sufficient for addressing the bread-and-butter analytics needs of most businesses.  

Existing tools and methods often provide adequate solutions for many common analytics needs 

Here’s the rub: LLMs are resource hogs. Training and running these models require massive computing power, leading to a significant carbon footprint. Sustainable IT is about optimizing resource use, minimizing waste and choosing the right-sized solution. Using an LLM to calculate a simple average is like using a bazooka to swat a fly. I’ve seen this firsthand. At a client in the high-end furniture sales industry, we were initially exploring LLMs for analyzing customer surveys to perform sentiment analysis and adjust product sales accordingly. The allure of using cutting-edge AI was strong, but after a thorough assessment, we recommended a more practical approach. By leveraging existing natural language processing libraries within a Python environment, we could achieve the desired sentiment analysis with a fraction of the computational resources, significantly reducing both cost and environmental impact. 

Typically, the initial excitement about the “latest and greatest” technology can blind us to practical considerations. Another client that comes to mind is a company that monetizes operational benchmarking of clinical facilities. Their data primarily consisted of a huge volume of member surveys. While an LLM comes to mind when we consider survey analysis, we demonstrated that simpler tools like Snowflake and dbt were perfectly capable of efficiently analyzing trends and generating valuable insights without the resource burden of an LLM. This not only saved the client significant costs but also aligned with their commitment to sustainable operations. This experience reinforced the importance of carefully evaluating the true needs of a project before jumping to the most complex solution.  

LLMs have a significant environmental impact due to their high energy consumption 

So, when are LLMs worth considering? When you’re dealing with truly complex, unstructured data like text, voice and images. Think sentiment analysis of customer reviews, summarizing lengthy documents or extracting information from medical records. They’re also useful for dynamic situations where data and requirements are constantly changing. And let’s not forget the enhanced user experience of natural language queries. 

For example, a client that designs and manufactures home furnishings uses a sophisticated modeling approach to predict future sales. They leverage around 15 different models. The results of these models are then combined using a simple algorithm to determine the best-performing model for a given item, which is then used for prediction. While this process is complex and data-intensive, it relies on structured data and established statistical methods. An LLM would be overkill for this type of analysis. However, imagine if this furniture manufacturer also wanted to incorporate customer reviews, social media sentiment and even images of room designs into their sales predictions. This is where an LLM could become invaluable, providing the ability to analyze this unstructured data and integrate it with the existing structured data models. This type of complex, multi-modal data analysis, where structured and unstructured data converge, is precisely where LLMs can shine.  

Another compelling use case is in the automotive industry. While my experience consulting at American Honda Motors almost three decades ago involved evaluating manufacturer engineering guides (details of which are now somewhat hazy), I can easily envision how an LLM-powered AI assistant could revolutionize this process today. Imagine such a system processing unstructured text data like historical maintenance logs, technician notes, defect reports and warranty claims, and correlating it with structured sensor data such as IoT readings and machine telemetry. This could provide predictive maintenance insights, identify design flaws and ultimately improve vehicle reliability and safety. These types of complex, multi-modal data analyses, where structured and unstructured data converge, are precisely where LLMs can shine. These examples highlight the power of LLMs to unlock insights hidden within unstructured data but also underscore the importance of using them strategically.  

LLMs are best suited for complex, unstructured data, dynamic use cases and enhancing user experience through natural language 

But even then, a hybrid approach is often best. Use traditional tools for structured data and reserve LLMs for the truly complex stuff. This approach allows organizations to leverage the strengths of both traditional analytics tools and LLMs, maximizing efficiency and minimizing resource consumption. It’s about finding the right balance between power and practicality.  

The path to sustainable analytics is about choosing the right tool for the job, not just chasing the latest trend. It’s about investing in skilled analysts and robust data governance. It’s about making sustainability a core priority. This means fostering a culture of data literacy and empowering analysts to critically evaluate the tools and techniques at their disposal. It also means establishing clear data governance frameworks to ensure data quality, security and ethical use. 

In the race for AI dominance, let’s not forget that the simplest solution is often the most sustainable. Let’s not use a sledgehammer when a well-placed tap will do. As IT leaders, we need to be the voice of reason, ensuring that technology decisions are driven by business needs and sustainability considerations, not just hype. By embracing a pragmatic and sustainable approach to analytics, we can unlock the true potential of data while minimizing our environmental impact. 

Chitra Sundaram is the practice director of data management at Cleartelligence, Inc., with over 15 years of experience in enterprise data strategy, governance and digital transformation. She specializes in data-driven decision-making, cloud modernization and building scalable data governance frameworks to drive business success. Chitra is a member of the IASA CAF and SustainableArchitecture.org communities. She is interested in helping expand its membership with IT architects interested in ensuring ESG mandates in IT are met. 

This article was made possible by our partnership with the IASA Chief Architect Forum. The CAF’s purpose is to test, challenge and support the art and science of Business Technology Architecture and its evolution over time as well as grow the influence and leadership of chief architects both inside and outside the profession. The CAF is a leadership community of the IASA, the leading non-profit professional association for business technology architects.