Data transformation takes flight at Atlanta’s Hartsfield-Jackson airport

At Atlanta’s Hartsfield-Jackson International Airport, an IT pilot has led to a wholesale data journey destined to transform operations at the world’s busiest airport, fueled by machine learning and generative AI.

Jon Pruitt, director of IT at Hartsfield-Jackson Atlanta International Airport, and his team crafted a visual business intelligence dashboard for a top executive in its Emergency Response Team to provide key metrics at a glance, including weather status, terminal occupancy, concessions operations, and parking capacity.

The original proof of concept was to have one data repository ingesting data from 11 sources, including flat files and data stored via APIs on premises and in the cloud, Pruitt says.

The dashboard now in production uses Databricks’ Azure data lake to ingest, clean, store, and analyze the data, and Microsoft’s Power BI to generate graphical analytics that present critical operational data in a single view, such as the number of flights coming into domestic and international terminals and average security wait times.

“The general manager needed a centralized view of our data, as opposed to accessing tabs in Excel spreadsheet and other sources, which was very cumbersome,” Pruitt says. “He is a very visual person, so our proof of concept collects different data sets and ingests them into our Azure data house. That enables the analytics team using Power BI to create a single visualization for the GM.”

That effort proved to be just the first phase of a major data and AI transformation under way at Hartsfield-Jackson, which began in earnest more than a year ago after the executive team saw the dashboard proof of concept and gave Pruitt and his IT team the green light to invest further in digitally enhancing the airport’s operations.

Since implementing the dashboard, the operations GM has enjoyed an 80% improvement in time to insight, from 30 minutes to 5 minutes, which allows much speedier problem resolution and increased opportunities for spotting ways to improve operations.

Phase 2: Data governance at scale

Once the Department of Aviation executive team saw the value of the dashboard, the IT team began planning phase two, which will incorporate data governance and data cleansing as well as analytics for departments besides operations, including finance, technology, commercial, administration, and infrastructure, Pruitt says.

As part of its plan, the IT team conducted a wide-ranging data assessment to determine who has access to what data, and each data source’s encryption needs. “There are a lot of variables that determine what should go into the data lake and what will probably stay on premise,” Pruitt says.

Data integrity presented a major challenge for the team, as there were many instances of duplicate data. Identifying and eliminating Excel flat files alone was very time consuming. Building this single source of truth was the only way the airport would have the capacity to augment the data with a digital twin, IoT sensor data, and predictive analytics, he says.

Data analysis at Hartsfield-Jackson will move beyond BI to machine learning and generative AI models from Microsoft and Databricks to automate tasks, generate new insights, and enable the airport to be more efficient and more profitable.

The Atlanta airport has partnered closely with Databricks, which “rents out” its data platform to Microsoft to create a custom Azure Databricks platform that is cloud-agnostic, Pruitt says. Like many corporate enterprises, Hartsfield-Jackson has taken a multi-cloud approach, with Microsoft Azure as its primary cloud but also uses AWS and Google Cloud for specific workloads.

According to Pruitt, one major benefit of partnering with a cloud-agnostic data giant such as Databricks and developing a sophisticated data governance strategy is “just being able to have a single source of truth.”

“It’s a big win for us — being able to look at all of our data in one repository and build machine learning models off of that,” he says. “This allows us to excel in this space, and we can see some real-time ROI into those analytic solutions.”

Pruitt says the airport’s new capabilities provide data-driven insights for improving operations, passenger experience, and non-aeronautical revenue across airport business units.

Applying AI to elevate ROI

Pruitt and Databricks recently finished a pilot test with Microsoft called Smart Flow. The application enables IT to select a period of time to analyze passenger traffic — say, 50,000 customers between January and July in one area of the airport. By applying BI and predictive forecasting, Smart Flow then helps airport operations determine whether it should offer additional services and products to those passengers or, during periods of reduced foot traffic, reassign resources to more heavily trafficked areas of the airport.

 “We can generate a lot of revenue from this,” Pruitt says, by examining this utilization data and highlighting premier areas.

Recently, Databricks met with Pruitt and airport executives to discuss Phase 2’s application of machine learning and large language models (LLMs) to the data — the most significant aspects of the project, which will allow Pruitt to generate insights, for example, about how to better utilize airport gates and improve security wait times.

“They want to go from a reactive state to a proactive state where they’re able to start predicting asset management and operations and become far more efficient,” says Will McKinney, a strategic account executive at Databricks who oversees data-driven decisions for state governments.

For instance, Delta “owns” or leases 169 gates out of the total 199 gates, making Atlanta the hub for that carrier. With the new technology in place, Hartsfield-Jackson can discover whether gates are under-utilized, and if so, renegotiate leasing arrangements with big carriers.

“They’re trying to get a handle on their data estate right now. Once they have that, they can start applying the data science and machine learning to predict how they can be more efficient with the gates,” says McKinney, who has partnered with Pruitt on the project.

Paul Baier, CEO and principal analyst at GAI Insights, says investment in gen AI in particular will offer airports significant opportunities to save money and improve customer satisfaction.

“So much of these processes require data that are unstructured schedules on wiki or SharePoint, technical manuals in PDF, list of vegan restaurants, for instance, and gen AI excels at finding value with unstructured data,” Baier says. “Moreover, gen AI is fantastic at language translation tool and with AI chips coming to smartphones next years, airports will have the ability to increase customer satisfaction by cost-effectively offering information in 100 languages on smartphones.”