The federal government spends more than $100 billion a year on IT and cyber-related investments, and agencies have reported about 80% goes to operating and maintaining existing IT. As of February 2025, agencies had completed just three of the 10 critical legacy modernizations GAO identified in 2019. The numbers reveal a federal technology estate that is expensive to maintain and slow to modernize.
It’s easy to understand why federal IT is slow to upgrade. The systems managing services like tax, benefits and health operate at national scale and support services people rely on every day. Agencies can’t simply take those systems offline while modernization is underway. New capabilities such as AI have to be introduced alongside the existing technology that keeps those services running.
So how do you deploy AI while the legacy systems delivering those services remain online? You start by deciding what needs to change, what can stay in place, and which workloads belong on a common infrastructure.
Not all legacy systems are the same
“Legacy” means different things across government, and the kind of legacy system you’re operating will influence what you do next. Some legacy systems date to the mainframe era, with hardware, operating systems, and applications tightly coupled over decades.
GAO found eight of the 11 most critical systems it reviewed in 2025 use outdated programming languages, while four have unsupported hardware or software. These systems are among the hardest and riskiest to change directly. Depending on their supportability, security risk, and dependencies, you may need to replace or retire them, while in other cases new services can be introduced alongside them as their own modernization proceeds.
Other legacy systems are simply older applications running on virtual machines in an agency data center. In these cases, you can modernize the underlying VM infrastructure without changing the application running on top of it. Servers, virtualization, storage, networking, and orchestration make up that infrastructure, whether it sits in a data center, a cloud environment, or at the edge.
AI introduces another consideration. Agencies now need to run workloads that place different demands on the infrastructure, including specialized compute, access to data held in other systems, and their own security and performance requirements. The question becomes how to add those capabilities without creating another isolated environment alongside what already exists.
What changes when the new workload is AI?
AI introduces infrastructure requirements existing environments were not necessarily designed to handle.
Where will inference run? Data sensitivity, network isolation, and latency can influence whether an AI workload runs in an agency data center, a government cloud region, or an edge location.
How will the model access permitted data? The information an AI service needs often remains inside existing systems, so access has to work through approved interfaces and existing authorization.
What computing resources does the workload need? Many AI workloads require GPUs or other specialized resources that traditional CPU-oriented virtualized environments were not designed around.
If you answer those questions independently for every new AI workload, then each project will end up with its own infrastructure stack, access path, and compute environment. Over time, the infrastructure supporting AI will become fragmented just like the application environments around it.
Gartner identified siloed strategies and legacy systems as key challenges to adopting digital solutions, cited by 41% and 31%, respectively, of 138 respondents from government organizations worldwide in a survey conducted between July and September 2025.
Infrastructure management determines whether each new AI workload adds another isolated environment or fits into the technology stack you already have in place. A common infrastructure gives agencies a way to handle those requirements consistently without designing the underlying environment from scratch for every workload.
Where a common infrastructure helps
A common infrastructure breaks that cycle by giving new workloads a shared foundation from the start.
A common operating platform becomes useful when several workloads have infrastructure needs that can be managed together. VM-based applications running on aging virtualization infrastructure are candidates for consolidation. New containerized services also fit when each project would otherwise create its own environment.
Other workloads can stay where they are. Mainframe systems and tightly coupled applications can connect to newer services through approved interfaces when moving them would create unnecessary risk or require a significant modernization effort.
Standardizing the infrastructure beneath your applications lets you manage the underlying environment consistently without forcing the applications themselves into the same mold. Data center, cloud, and edge environments follow shared practices while each workload retains its own lifecycle, interfaces, and security requirements. A long-established benefits application still needs a different testing and release process from a new AI service, while the infrastructure supporting each can follow consistent practices.
Change management becomes more consistent when workloads share an operating platform. You can validate dependencies, test changes in a production-like environment, monitor deployments, and maintain a defined path back if something goes wrong. Each project no longer has to establish those capabilities independently. The infrastructure evolves underneath existing applications while new workloads use the same foundation.
Putting it into practice
Say you run a benefits office and want to use AI to summarize incoming correspondence so caseworkers can triage it faster, while caseworkers continue to make every eligibility decision themselves. The service needs documents from an imaging system, case data from a mainframe, and a place to put its output where a caseworker can see it. You don't need to move all three systems to make the new service work. You need to decide where each one belongs and how the systems connect.
The mainframe case system stays where it is, with the new service reaching it through an existing interface rather than replacing it. The imaging system, however, runs on VM infrastructure that's overdue for a refresh, which makes it a natural candidate for a common operating platform already supporting other workloads across the agency. That gives you a path to introduce the AI service without building an entirely new environment around it. GPU support, security controls, and monitoring are already available on the platform, so you're extending infrastructure that is already running and can reuse appropriate controls and evidence rather than creating a new stack duplicating security work.
Without a shared platform, the imaging system and AI service may require separate infrastructure, approvals, and security work on different timelines. If you already have a platform with suitable GPU capacity, controls, and management practices in place, you can reuse that foundation where it fits rather than building a new environment from scratch.
With the shared platform in place, you can test the summarizer against past cases before it touches live work. If the results don't meet requirements, you can retain a manual triage path and limit the initial deployment to one office before expanding further.
You can also manage separate environments consistently when workloads need to remain isolated.
How federal agencies are currently deploying AI
In December 2024, the Office of Personnel Management announced a two-year effort to modernize COBOL code in its federal retirement system, some of it roughly 40 years old, using AI to rewrite the code with staff reviewing the results.
The IRS runs at least one system on a programming language roughly 60 years old and is using AI tools to help staff understand and rewrite that code, which the IRS says will speed modernization.
The Social Security Administration added generative AI to a specific workflow for recording and transcribing disability hearings and announced in March 2025 that it was finishing the nationwide rollout. The system is expected to benefit about 500,000 customers each year.
The three cases show different ways agencies are introducing AI alongside existing technology. OPM and the IRS are using AI to help modernize legacy code, while SSA is using it in an operational workflow. They don’t show that old and new workloads need to share the same infrastructure. But where infrastructure requirements overlap, a common infrastructure layer can give agencies a consistent way to support new AI workloads as they are introduced.
Spectro Cloud helps agencies operate a changing technology estate
Selective modernization leaves agencies running a mix of old and new infrastructure. A mainframe may stay in place while some VM-based applications move and new AI services arrive, and the infrastructure team still has to secure and maintain all of it.
Palette VerteX is Spectro Cloud's platform for government environments and provides a common platform for that changing mix, with FIPS 140-3 validated cryptography and support for air-gapped deployments. PaletteAI VerteX extends the same foundation to AI workloads.
For workloads you choose to move, Palette VerteX can run virtual machines alongside containers on the same platform, and its migration tools can bring over VMs from VMware vSphere 7 and 8 when their operating systems are supported.
The platform also provides the infrastructure capabilities needed to make AI deployments repeatable. Cluster profiles capture the full environment as a reusable blueprint and allow test environments to match production and deployments to start from the same definition. GPU and bare metal provisioning provides the specialized compute needed for AI workloads, while PaletteAI VerteX adds lifecycle management for larger AI environments.
If you're looking into how to introduce AI while continuing to operate the systems your agency already depends on, book a demo to see how Palette VerteX makes it possible.