The enterprise MCP scaling paradox: why every agent needs a tailored backend

The Model Context Protocol (MCP) has standardized how AI applications connect to external tools and systems. As organizations move from prototypes to wider deployment, the next challenge is about operational scale: how to scope, govern, deploy, and observe a growing MCP estate.

According to recent Gartner projections, up to 40% of enterprise applications will feature task-specific AI agents by 2026 (see https://www.gartner.com/en/newsroom/press-releases/2025-08-26-gartner-predicts-40-percent-of-enterprise-apps-will-feature-task-specific-ai-agents-by-2026-up-from-less-than-5-percent-in-2025). This isn't just a coding challenge; it's an architectural upheaval. The infrastructure currently being built to support these agents is fragmented, bloated, and dangerously uncoupled from traditional enterprise governance.

If your strategy is to spin up a new MCP server for every business service, the operational burden compounds quickly.

Two fronts: third-party and internal systems

Scaling the MCP landscape involves two problems, both difficult.

The first one is external third-party tooling. Teams want to connect agents to SaaS platforms, CI/CD pipelines, ticketing systems, cloud APIs - the standard Software Development Life Cycle toolchain. Vendors increasingly ship MCP Servers for this, which sounds convenient until you look closer.

Official MCP Servers from tool vendors tend to be maximalist by design. They expose everything. Some vendor-provided MCP servers expose broad tool catalogs, with far more tools than a particular workflow needs, which means agents receive tool catalogs with dozens or hundreds of entries - far more than any single workflow needs. This has real costs: token consumption increases, latency climbs, and the attack surface for prompt injection or privilege escalation grows with every unnecessary tool exposed. The lack of per-agent tool-selection policy model is also pushing access constraints to be set at the infrastructure or application layers.

On-premises deployments add further friction. An MCP Server that works seamlessly in a SaaS context becomes a hosted service you now own: provisioned, monitored, patched, and scaled by your platform team. And the cost model for MCP-as-a-bundle is still evolving - not all vendors have figured out how to price it, and some haven't shipped it at all.

The second front is internal systems. This is where the deeper scaling problem lives. The challenge inside the firewall is equally daunting. Organizations are seeing a sprawl of "shadow-developed" MCP servers.

The internal MCP server scaling problem

Most enterprises have spent the last decade building out internal APIs - REST, gRPC, GraphQL - that represent real institutional investment. The promise of agentic workflows is that these systems become programmable via natural language interfaces. The reality is that somebody has to write the MCP Server that sits in front of each of them.

Sourcing is a real constraint

Who builds these servers? Well: whoever is available. This pattern has already produced a recognizable anti-pattern: MCP Servers are being developed informally, with unclear ownership, limited documentation, and handed off to operations teams without context. The institutional knowledge embedded in these implementations walks out the door when the developer does.

This isn't a criticism of any individual - it reflects a structural gap. There's no established ownership model for MCP Servers yet, and the organizations that haven't defined one are quietly accumulating technical debt.

Building requires expertise that doesn't compose well

Frameworks like Spring AI, Quarkus MCP, and FastMCP have lowered the barrier to MCP Server development considerably. But "lower" isn't "zero." Writing a production-grade MCP Server still requires fluency across multiple domains simultaneously: MCP protocol semantics, network proxy behavior, authentication integration (SPIFFE? API keys? OAuth? all three?), and observability instrumentation. This isn't a reasonable expectation for every backend developer on every team.

A first observation is that the resulting code is usually boilerplate-bloated. Very few MCP Servers really hold business logic other than adaptation and maybe orchestration. The final result is also very fragmented. Teams independently choose Python or Java or TypeScript, integrate with identity providers in different ways, instrument differently, and deploy differently. When the CISO team needs to review a new MCP Server, they're effectively auditing a novel piece of infrastructure each time - there's no shared baseline to evaluate against.

> A question worth asking: Is the real problem MCP Server proliferation itself, or is it the absence of a platform contract that defines what a compliant MCP Server looks like in your organization? These have different solutions.

The operational burden compounds

Perhaps the most underappreciated dimension of internal MCP scaling is what happens when adoption succeeds. If agentic workflows become genuinely useful - Deloitte's 2026 Tech Trends report projects that by 2028, one-third of enterprise software will embed agentic AI, enabling 15% of daily work decisions to be made autonomously (see https://www.deloitte.com/us/en/insights/topics/technology-management/tech-trends/2026/agentic-ai-strategy.html\&sa=D\&source=docs\&ust=1784195584577063\&usg=AOvVaw1PaRp1rHU7oXImPRCOG511) - then the number of MCP Servers in a typical enterprise won't be ten or twenty. It will be in the hundreds.

Each one is a service: compute, memory, network, on-call rotation. If every internal system surfaces multiple MCP Servers (tuned for different agent personas, security contexts, or workflow scopes), resource consumption could plausibly double. Capacity planning for this is genuinely complex, because agentic workloads are non-deterministic and usage patterns don't follow the curves infrastructure teams are used to modeling.

The infrastructure implications of a successful adoption trajectory are not yet well understood.

The core tension: generic APIs vs. context-aware tools

There's a design mismatch at the heart of this problem that doesn't get enough attention.

Existing enterprise APIs are built for broad reuse. A POST /query endpoint that accepts arbitrary SQL is useful because it's general. But agentic clients are most reliable when tools are narrow and explicit - a get_revenue_for_month(month, year) function gives an agent less room to go wrong than execute_sql(query) - a common pattern from vendors' MCP servers, though. The former constrains the parameter space, reduces hallucination risk, and makes authorization easier to reason about.

This means that even when a perfectly good internal API exists, it often needs a translation layer before it's suitable for agent consumption. That translation layer is where most of the real engineering work lives - and it's currently invisible in most teams' architecture diagrams.

Emerging approaches

Several patterns and tools are beginning to address this problem space. None of them is a complete solution yet, and the ecosystem is moving fast.

API Gateway-centric approaches propose treating MCP exposure as a concern of the API gateway layer, rather than individual service teams. If your gateway already enforces authentication, rate limiting, and observability, extending it to translate HTTP APIs into MCP tool definitions is architecturally cleaner than deploying N independent MCP Servers. Projects like agentgateway (an AAIF project) are exploring this direction. The tradeoff is that gateways traditionally haven't carried semantic knowledge about what a tool means, only how to route to it - and that gap matters for agent reliability.

Declarative-driven MCP generation offers to eliminate handwritten server code by deriving MCP tool definitions from existing API contracts - OpenAPI specs, AsyncAPI, GraphQL schemas or gRPC service definitions. If the API is already formally described, the MCP layer can, in principle, be generated or interpreted on the fly. Various open-source initiatives and tools are working in this space. The open question is how a schema - that tells you the shape of the data, not when it's appropriate to call - should be tuned and enriched to communicate the contextual intent that agents actually need.

The Backend for Agents (BFA) pattern - articulated in several recent architecture posts - proposes a dedicated aggregation layer that assembles tool subsets, enforces policy, and presents a purpose-built surface to each agent persona. This is similar in spirit to the Backend for Frontend pattern from microservices. It's conceptually sound, but it introduces another tier of infrastructure and shifts complexity rather than eliminating it.

Platform-level MCP governance is the least mature but arguably most important piece. The idea is that an organization defines a validated platform for MCP Server deployment - with pre-integrated identity, observability, and policy controls - so that individual teams can deploy servers without each one requiring a full security review. This is analogous to how internal developer platforms (IDPs) work for microservices today. The challenge is that building this requires organizational investment and cross-team alignment, which doesn't happen by itself.

> A challenge to the "one platform" framing: Platform standardization has historically required a team with enough credibility and resources to define and enforce it. In enterprises where the CISO, platform engineering, and application teams all have different incentive structures, who owns the MCP platform? This is a governance question as much as a technical one, and the technical community tends to underweight it.

What open source and open governance offer here

The value of having MCP infrastructure in open-source, openly governed projects goes beyond cost. It's about auditability and trust.

When a security team reviews an MCP Server, they're asking: what can this thing do, and can we verify it? Proprietary or undocumented MCP infrastructure can make that question harder to answer. Openly governed implementations can provide a shared, inspectable baseline that security and audit teams can review and reuse across deployments.

This is particularly relevant for the identity integration problem. MCP Servers that integrate with enterprise identity providers need to handle tokens, scopes, and delegation correctly. Getting this right in isolation for each MCP Server is both expensive and error-prone. A platform-level open-source implementation that provides a consistent identity integration point can reduce duplicated work and make security controls easier to apply consistently.

Observability follows the same logic. Agentic workloads are non-deterministic by nature, which makes them genuinely hard to debug, cost-attribute, and audit - and when every MCP Server emits telemetry in its own shape, that difficulty compounds across the hundreds of servers a successful adoption produces. A security team trying to answer "what did this agent actually do, and what did it cost?" ends up correlating traces that don't share a vocabulary. This is the observability equivalent of the fragmentation problem described earlier: teams instrument differently, so there's no baseline to reason against.

The OpenTelemetry GenAI semantic conventions offer an open, governed approach to that gap. Developed in the open under the CNCF, they define a shared schema for agent, tool, and model telemetry - standard span names, attributes, and metrics for things like tool invocations, token usage, and model calls. Adopted at the platform level, they let hundreds of MCP Servers become observable against one baseline rather than a patchwork of bespoke instrumentation. That shared vocabulary is what makes the capacity-planning problem tractable - you can actually model non-deterministic workload growth when the telemetry is consistent - and it is what turns "can we verify what this thing does?" into a question audit processes can answer. Solving telemetry once, via an open convention, is the same move as solving identity once: a platform contract that individual teams inherit rather than reinvent.

Where this is headed

The current phase of MCP adoption in enterprises looks a lot like the early days of microservices or container adoption: teams are moving fast, patterns are emerging organically, and the operational debt is accumulating quietly. The organizations that will fare best are those that treat MCP infrastructure as a platform concern early - before the proliferation problem becomes a remediation problem.

The technical community can help by being precise about what we're actually solving for. "Deploy an MCP Server" is not a complete unit of work. The complete unit includes: scoping the tools appropriately for the target agent context, integrating with the organization's identity model, instrumenting for observability, establishing an ownership and maintenance model, and fitting into a platform and capacity planning framework that can accommodate non-deterministic workload growth.

That's a harder problem than it first appears. But it's the actual problem - and naming it clearly is the prerequisite to solving it.

> The projects and patterns referenced in this post are evolving rapidly. If your organization is working through these challenges and has observations worth sharing, the AAIF working groups and CNCF SIGs are active venues for that conversation.