Optimizing AI Model Distribution in Multi-Tenant Kubernetes Environments

A hands-on lab with Dragonfly and vCluster demonstrates how a centralized model distribution layer can effectively reuse a public model across multiple tenants while maintaining control over node-level permissions.

In multi-tenant Kubernetes platforms, every team typically operates with its own API boundary, fostering independent workflows. Unfortunately, this setup can lead to redundancy where each team independently pulls large AI model files, causing unnecessary network loads and duplication of data storage.

By implementing a cache within each tenant cluster, it safeguards data ownership but limits the ability to share resources effectively. While a node-level cache offers broader data sharing capabilities, it often requires configurations like hostPath mounts or privileged containers, which can unnecessarily complicate security. To address this, I created a lab to test a platform-controlled peer-to-peer caching mechanism that would enable shared access to identical model payloads across two distinct virtual clusters, without compromising tenant workload security.

The Architecture of the Platform-Owned Data Path

In a manager-less configuration, Dragonfly operated solely on the host cluster, with task coordination managed by schedulers and seed peers. Each node ran a client DaemonSet that exposed an HTTP proxy on port 4001, wholly owned by the platform team.

Tenant A and Tenant B utilized separate open-source vCluster API servers for their model-download jobs, operating as non-root Pods. Their configurations avoided hostPath, hostNetwork, hostPID, and other elevated privileges. The Job fetched its node IP via Kubernetes' downward API and routed the model through the node-local Dragonfly proxy.

The download began with a HEAD request to Hugging Face to fetch the model file’s signed CDN URL. From there, the payload was retrieved via Dragonfly. A key detail of the proxy configuration ensured that specific signed query parameters were stripped from the cache identity, allowing both tenants to align their requests with the same content path, thereby facilitating reuse.

Figure 1. Host-managed Dragonfly data plane shared by two vCluster tenants.

Key Observations from the Test

During testing, Tenant A downloaded roughly 267,954,768 bytes, achieving a SHA-256 checksum of 5e3f1108e3cb34ee048634875d8482665b65ac713291a7e32396fb18f6ff0063 in approximately 11 seconds on the host control-plane node.

When Tenant B, positioned on a different worker node (dragonfly-host-worker2), made its request, it also received the same file size and checksum, but noted a download time of just three seconds. This supported the validity of shared task IDs assigned to both requests, along with logs that evidenced transfer from the control-plane client to Worker 2. Yet, I refrained from independently measuring the entire traffic load from the origin, meaning I couldn't assert absolute request reductions to Hugging Face.

Insights for Platform Engineering

This experiment illustrates a shift in perspective for caching as a platform service. By maintaining ownership over deployment and supervision, while exposing only minimal interfaces to tenants, one can streamline efficiency. In addition, cache identity becomes a vital architectural consideration; the normalization of signed model-hub URLs is critical, and specific methodologies shouldn't be arbitrarily transferred to private models without stringent controls.

Another discovery emphasized the need to prioritize transfer-source logs over mere timing. A quicker pull may hint at cache reuse but doesn’t definitively indicate the source of the data—be it local cache, remote peer, or original source. Lastly, just because APIs enforce separation doesn't equate to curtailing tenant interactions. My test didn't encompass adversarial scenarios, instead adhering to standard tenant workflows with constrained privileges.

Exploring Alternatives and Practical Guidelines

A platform team could also consider strategies such as a node-specific registry mirror, a pull-through cache, or a shared persistent-volume cache. The choice among these methods will rely on factors like artifact types, trust stipulations, and design for operational ownership. In this case, Dragonfly alongside vCluster offered a pragmatic way to assess peer-to-peer resource sharing while retaining cache control at the platform level.

The guiding principle I would advocate going forward is straightforward: consolidate repeated artifact distribution functions as a platform-centric capability, revealing only the narrowest necessary tenant interfaces while ensuring accurate transfer verification. Focus initially on public artifacts and explicitly defined cache keys, treating private model checks, eviction strategies, failure recoveries, and capacity assessments as distinctive production needs.

Author Bio

Pavan Madduri serves as a Senior Cloud Platform Engineer at W.W. Grainger and is the elected Tech Lead of CNCF’s TAG Workloads Foundation (2026-2028). He is active in contributing to Dragonfly, a widely recognized CNCF project, and holds titles as a vCluster Ambassador and CNCF Golden Kubestronaut.

Disclosure: The author participates in the vCluster Ambassador Program. This piece solely discusses open-source vCluster capabilities and has not received external review or sponsorship from vCluster Labs.