Eric Sloof

VMware’s latest technical paper, vSAN Availability Technologies, provides an in-depth look at how vSAN ensures data availability and resilience within modern software-defined data centers.

The publication focuses primarily on the vSAN Express Storage Architecture (ESA) — the foundation for VMware Cloud Foundation (VCF) 9.0 — and how it redefines storage reliability at scale.

The original Original Storage Architecture (OSA) relied on disk groups with separate cache and capacity devices. While effective for its time, this approach introduced larger “blast zones” during device failures.

In contrast, the Express Storage Architecture (ESA) uses a single-tier model, dramatically reducing failure boundaries and resynchronization times. The result is a leaner, faster, and far more resilient vSAN layer that can instantly recover compliance after transient or sustained outages.

vSAN’s data availability is governed by storage policies that define the number of failures an object can tolerate (FTT).

With ESA, these policies become even smarter. Administrators can use Auto-Policy Management, allowing vSAN to automatically select the optimal resilience level and data placement scheme based on the cluster configuration — for example, RAID-5 for smaller clusters or RAID-6 for environments with six or more hosts.

Each host in a vSAN cluster acts as its own fault domain, ensuring that data remains accessible even when individual hosts fail.

For more complex designs, administrators can define custom fault domains to represent racks or rooms, achieving rack-level resilience.

vSAN 9.0 within VCF also extends these capabilities with site maintenance mode and manual site takeover, improving the operational control of stretched and two-node clusters.

New concepts such as durability components ensure that even during maintenance or failure events, vSAN preserves the most recent writes.

The ESA also introduces low-level metadata mirroring, proactive hardware analytics, and NVMe endurance tracking, further strengthening resilience without requiring administrator intervention.

While vSAN provides continuous data availability and durability, data protection — recovery to a specific point in time — is achieved through features like vSAN Data Protection.

This native capability leverages the ESA’s snapshot engine for efficient local restores and integrates with vSAN-to-vSAN replication for remote protection. Together, they form a cornerstone for robust disaster recovery strategies within VMware Cloud Foundation 9.0.

The ESA represents a major step forward in vSAN’s evolution — simplifying design, reducing operational risk, and providing unmatched data availability for modern private and hybrid clouds.

For architects, administrators, and engineers designing resilient VCF infrastructures, the full whitepaper is a must-read:

👉 Download the complete “vSAN Availability Technologies” document

During the NSX upgrade, I ran into an issue where the "repo_sync" failed on the NSX Managers, resulting in an incomplete repository synchronization. After consulting VMware’s official documentation, I decided to try my own solution.

I connected to the NSX Manager via SFTP and discovered that the repository contents on one of the NSX Managers were not correctly synced. I copied the repository contents from a working NSX Manager to the failing NSX Manager.

After copying, I noticed that the file permissions were not aligned with the expected defaults. To ensure proper access, I manually adjusted the permissions of the synchronized repository files and directories to match those on the working NSX Manager.

Example commands used:

# Compare permissions on the working NSX Manager

ls -l /repository

# Adjust ownership to match the working node (typically uuc:grepodir)

chown -R uuc:grepodir /repository/<version>

# Ensure directory permissions are set correctly

chmod -R 770 /repository/<version

Finally, I executed the "resolve" option on the failed NSX Manager, which successfully fixed the issue.

On the working NSX Manager, there were two directories in the /repository folder:

• 4.2.0.2.0.24278654
• 4.2.3.0.0.24866349

There was also a file called current_version, which contained the text specifying the current version, in this case:

4.2.0.2.0.24278654

VMware Solution: Repo_Sync Status Fails on NSX Managers During Upgrade Pre-checks

Article ID: 392736
Last Updated: 06-27-2025

The NSX Manager upgrade pre-check fails with the following error on the NSX GUI:

"Failed to execute check for the version sync status of UC on all the MP nodes."

Additionally, the "Repository sync is not complete on all 3 NSX Managers" error appears on the NSX GUI.

Log snippets in var/log/nsxapi.log show errors similar to the following:

####-##-##T##:##:##.### INFO http-nio-127.0.0.1-7440-exec-10 RepositoryServiceImpl 4598 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" reqId="########-####-####-####-###########" subcomp="manager" username="uproton"] Repo-sync status for <IP of node> node is FAILED.

• NSX Version: 4.1.0.2

When upgrading NSX Managers from one version to another, the /repository folder on the Orchestrator node should contain both the current (from) version and the target (to) version directories. For example, when upgrading from 4.1.0.2 to 4.2.1.3, the /repository folder should contain both 4.1.0.2 and 4.2.1.3 directories.

VMware recommends following the KB steps to resolve the issue:

1. After replacing NSX Managers or during upgrade pre-checks, if the repo_sync fails, deploy both MUB files into the /repository folder of the Orchestrator node.

2. Once the repo_sync is successful on the Orchestrator node, click Resolve for repo_sync on the other two NSX Managers in the NSX GUI to sync the /repository folder across all nodes.

• Related KBs:

  • Repo_Sync Fails After Replacing Managers or During Upgrade Pre-checks

  • NSX Manager Upgrade Failures and Related Errors

By following the official VMware solution or the manual synchronization method I discovered, you can resolve the "repo_sync" failure issue during the NSX Manager upgrade process.

AI and machine learning are no longer experimental technologies—they have become essential in today’s enterprise IT landscape. While the training of AI models often happens in specialized environments, inference—the actual application of trained models—is increasingly being integrated into existing infrastructure. VMware Cloud Foundation (VCF) enables this through powerful distributed inference capabilities.

What is distributed inference?

Distributed inference allows AI models to be executed in parallel and at scale across multiple GPUs and clusters within VCF. This means inference tasks are not limited to a single server or GPU but can be distributed across the entire infrastructure. The result is faster processing and better utilization of available resources.

Integration in VMware Cloud Foundation

Within VCF, distributed inference is powered by the combination of:

• vSphere for compute virtualization and GPU partitioning

• NSX for secure and efficient network connectivity

• vSAN for high-performance, shared storage

• Tanzu Kubernetes Grid (TKG) for hosting and scaling AI workloads in containers

This integration enables inference workloads to be deployed, scaled, and secured with flexibility—without the overhead of managing a separate AI infrastructure.

Benefits for organizations

• Scalability: inference workloads can be distributed and scaled automatically across multiple GPUs.

• Efficient resource utilization: GPU capacity is used to its fullest, lowering costs.

• Security and isolation: with NSX micro-segmentation, AI workloads remain securely separated.

• Consistent infrastructure: AI applications run within the same trusted VCF environment as traditional workloads.

Conclusion

Distributed inference with VMware Cloud Foundation demonstrates how AI and enterprise IT are converging. By making inference workloads scalable, secure, and efficient within the existing infrastructure, organizations can accelerate the time-to-value of their AI models.

Want to learn more?

For a deep dive into all the concepts, configuration tips, and implementation best practices for distributed inference with VMware Cloud Foundation, get a copy of the full whitepaper.

Data security is at the top of the agenda for nearly every organization. Encryption plays a crucial role here: it ensures that data is only accessible to systems and users with the correct keys. VMware vSAN provides integrated encryption services that can easily be enabled at the cluster level.

Two Forms of Encryption

vSAN offers two types of encryption that can be used independently or together:

• Data-at-Rest Encryption: encrypts all data as it lands on physical storage devices.

• Data-in-Transit Encryption: secures all vSAN traffic transmitted between hosts in the cluster.

Both methods use the built-in VMkernel cryptographic module in vSphere, which has achieved FIPS 140-2 and 140-3 validation. This ensures compliance with the strictest security standards.

ESA versus OSA

The implementation of encryption differs significantly between the Original Storage Architecture (OSA) and the Express Storage Architecture (ESA).

• In the OSA, data-at-rest encryption is applied as the final step in the I/O path. This preserves the ability to leverage deduplication and compression, but it comes with higher resource overhead.

• In the ESA, encryption occurs higher in the stack, immediately after compression. This means that data only needs to be encrypted once, which reduces CPU and network usage and improves efficiency across the cluster.

Key Management: KMS or Native Key Provider

There are two main options for managing encryption keys:

1. External Key Management Server (KMS) – based on the KMIP protocol and often deployed in a highly available cluster configuration.

2. vSphere Native Key Provider (NKP) – a built-in key management solution that operates entirely within vSphere, and can leverage TPM chips on hosts for additional resilience and secure key persistence.

VMware strongly recommends using TPM chips on all vSAN hosts to ensure keys remain securely stored and available even if the key provider is temporarily unreachable.

Conclusion

vSAN Encryption Services provide organizations with powerful tools to protect sensitive data. With the ability to secure data both at rest and in transit, and flexible options for key management, vSAN ensures compliance, resilience, and peace of mind. Especially with the ESA architecture, organizations benefit from stronger security with lower performance overhead.

Want to learn more?

For a deep dive into all the available techniques, trade-offs, and configuration details, get a copy of the full whitepaper.

Storage capacity remains a critical concern in modern datacenters. Organizations want to host more and more workloads, while physical storage growth is often expensive or limited. VMware vSAN addresses this challenge with a range of space efficiency technologies that help reduce costs and improve performance.

Opportunistic vs. Deterministic Techniques

The whitepaper distinguishes between two categories of techniques:

• Opportunistic: dependent on the characteristics of the data, which means savings are not guaranteed. Examples include deduplication & compression (OSA), compression-only, thin provisioning, and TRIM/UNMAP space reclamation.

• Deterministic: delivers predictable and guaranteed space savings. This is the case with RAID-5/6 erasure coding.

Both types can be used independently or combined to maximize efficiency.

ESA versus OSA

With the introduction of the vSAN Express Storage Architecture (ESA) in vSAN 8, the options have significantly improved. While the Original Storage Architecture (OSA) had limitations around performance and deduplication domains, ESA provides much more effective data reduction without noticeable performance trade-offs.

• Compression in ESA: Data is compressed as soon as it enters the vSAN stack, reducing CPU load and network traffic.

• Global Deduplication in ESA: The deduplication domain spans the entire cluster, making data reduction far more effective than OSA’s per–disk group approach.

For existing environments still running on OSA, deduplication & compression or compression-only remain valid options, but they come with more performance considerations.

Thin Provisioning and TRIM/UNMAP

With thin provisioning, storage is only allocated when actually needed. This allows oversubscription, a common practice to make better use of capacity.

A well-known drawback is that released space inside a VMDK is not automatically reclaimed. TRIM/UNMAP solves this issue by freeing unused blocks. This not only increases usable capacity but also speeds up recovery and rebalance operations.

Erasure Coding: RAID-5 and RAID-6

Where RAID-1 mirroring always creates a full copy of the data—consuming large amounts of storage—erasure coding provides a more efficient alternative.

• RAID-5 (FTT=1) consumes 1.33x the original data in OSA, or just 1.25x in ESA.

• RAID-6 (FTT=2) consumes 1.5x the original data, compared to 3x with mirroring.

In ESA, these techniques are available without any performance trade-offs, making them the logical standard for most environments.

Conclusion

By combining opportunistic and deterministic space efficiency techniques, vSAN allows organizations to use storage more intelligently and cost-effectively. Especially with ESA, both performance and efficiency are greatly enhanced. For environments still running OSA, migrating to ESA is a worthwhile step to optimize both capacity and performance.

Want to learn more?

For a deep dive into all the available techniques, trade-offs, and configuration details, get a copy of the full whitepaper.