WilliamLam.com

Quick Tip: The Fastest Way to Clear Partitions for ESX Reinstallation

by // 1 Comment

I often redeploy my VMware Cloud Foundation (VCF) lab environment for a variety of reasons, from testing customer scenarios to testing the next version of our software. Because I have fully automated the ESX installation and configuration process in my lab, reinstalling ESX is a trivial task that takes less than 5 minutes from power-on to a fully configured system ready for VCF.

When reinstalling ESX, especially on systems configured with NVMe Tiering, vSAN, and local VMFS storage, the real challenge is clearing the existing partitions so the system is ready for a fresh installation. Traditionally, this can involve several manual steps and, in some cases, booting from a Linux live CD to remove stubborn partitions.

Given how frequently I redeploy my environment, I have developed the following workflow that consistently cleans up the various storage configurations without requiring a separate live CD environment. As I was performing yet another ESX reinstallation while writing this post, I realized I had never documented my workflow, so I thought I would share it in case it helps others.

[Read more...]

Quick Tip: Reducing High CPU Utilization in VCF Automation (VCFA) on AMD Zen4/Zen5 CPUs

by // 3 Comments

A couple of weeks ago, I received an interesting email from a reader who had observed high CPU utilization after upgrading VCF Automation (VCFA) to the latest VCF 9.1 release. They were running VCF 9.1 on the popular Minisforum MS-A2 platform, which uses AMD Zen4/Zen5 (Ryzen) consumer processors which are not supported on the Broadcom Compatibility Guide (BCG). As I have covered previously (here and here), these processors have been known to exhibit slower entropy generation that can lead to higher than expected CPU utilization without the appropriate optimizations.

With the help of Claude to analyze the issue, the reader arrived at the following conclusion:

Five JVM-based services were dying repeatedly during startup because FIPS_MODE=strict was slowing down TLS handshakes (both database and internal service-to-service HTTPS) enough to occasionally exceed connection timeouts. Every restart meant a full JVM boot: Spring context initialization, Liquibase migrations, Hibernate, index rebuilding, which is expensive. The crash loop itself became the CPU load. Setting FIPS_MODE=disabled on the affected deployments stopped the crash loop entirely, and CPU usage returned to near-idle.

Interestingly, around the same time, I also heard from an internal colleague who had independently observed that disabling FIPS for specific VCFA services significantly reduced CPU utilization. Since FIPS is enabled by default for VCF 9.1 deployments, and there are known performance implications with FIPS and OpenSSL 3.x, these effects can be amplified on AMD Zen4/Zen5 (Ryzen) consumer processors due to their slower entropy generation, resulting in higher than expected CPU utilization.

Fortunately, both individuals had identified workarounds for the issue, but their recommendations did not completely overlap. Before sharing the guidance more broadly, I wanted to spend some time validating the solution to identify the minimal set of changes required to consistently achieve the same outcome.

[Read more...]

VCF 9.1 - Quick Tip: Optimized Workflow for Configuring Distributed Transit Gateway (DTGW) with NSX Virtual Network Appliance (VNA)

by // Leave a Comment

With VMware Cloud Foundation (VCF) 9.1, the Distributed Transit Gateway (DTGW) connectivity model now supports NSX stateful network services through the Virtual Network Appliance (VNA) cluster. This makes it easy for VI Admins to provision Virtual Private Clouds (VPCs), which can then be used to deploy vSphere Kubernetes Service (VKS) workloads using either vCenter Server or VCF Automation (VCFA).

One of the most underappreciated benefits is the ability to complete various NSX workflows including an entire VNA cluster deployment and DTGW configuration directly from vCenter Server, without having to switch to the NSX Manager UI. While NSX Manager still provides the underlying micro-frontend interfaces that are integrated into the vSphere UI, I appreciate having the flexibility to perform these workflows from either the vCenter Server or NSX Manager interface, depending on your roles and responsibilities within an organization.

While there are a number of community blog posts covering DTGW and VNA deployments, they typically start in the vSphere UI for the initial VNA deployment before switching to the NSX Manager UI to complete the remaining configuration, which is perfectly valid. Personally, I prefer using the vSphere UI for as many workflows as possible, especially since enabling vSphere Supervisor and vSphere Kubernetes Service (VKS) is also performed through the vSphere UI.

[Read more...]