ESXi Host Performance Tuning for Production Workloads

Introduction

ESXi performance tuning is the art of getting maximum throughput from your physical hardware while maintaining stability. Poor tuning leads to VM latency, CPU ready, and balloon memory — all signs of resource contention. This guide covers systematic performance analysis and tuning.

Performance Monitoring Tools

POWERSHELL

# PowerCLI: Get CPU ready (high value = CPU contention)
Get-Stat -Entity (Get-VM "my-vm") -Stat cpu.ready.summation -MaxSamples 20 | 
  Select-Object Value | Measure-Object -Property Value -Average

# Get memory balloon (high = memory pressure)
Get-Stat -Entity (Get-VM "my-vm") -Stat mem.vmmemctl.average -MaxSamples 5

# Get disk latency
Get-Stat -Entity (Get-VM "my-vm") -Stat disk.totalLatency.average -MaxSamples 5

CPU Tuning

NUMA Awareness

Modern servers have multiple NUMA nodes (CPU socket + local RAM). VMs that span NUMA nodes suffer performance penalties.

POWERSHELL

# Check NUMA topology
Get-VMHost | Get-View | Select-Object -ExpandProperty Hardware | 
  Select-Object -ExpandProperty NumaInfo

# Constrain VM to a NUMA node
$vm = Get-VM "database-server"
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$spec.NumaInfo = New-Object VMware.Vim.VirtualMachineVirtualNumaInfo
$spec.NumaInfo.CoresPerNumaNode = 8  # Match physical NUMA topology
($vm | Get-View).ReconfigVM_Task($spec)

CPU Hot Add vs Cold Add

POWERSHELL

# Enable CPU hot add (allows adding vCPUs without reboot)
$vm = Get-VM "app-server"
$spec = New-Object VMware.Vim.VirtualMachineConfigSpec
$spec.CpuHotAddEnabled = $true
($vm | Get-View).ReconfigVM_Task($spec)

Avoid vCPU Overcommit

Rule of thumb: don't exceed 4:1 vCPU:pCPU ratio for production workloads.

POWERSHELL

# Check vCPU to pCPU ratio per host
Get-VMHost | ForEach-Object {
    $host = $_
    $vCPUs = (Get-VM -Location $host | Measure-Object -Property NumCpu -Sum).Sum
    $pCPUs = $host.NumCpu
    [PSCustomObject]@{
        Host = $host.Name
        pCPUs = $pCPUs
        vCPUs = $vCPUs
        Ratio = [math]::Round($vCPUs / $pCPUs, 2)
    }
}

Memory Tuning

Transparent Page Sharing (TPS)

TPS deduplicates identical memory pages across VMs. In security-hardened environments, TPS between VMs from different security domains should be disabled.

BASH

# On ESXi host
esxcli system settings advanced set -o /Mem/ShareScanGHz -i 0  # Disable TPS
esxcli system settings advanced set -o /Mem/ShareForceSalting -i 2  # Salt pages per VM

Memory Overcommit Mechanisms

ESXi has 4 memory reclamation techniques (in order of preference):

TPS: Share identical pages
Balloon driver: Guest OS voluntarily gives up pages
Swap: Write VM pages to disk (avoid!)
Compression: Compress pages in memory (better than swap)

POWERSHELL

# Check memory reclamation on VMs
Get-VM | Get-Stat -Stat mem.vmmemctl.average,mem.swapped.average -MaxSamples 5 |
  Where-Object {$_.Value -gt 0} |
  Select-Object Entity, MetricId, Value

Storage Performance

Storage I/O Control (SIOC)

SIOC prevents a single VM from monopolizing datastore I/O during contention.

POWERSHELL

# Enable SIOC on datastore
Get-Datastore "production-ds" | Set-Datastore -StorageIOControlEnabled $true

# Set I/O shares per VM
Get-VM "database-server" | Get-HardDisk | ForEach-Object {
    $spec = New-Object VMware.Vim.VirtualDiskConfigSpec
    $spec.Operation = "edit"
    $spec.Device = $_.ExtensionData
    $spec.Device.StorageIOAllocation.Shares.Level = "high"
    $spec.Device.StorageIOAllocation.Shares.Shares = 2000
}

VMware Paravirtual SCSI (PVSCSI)

PVSCSI controller provides higher throughput than LSI Logic for I/O-intensive VMs:

POWERSHELL

# Change VM to PVSCSI (requires shutdown)
$vm = Get-VM "high-io-vm"
$controller = New-Object VMware.Vim.VirtualDeviceConfigSpec
$controller.Operation = "add"
$controller.Device = New-Object VMware.Vim.ParaVirtualSCSIController
$controller.Device.BusNumber = 0
$controller.Device.SharedBus = "noSharing"

Network Performance

VMXNET3 vs E1000

Always use VMXNET3 for production VMs — it has hardware offload for RSS, LRO, and TSO.

POWERSHELL

# Check VM NIC types
Get-VM | Get-NetworkAdapter | Where-Object {$_.Type -ne "Vmxnet3"} |
  Select-Object VM, Name, Type

VMkernel NIC Offloads

BASH

# Enable LRO (Large Receive Offload) on vmkernel
esxcli system settings advanced set -o /Net/TcpipDefLROEnabled -i 1

# Check NIC offload capabilities
esxcli network nic get -n vmnic0 | grep -i offload

Performance Profiling Workflow

Identify problem: Guest OS shows high latency/CPU
Check CPU ready in vCenter Performance charts
Check memory balloon/swap stats
Check disk latency (> 20ms is problematic)
Check network drops/errors
Esxtop for real-time analysis:

BASH

# SSH to ESXi, run esxtop
esxtop
# Press: c=CPU, m=memory, d=disk, n=network
# Look for: %RDY (CPU ready), MCTL (balloon), KAVG (kernel latency)