Deployment for VMware

• VMware Cloud

• VMware ESXi 8.0

• VMware ESXi 7.0, 7.0 U1, 7.0 U2, 7.0 U3

• VMware ESXi 6.5 and 6.7

• If a minor release version is listed as supported, then all patch releases applicable to that minor release are supported.

• Public cloud deployments such as VMware cloud on AWS and Azure VMware solution are supported.

• While ESXi 7.0 U3 is supported, this is only for patch releases 3c and newer, due to critical issues in the earlier patch versions.

8 vCPUs

• CPU resource shortfalls can occur both on an over-committed host as well as competition for host resources during high IO utilization.

• CPU reservations are strongly recommended for the Delphix VM so that Delphix is guaranteed the full complement of vCPUs even when resources are overcommitted.

• It is suggested to use a single core per socket unless there are specific requirements for other VMs on the same ESXi host.

Never allocate all available physical CPUs to virtual machines

• CPU for the ESXi Server to perform hypervisor activities must be set aside before assigning vCPUs to Delphix and other VMs. 

• We recommend that a minimum of 8-10% of the CPUs available are reserved for hypervisor operation. (e.g. 12 vCPUs on a 128 vCore system).

128 GB vRAM (recommended)

64GB vRAM (minimum)

• The Delphix Engine uses its memory to cache database blocks. More memory will provide better read performance.

• Memory reservations are required for the Delphix VM. The performance of the Delphix Engine will be significantly impacted by the over-commitment of memory resources in the ESX Server. 

• Reservations ensure that the Delphix Engine will not be forced to swap pages during times of memory pressure on the host. A swapped page will require orders of magnitude more time to be brought back to physical memory from the ESXi swap device.

Never allocate all the available physical memory to virtual machines. 

• The default ESX minimum free memory requirement is 6% of the total RAM. When free memory falls below 6%, ESX starts swapping out the Delphix guest OS. Delphix recommends leaving about 8-10% free to avoid swapping.

• For example, when running on an ESX Host with 512GB of physical memory, no more than 470GB (92%) should be allocated to the Delphix VM (and all other VMs on that host).

Memory for the ESX Server to perform hypervisor activities must be set aside before assigning memory to Delphix and other VMs. 

Failure to ensure sufficient memory for the host can result in a hard memory state for all VMs on the host which will result in a block for memory allocations.

The ova is pre-configured to use one virtual ethernet adapter of type VMXNET 3. 

• Jumbo frames are highly recommended to reduce CPU utilization, decrease latency and increase network throughput. (typically 10-20% throughput improvement)

• If additional virtual network adapters are desired, they should also be of type VMXNET 3.

A 10GbE NIC in the ESX Server is recommended.

• For VMs having only gigabit networks, it is possible to aggregate several physical 1GbE NICs together to increase network bandwidth (but not necessarily to reduce latency). Refer to the VMware Knowledge Base article NIC Teaming in ESXi and ESX. However, it is not recommended to aggregate NICs in the Delphix Engine VM.

If the network load in the ESX Server hosting the Delphix engine VM is high, dedicate one or more physical NICs to the Delphix Engine.

• Adding NICs only works if VMs are discovered using different interfaces. The NFS/iSCSI mounts will only use the network associated with the discovery.

• See General Network and Connectivity Requirements for information about specific port configurations, and Network Performance Configuration Options for information about network performance tuning

PVSCSI (default)/ LSI Logic Parallel

When adding virtual disks make sure that they are evenly distributing the load across the maximum of 4 virtual SCSI controllers. Spreading the disks across available SCSI controllers evenly will ensure optimal IO performance from the disks. For example, a VM with 4 SCSI controllers and 8 virtual disks should distribute the disks across the controllers as follows:

disk0 = SCSI(0:0) - System Disk on Controller 0 Port 0 

(ignore for purposes of load balancing)

disk1 = SCSI(0:1) - Data Disk on Controller 0 Port 1

disk2 = SCSI(1:1) - Data Disk on Controller 1 Port 1

disk3 = SCSI(2:1) - Data Disk on Controller 2 Port 1

disk4 = SCSI(3:1) - Data Disk on Controller 3 Port 1

disk1 = SCSI(0:2) - Data Disk on Controller 0 Port 2

disk2 = SCSI(1:2) - Data Disk on Controller 1 Port 2

disk3 = SCSI(2:2) - Data Disk on Controller 2 Port 2

disk4 = SCSI(3:2) - Data Disk on Controller 3 Port 2

For load purposes, we generally focus on the DB storage and ignore the controller placement of the system disk.

IDE and SATA controllers are not supported on VMware platforms.

Storage used for Delphix must be provisioned from storage that provides data protection. 

For example, using RAID levels with data protection features, or equivalent technology. 

The Delphix engine product does not protect against data loss originating at the hypervisor or SAN layers.

For more information refer to, Optimal Storage Configuration Parameters for the Delphix Engine.

The VMFS volume should have enough available space to hold all ESX configuration and log files associated with the Delphix Engine.

If a memory reservation is not enabled for the Delphix Engine (in violation of memory requirements stated above), then space for a paging area equal to the Delphix Engine's VM memory must be added to the VMFS volume containing the Delphix VM configuration data.

The Delphix Engine system disk should be stored in a VMDK.

• The VMDK for the Delphix Engine System Disk Storage is often created in the same VMFS volume as the Delphix VM definition. In that case, the datastore must have sufficient space to hold the Delphix VM Configuration, the VDMK for the system disk, and a paging area if a memory reservation was not enabled for the Delphix Engine.

The Delphix .ova file is configured for a 127GB system drive. 

• The VMFS volume where the .ova is deployed should, therefore, have at least 127GB of free space prior to deploying the .ova.

A minimum of 4 VMDKs or RDMs should be allocated for database storage.

• Allocating a minimum of 4 VMDKs or RDMs for database storage enables the Delphix File System (DxFS) to make sure that its file systems are always consistent on disk without additional serialization. This also enables the Delphix Engine to achieve higher I/O rates by queueing more I/O operations to its storage.

If using VMDKs:

• Each VMDK should be the only VMDK in its VMFS volume

• The VMFS volumes should be assigned to dedicated physical LUNs on redundant storage. 

• The VMDKs should be created with the Thick Provision Lazy Zeroed option.

• Provisioning VMDKs from isolated VMFS volumes on dedicated physical LUNs:

  • Reduces contention for the underlying physical LUNs

  • Eliminates contention for locks on the VMFS volumes from other VMs and/or the ESX Server Console

  • Enables higher availability of the Delphix VM by allowing vSphere to vMotion the VM to a different ESX host in the event of a failure of the Delphix ESX host

The quantity and size of VDMKs or RDMs assigned must be identical across all 4 controllers

• If the underlying storage array allocates physical LUNs by carving them from RAID groups, the LUNs should be allocated from different RAID groups. This eliminates contention for the underlying disks in the RAID groups as the Delphix engine distributes IO across its storage devices.

The physical LUNs used for VMFS volumes and RDMs should be of the same type in terms of performance characteristics such as latency, RPMs, and RAID level.

• The total number of disk drives that comprise the set of physical LUNs should be capable of providing the desired aggregate I/O throughput (MB/sec) and IOPS (Input/Output Operations per Second) for all virtual databases that will be hosted by the Delphix Engine.

The physical LUNs used for VMFS volumes can be thin-provisioned in the storage array.

• If the storage array allocates physical LUNs from storage pools comprising dozens of disk drives, the LUNs should be distributed evenly across the available pools.

For best performance, the LUNs used for RDMs should not be thin-provisioned in the storage array but should be thick-provisioned with a size equal to the amount of storage that will be initially allocated to the Delphix Engine. The RDM can be expanded in the future when more storage is needed.

• Using thin-provisioned LUNs in the storage array for VMFS volumes can be useful if adding storage to the Delphix engine in the future is anticipated. In this case, the LUNs should be thin-provisioned with a size larger than the amount of storage that will be initially allocated to the Delphix Engine. When appropriate, to add more storage to the Delphix engine, use vSphere to expand the size of the VMDKs. Be sure to specify that the additional storage is also thick-provisioned and eager-zeroed.