This article outlines the requirements for deploying the Delphix Engine on VMware (including supported versions and instance configurations), as well as recommended configuration parameters for optimal performance.
The Delphix Engine is an intensive platform, both from a network and a storage perspective. If the Delphix Engine competes with other virtual machines on the same host for resources it will result in increased latency for all operations. As such, it is crucial that the ESXi host is not over-subscribed, as this eliminates the possibility of a lack of resources for the Delphix Engine. This includes allowing a percentage of CPU resources for the hypervisor itself as it can de-schedule an entire VM if the hypervisor is needed for managing IO or compute resources.
Supported ESX versions
Virtual machine hardware versions
The Delphix Engine VM is distributed as an OVA for VMware, and is configured with the hardware version corresponding to the lowest ESXi version that release is qualified for. As the Engine is upgraded, the ESXi versions supported may change, but the hardware version may remain the same based on the original deployment.
Users who wish to upgrade the VM hardware version for compatibility, enhanced feature support, or other reasons may feel free to do so, with consideration for any compatibility concerns that may arise in environments where multiple ESXi versions are present, as an upgraded hardware version can affect other VMware operations (vMotion, etc). VMware support and documentation should be consulted before committing any hardware version upgrade for a guest VM, but Delphix does not maintain any version requirements in this regard.
Never allocate all available physical CPUs to virtual machines
128 GB vRAM (recommended)
64GB vRAM (minimum)
Never allocate all the available physical memory to virtual machines.
Memory for the ESX Server to perform hypervisor activities must be set aside before assigning memory to Delphix and other VMs.
Failure to ensure sufficient memory for the host can result in a hard memory state for all VMs on the host which will result in a block for memory allocations.
The ova is pre-configured to use one virtual ethernet adapter of type VMXNET 3.
A 10GbE NIC in the ESX Server is recommended.
If the network load in the ESX Server hosting the Delphix engine VM is high, dedicate one or more physical NICs to the Delphix Engine.
To bootstrap a networking configuration to reach the Delphix Engine, after deploying it into your environment for the first time, you can use one of the following options:
Cloud-init for public clouds
You can login to the serial console to configure networking via the CLI
OVF guest customizations to pass in network configuration to the VM before it has a network connection. For the customization steps:
Type the name of the profile and click Next
The domain will be ignored by Delphix. Click Next
The time zone setting will be ignored by Delphix, Click Next
Delphix doesn’t allow scripts, click Next
Manually select custom settings, select a NIC and click Edit
Provide Netmask and Gateway
Select one of the options for IP
Confirm changes and click Next
Confirm creation of profile, click Finish
You can use this template to clone a VM
PVSCSI (default)/ LSI Logic Parallel
When adding virtual disks make sure that they are evenly distributing the load across the maximum of 4 virtual SCSI controllers. Spreading the disks across available SCSI controllers evenly will ensure optimal IO performance from the disks. For example, a VM with 4 SCSI controllers and 8 virtual disks should distribute the disks across the controllers as follows:
disk0 = SCSI(0:0) - System Disk on Controller 0 Port 0
(ignore for purposes of load balancing)
disk1 = SCSI(0:1) - Data Disk on Controller 0 Port 1
disk2 = SCSI(1:1) - Data Disk on Controller 1 Port 1
disk3 = SCSI(2:1) - Data Disk on Controller 2 Port 1
disk4 = SCSI(3:1) - Data Disk on Controller 3 Port 1
disk1 = SCSI(0:2) - Data Disk on Controller 0 Port 2
disk2 = SCSI(1:2) - Data Disk on Controller 1 Port 2
disk3 = SCSI(2:2) - Data Disk on Controller 2 Port 2
disk4 = SCSI(3:2) - Data Disk on Controller 3 Port 2
For load purposes, we generally focus on the DB storage and ignore the controller placement of the system disk.
VMware offers options for disk storage, which include "Independent - Persistent" and "Independent - Non-persistent". The non-persistent setting is catastrophic to the Delphix platform when the VM reboots, thus, Independent - Persistent is required for installation.
Storage used for Delphix must be provisioned from storage that provides data protection.
For example, using RAID levels with data protection features, or equivalent technology.
The Delphix engine product does not protect against data loss originating at the hypervisor or SAN layers.
For more information refer to, Optimal Storage Configuration Parameters for the Delphix Engine.
Delphix storage options
There are three types of data that Delphix stores on disk, which is:
Delphix VM Configuration Storage: stores data related to the configuration of the Delphix VM. VM Configuration Storage includes the VMware ESX configuration data as well as log files.
Delphix Engine System Disk Storage: stores data related to the Delphix Engine system data, such as the Delphix .ova settings.
Database Storage: stores data used by Delphix objects such as dSources and virtual databases (VDBs).
Delphix VM configuration storage
The Delphix VM configuration should be stored in a VMFS volume (often called a "datastore").
The VMFS volume should have enough available space to hold all ESX configuration and log files associated with the Delphix Engine.
If a memory reservation is not enabled for the Delphix Engine (in violation of memory requirements stated above), then space for a paging area equal to the Delphix Engine's VM memory must be added to the VMFS volume containing the Delphix VM configuration data.
Delphix engine system disk storage
The VMFS volume must be located on shared storage in order to use vMotion and HA features.
The Delphix Engine system disk should be stored in a VMDK.
The Delphix .ova file is configured for a 127GB system drive.
Option 1: Block Storage for database storage
Shared storage is required in order to use vMotion and HA features. In addition to making sure the latest VMware patches have been applied, check with the organizations hardware vendor for updates specific to the hardware configurations. VMDKs (Virtual Machine Disks) or RDMs (Raw Device Mappings) operating in virtual compatibility mode can be used for database storage.
A minimum of 4 VMDKs or RDMs should be allocated for database storage.
If using VMDKs:
The quantity and size of VDMKs or RDMs assigned must be identical across all 4 controllers
The physical LUNs used for VMFS volumes and RDMs should be of the same type in terms of performance characteristics such as latency, RPMs, and RAID level.
The physical LUNs used for VMFS volumes can be thin-provisioned in the storage array.
For best performance, the LUNs used for RDMs should not be thin-provisioned in the storage array but should be thick-provisioned with a size equal to the amount of storage that will be initially allocated to the Delphix Engine. The RDM can be expanded in the future when more storage is needed.
In addition to making sure the latest VMware patches have been applied, check with the organization's hardware vendor for updates specific to the hardware configurations.
Option 2: Elastic Data where Object storage is used for database storage and block storage is used for cache.
We support Elastic Data with on prem object storage for the database storage. Traditional disks as cache are used to reduce latencies for frequently read data and as temporary storage for synchronous writes before the writes are sent to object storage.
For on prem object storage we currently support storage vendors that confirm to the following
s3 REST API compatibility
strong read-after-write consistency
supports s3 key id and secret access key authentication
perpetual key support
On-premise object storage may require a security certificate to create secure connections between the Delphix engine and object storage. If a certificate is required, then it must be installed on the Delphix engine prior to configuring the storage. Refer to Certificate management for more information.
Refer to Initial setup for additional details for setting up the Elastic Data Engine.
Additional VMware configuration notes
Running Delphix inside of vSphere is supported.
Using vMotion on a Delphix VM is supported.
Device passthrough is not supported.
Procedure to install an OVA
Use the Delphix-supplied OVA file to install the Delphix Engine. The OVA file is configured with many of the minimum system requirements. The underlying storage for the install is assumed to be redundant SAN storage.
Download the OVA file from https://download.delphix.com. You will need a support login from your sales team or a welcome letter.
Navigate to the Delphix Product Releases/<Current Version>/Appliance Images page.
Login using the vSphere client to the vSphere server (or vCenter Server) where you want to install the Delphix Engine.
In the vSphere Client, click File.
Select Deploy OVA Template.
Browse to the OVA file.
Select a hostname for the Delphix Engine.
This hostname will also be used in configuring the Delphix Engine network.
Select the data center where the Delphix Engine will be located.
Select the cluster and the ESX host.
Select one (1) data store for the Delphix OS. This datastore can be thin-provisioned and must have enough free space to accommodate the 127GB comprising the Delphix operating system.
For traditional block storage engines
Select four (4) or more data stores for Database Storage for the Delphix Engine. The Delphix Engine will stripe all of the Database Storage across these VMDKs, so for optimal I/O performance, each VMDK must be equal in size and be configured Thick Provisioned - Eager Zeroed. Additionally, these VMDKs should be distributed as evenly as possible across all four SCSI I/O controllers.
For Elastic data
The VMDK will be used as cache to reduce latencies for frequently read data and as temporary storage for synchronous writes before the writes are sent to object storage. For optimal I/O performance, each VMDK must be equal in size and be configured Thick Provisioned - Eager Zeroed. Make sure the disks satisfy the I/O needs of the engine. Eg: At 500 IOPS per 1GiB ratio, a 32 GiB volume can be configured to have the 16K IOPS limit. Two devices would be sufficient for the instance that requires 30K IOPS. You can always add disks later if the cache is insufficient. You can only reduce the cache by removing disks, so if you think you are over provisioning the cache, increase the number of disks used and reduce the size of each disk so that removal is possible at a later point in time.
Select the virtual network you want to use.
If using multiple physical NICs for link aggregation, you must use vSphere NIC teaming. Do not add multiple virtual NICs to the Delphix Engine itself. The Delphix Engine should use a single virtual network. For more information, see Optimal network architecture for the Delphix engine.
The installation will begin and the Delphix Engine will be created in the location you specified.
Once the installation has completed, power on the Delphix Engine and proceed with the initial system configuration as described in Setting up network access to the Delphix engine