Replication overview

Delphix facilitates the replication of data objects across different Delphix Engines. The Delphix Engines can be running different Delphix versions, but with some limitations to older versions, as mentioned below.

Replication features

• Data recovery and distribution: If the source engine fails, the target engine can be activated to mirror the state of the source. This feature also enables geographical data distribution and remote VDB provisioning from replicated objects.

• Replication scheduling: While ad hoc replication is possible, it is commonly executed on a preset schedule. Only the changes made since the last update are transmitted in subsequent updates.

• Versatile data management: Delphix Engines (functioning as virtual appliances) offer robust data management capabilities that include backing up, restoring, replicating, and migrating various data objects such as groups, dSources, VDBs, Jet Stream data templates and containers, along with their dependencies.

• Enhanced capabilities: The Continuous Data Engine's native replication adds advanced options to include selective replication of data objects, consolidating multiple sources to a single target, and the ability to provision VDBs from replicated dSources and VDBs without interrupting updates.

• Flexible configuration: Replication is set up on the source engine, enabling the transfer of selected dSources and VDBs to a target engine. This process supports both manual and scheduled incremental updates. Detailed guidance for setting up replication can be found in the Configuring replication page.

• Provisioning from replicated data: The replicated dSources and VDBs on the target engine can be used to create new VDBs. These can be refreshed with data from incremental replication updates, provided the original source objects remain intact. For more insights, refer to the Provisioning from replicated dSources or VDBs page.

• Disaster recovery and activation: During replication, the replicated objects are stored in an inactive state on the target engine. In case of a disaster, a failover operation can be initiated to break the replication link and activate these objects. More information on this process is available under Replicas and failover.

Limitations

• No synchronous semantics: Replication in Delphix does not offer synchronous semantics. This means there is a potential loss of data on the target engine, equivalent to the changes made since the last replication update, in the event of a failover.

• Not ideal for rapid failover and failback: In high-availability scenarios, it is advisable to utilize the capabilities of the underlying hypervisor or storage platform.

• Older engine version compatibility: Engine version 5.3.3 or above is required to allow for replication between engines running different Delphix versions, as elaborated in the Forward Compatible Replication (FCR) section.

Further guidance

For a detailed understanding of how Delphix Engine replication aligns with your data recovery needs, refer to the Backup and recovery strategies for the Delphix Engine.

Replication details

When you select objects for replication, the engine will automatically include any dependencies, including parent objects, such as groups, and data dependencies such as VDB sources. This means that replicating a VDB will automatically include its group, the parent dSource, and the group of the dSource, as well as any environments associated with those databases. When replicating an entire engine, all environments will be included. When replicating a database or group, only those environments with the replicated databases are included.

Only database objects and their dependencies, as well as certain non-database objects, are copied as part of a backup or replication operation, including:

• dSources

• VDBs

• Groups

• Self-service (Jet Stream) Data Templates and Data Containers

• Environments

• Environment configuration (users, database instances, and installations)

• Delphix users, roles, permissions, and authorizations

• Policies

• Database configuration templates

The following objects are NOT copied as part of a backup or replication operation:

• Events and faults

• Job history

• System services settings, such as SMTP

• Hook templates

• Alert profiles

After failover, you must recreate these settings on the target.

On-premises Replication to Azure/OCI/GCP/Hyper-V

Replicating from on-premises engines with an underlying storage block size of 512B will experience disk usage inflation when replicating to target engines with different underlying block sizes.

Azure, GCP, Hyper-V, and OCI are known to have 4K block sizes and therefore will require extra disk capacity when receiving replication from an on-premises engine. This behavior is due to the fact that the underlying storage block size is different (512B vs. 4K) between the two Delphix Engines (one on-prem, one on Azure/OCI/GCP/Hyper-V), resulting in a lower compression rate on the replication target.

It is expected that 1.5-1.6x the amount of space is taken from objects on-premises in these cases.

Resumable replication

This feature allows for the continuation of large and time-intensive initial replications or incremental updates from a midpoint, rather than starting anew. This is crucial given that replications can fail due to various environmental or internal factors.

Real-world scenario

Imagine a scenario where a replication is set up between a source and a target, projected to take several weeks. If a power outage at the source's data center interrupts this process, the source machine, upon rebooting, will automatically reconnect to the target and resume replication from where it was interrupted. In the user interface (UI), the source will continue to show the ongoing replication job, while the target will display a new replication reception job, tracking the overall progress.

Applicability of resumable replication

Resumable replication is effectively utilized during source reboots, target reboots, and network partitions. It ensures the continuity and efficiency of the replication process in the face of unforeseen disruptions.

Replicating Delphix Self-Service templates

Templates can be replicated and accessed on the target engine via Delphix Self-Service (Jet Stream). Replicated templates can be replicated into the target space with or without their containers. On the new target engine, the newly created replicated template can be used to create new containers that are assigned to users. You cannot change the replicated template’s name or the names of the containers from which it was replicated.

Replication of non-data objects

As of 6.0.5.0, the replication of non-data objects is supported.

Non-data objects refer to:

• Delphix users, roles, permissions, and authorizations

• Policies

• Database configuration templates

These objects will not be presented as selectable objects when creating a replication spec, but will instead be passively included by association, the same way environments are. Replication of non-data objects will follow these rules:

• If the entire engine is replicated, all non-data objects will be included

• If specific data objects are replicated, all associated non-data objects will be included

For example, when a container is replicated, then policies that apply to that container, as well as users who have authorizations on that container, will all be included for replication.

Replicated users will be shown on the Received Replicas page on the Delphix Engine UI.

Automatic conflict resolution

Also as of 6.0.5.0, the automatic conflict resolution option is chosen by default; it is the recommended way of handling the failover of non-data objects, as non-data objects are expected to cause collisions. Starting with version 6.0.17.0, automatic conflict resolution is always enabled and is no longer optional.

Automatic conflict resolution will resolve non-data object conflicts by the following rules:

• Users will be consolidated if both username and email match. Otherwise, the replicated user will be renamed.

• Roles, permissions, and authorizations will be consolidated

• Policies and database configuration templates will be renamed, except for "None" type policies, which will be consolidated. Following failover, replicated policies will continue to apply to the same replicated data objects.