Linking a dSource from a SQL server: An overview

When linking a dSource from a SQL Server source database, Delphix offers several different methods of capturing backup information:

Delphix Managed Backups, where the Delphix Engine schedules and initiates the backups from the source database, and captures them
SQL Server Managed Backups, where the SQL Server source database schedules and initiates backups and the Delphix Engine captures them
- Full backups
- Full or differential backups
- Transaction log backups (with LogSync disabled)
- Transaction log backups (with LogSync enabled)

Delphix Managed Backups are conceptually a lot simpler to explain, but they can be considered less desirable because they might be viewed as intrusive. SQL Server Managed Backups are explained in two sections:

the initial load of the dSource from the source database
subsequently keeping the dSource synchronized with the source database.

Source databases cannot be in Read-Only mode at the time that a backup is taken. While the snapshot will succeed, attempts to provision VDBs from those snapshots may fail with an error during the provision process.

Temporarily setting the database to read-write mode, and taking a new backup and snapshot while the database is in in this state, will allow VDBs to be provisioned successfully.

Below is a brief explanation of how these three different modes of operation work.

Delphix managed backup mode

When the Enabled checkbox for Delphix Managed Backups is selected, the Delphix Engine will initiate a COPY_ONLY full backup of the source database for the initial load of the dSource. Thereafter, the Delphix Engine will initiate COPY_ONLY full backups of the source database using the schedule specified by the selected SnapSync Policy. If SnapSync Policy is set to None, the Delphix Engine will not automatically initiate COPY_ONLY backups, but you can initiate them manually using the snapshot (camera) icon.

When the Force Compression checkbox for Delphix Managed Backups is selected and the backups are not compressed, the Delphix Engine will take a compressed copy-only full backup of the source database and this will take effect during the next scheduled Snapshot or SnapSync operation.

COPY_ONLY backup files are written to the Delphix storage that has been mounted on the staging server. Delphix does not require space on the source or staging servers to hold the backup files.

You must also select the Staging Environment and the SQL Server instance onto which the backups will be restored.

SQL server managed backup modes

Initial load of the dSource

When the Delphix Managed Backups option is left unchecked (default), the Delphix Engine will initiate a backup only when the user selects to initiate a COPY_ONLY full backup that the Delphix Engine will use to keep the dSource in sync with the source database.

For the initial load of the dSource, you can choose one of the following:

have the Delphix Engine initiate a COPY_ONLY full backup
use the most-recent existing full or differential backup (default)
use a specific existing full or differential backup identified by its backup_set_uuid

Simple Recovery Model

If the source database is using a simple recovery model, using a new full COPY_ONLY backup initiated by the Delphix Engine is not supported for the initial load of a dSource.

After the initial load, you need to select the Backup Paths and tell the Delphix Engine where to look for backups of the source database.

If Autodiscover is selected, the exact path used to take the backup from the source database will be determined by querying by the source database instance. If this option is used, the Delphix Engine should take backups to a UNC path (Windows file share) so that they are accessible to the Staging Server.
If custom paths are specified, the Delphix Engine will query the source database instance, identify the filename of the source backup, and then recursively search the specified Backup Paths for this filename. These paths should also be UNC paths (Windows file share) which are accessible to the Staging Server.

The path used to restore backups must be readable by the Windows server hosting the staging instance, using the staging environment's configured Environment User.

The Delphix Engine supports source database backups that SQL Server creates natively, as well as backups created by Quest/Netvault LiteSpeed, Red Gate SQL Backup Pro, Veritas NetBackup, and Commvault. For more information, see the topic SQL Server Support Matrix.

Once you have decided how the dSource will be initially loaded, select the staging environment and the SQL Server instance onto which the backups will be restored. NetBackup and Commvault backups are not on local storage and therefore the Backup Paths will just be ignored. See Linking a dSource from a NetBackup SQL Server Backup and Linking a dSource from a Commvault SQL Server Backup for more information.

The staging instance opens the backup file for reading and may hold a lock on it when restoring the backup on the staging database. A new source database backup, initiated with the Append to the existing backup set option, may fail as SQL Server will not be able to open the locked backup file to append a new backup to it.

Keeping the dSource synchronized with the source database

Next, specify how the Delphix Engine will capture subsequent backups of the source database.

The selected Validated Sync mode determines how often the Delphix Engine will check for new backups, and which type of backups it will check for. You can always force synchronization with the source database by enabling Validated Sync Mode from the Data Management tab available under the Configuration tab for the selected dSource.

Validated sync and logSync

When you link a source database into the Delphix Engine, a staging database will still be required if the Validated Sync is not enabled, as described in Overview of Setting Up SQL Server Environments. In this process, the Delphix Engine continuously monitors the source database for new full and differential backups if the source database is using a simple recovery model, or transaction log backups if using a full recovery model. This will also depend on the selected backup mode. When it detects a new backup, it restores that backup to the staging database with the storage residing in Delphix. The result is a TimeFlow with consistent points from which you can provision a VDB, also known as snapshots.

Snapshots accumulate over time. To view a snapshot:

From the Datasets panel, click the group containing the dSource.
Select dSource.
Click the TimeFlow tab.

Each snapshot is displayed and includes some information about the captured database along with Snapshot database change number (SCN for Oracle and LSN for SQL Server). You can scroll through these cards to select the one you want, or you can enter a date and time to search for a specific snapshot.

Summary of validated sync modes

This table summarizes each mode of Validated Sync, displaying how often the Delphix Engine will poll to check for new backup files when it creates snapshots for the dSource, and whether point-in-time restores for provisioning and refreshing virtual databases (VDBs) is possible or not.

Validated Sync Mode	Polling Interval	Snapshot for each FULL backup	Snapshot for each DIFF backup	Snapshot for each TLOG backup	Allows Point-in-time Restores	Notes
Delphix Managed Backups	N/A	N	N	N	N	Takes COPY_ONLY full backups according to SnapSync schedule. For more information on this option, see Delphix as a Backup Solution to SQL Server.
Transaction log backups (LogSync DISABLED)	1-minute	N	N	Y	N	Log backups are not collected if: There are gaps in the sequence of log backups (a break in the "log chain") The available log backups do not include any changes since the last successful Delphix snapshot
Transaction log backups (LogSync ENABLED)	1-minute	N	N	Y	Y	Log files consume additional space on the Delphix Engine and are managed according to the defined retention policy for logs. For NetBackup and Commvault backups, Point-in-time restores are not supported.
Full or differential backups	1-minute	Y	Y	N	N
Full backups	1-minute	Y	N	N	N
None	Manual only	N	N	N	N	Only retrieves backups when you initiate a manual snapshot.

Timeflow cards

The Delphix Engine will create a Timeflow card for each backup it restores to the staging server. For example:

A database in Full backups Validated Sync mode, and daily backup configured on the dSource, would receive one Timeflow card per day
A database in Full or differential backups sync mode, with one daily backup and two differential backups per day, would receive three Timeflow cards per day
A database in Transaction log backups sync mode, with a log backup every 15 minutes, would receive 96 TimeFlow cards per day