Provisioning and managing virtual databases

After you have added a data source from an environment and created snapshots of a dSource, you can begin to provision virtual databases (VDBs), one of the key parts of using Delphix.

As discussed before, a dSource is a virtualized representation of a physical or logical source database, which cannot be accessed or manipulated using database tools. In order to use dSource snapshots, you must create a VDB, an independent, writable copy of a dSource snapshot.

You can create VDBs from other VDBs. Once a VDB has been provisioned to a target environment, you can also create a snapshot policy for that VDB, to capture changes within it as if it were any other logical or physical database.

The following operations and concepts exist for VDBs:

Provisioning virtual databases

Virtual databases are a key data management concept for Delphix. In order to create or provision a virtual database, you will need a linked dSource from a source host and a compatible target environment, as described in the overview for Managing environments and hosts. Since provisioning VDBs varies significantly across data platforms, details on this process can be found within the documentation for a specific data platform.

At their core, virtual databases are fully functional copies of data sources that can be created and managed at a fraction of the storage and time that are typically required. To end-users, VDBs act and perform just like a standard database.

The VDB timeflow

Both dSources and VDBs have timeflows, which are a collection of snapshots for a particular data set. Virtual databases can be provisioned from any snapshots in a Timeflow. The Virtual database's filesystem will be dependent upon the snapshot used to Provision, and as such, that snapshot can not be removed until the dependency is removed. In the product UI, a Timeflow is represented vertically with a series of individual snapshots as rows.

The VDB snapshot

Snapshots of dSources preserve specific points in time for use later during the virtual database provisioning workflow. Taking a snapshot will create a new snapshot entry in the VDBs Timeflow.

Refreshing virtual databases

Refreshing a VDB will re-provision it from the dSource. As with the normal provisioning process, you can choose to refresh the VDB from a snapshot or a specific point in time. However, you should be aware that refreshing a VDB will delete any changes that have been made to it over time. When you refresh a VDB, it will revert to its past state in the specified snapshot or point in time.

When performing a VDB refresh, there is an option to choose between either the faster or more accurate point in time for that database.

Although the VDB no longer contains the previous contents, the previous Snapshots and TimeFlow still remain in Delphix and are accessible through the Command Line Interface (CLI).

Prerequisites

To refresh a VDB, you must have the following permissions:

PROVISIONER permissions on the dSource associated with the VDB
PROVISIONER permissions on the group that contains the VDB
Owner permissions on the VDB itself
Data is a role that allows DB_ROLLBACK, DB_REFRESH, READ_ACTION, DB_SYNC, JOB_CANCEL.
Read is a role that allows the user to inspect objects via the READ_ACTION permission.

A user with admin credentials can perform a VDB Refresh on any VDB in the system.

Procedure

Perform the following steps to refresh a VDB.

Login to the Delphix Management application.
Navigate to Manage > Datasets.
Select the VDB that you want to refresh.
Click the Refresh VDB button. The Refresh VDB screen appears.
Select one of the refresh options and click Next.
1. Faster - This option will utilize the most recent snapshot in the timeflow.
2. More accurate - This option allows you to select a snapshot, a point in time, or a specific log ID.
Click Submit to confirm.
Click the Actions button to watch the progress of the refresh job.
If you want to know when the VDB was last refreshed/provisioned, check the Time Point on the Status page.

Rewinding virtual databases

Rewinding a VDB rolls it back to a previous point in its Timeflow and re-provisions the VDB. The VDB will no longer contain changes after the rewind point.

Rewind VDB

Although the VDB no longer contains changes after the rewind point, the rolled-over Snapshots and Timeflow still remain in Delphix and are accessible through the Command Line Interface (CLI). See the topic CLI Cookbook: Rolling forward a VDB for instructions on how to use these snapshots to refresh a VDB to one of its later states after it has been rewound.

Delphix clones a new Timeflow from the closest Snapshot older than or equal to the rewind point. This creates a dependency between the new Timeflow and the parent Snapshot and Timeflow. The parent Snapshot and Timeflow cannot be deleted because of this dependency. The VDB must first be refreshed before the parent Snapshot and Timeflow can be removed.

Prerequisites

To rewind a VDB, you must have the following permissions:

Auditor permissions on the dSource associated with the VDB
Owner permissions on the VDB itself

You do NOT need owner permissions for the group that contains the VDB. A user with admin credentials can perform a VDB Rewind on any VDB in the system.

Procedure

Login to the Delphix Management application.
Select the VDB you want to rewind.
Select the Timeflow tab.
Select the rewind point as a snapshot or a point in time.
Click Rewind.
If you want to use login credentials on the target environment other than those associated with the environment user, click Provide Privileged Credentials.
Click Rewind to confirm.

Using Timeflow bookmarks

You can use TimeFlow bookmarks as the rewind point when using the CLI. Bookmarks can be useful to:

Mark where to rewind to - before starting a batch job on a VDB for example.
Provide a semantic point to revert back to in case the chosen rewind point turns out to be incorrect.

For a CLI example using a TimeFlow bookmark, see CLI Cookbook: Provisioning a VDB from a Timeflow Bookmark

Undoing a rewind or refresh

An accidental refresh of one or more virtual databases (VDBs) from a source database has removed important data. To correct this situation users can undo VDB refresh or rewind actions.

Procedure

To undo a refresh or rewind via the Delphix Management application:

Login to the Delphix Management application.
Under Datasets, select the VDB you want to undo your rewind/refresh for.
From the Actions menu (...) select Undo Refresh or Rewind.
Confirm that you want to undo the last refresh or rewind for the selected VDB, by selecting Undo.
From the Action sidebar menu, you will be able to view the undo action.

Starting and stopping virtual databases

Stopping your virtual databases is necessary if you are performing any actions on the database that require it to be offline. In addition to properly stopping the database, the Delphix stop and start actions also unmount and mount the filesystems that are provided by the Delphix Engine as well as performing any Delphix-specific actions during startup or shutdown.

Procedure

Login to the Delphix Management application.
Select the VDB you want to manage.
To stop an active VDB, click the stop icon in the top right-hand corner.
To start an inactive VDB, click the start icon in the top right-hand corner.
If stopping or starting the VDB requires particular credentials for the target environment other than those of the default environment user:
1. Check Provide Privileged Credentials.
2. Enter the username and password.
3. Click Validate Credentials.
Click Yes to stop or start the VDB.
You can view the status of the stopping or starting of the VDB in several ways:
View the Active Jobs on the right of the screen or monitor the progress bar from the VDB Status tab.

Enabling and disabling virtual databases

Disabling a VDB is a prerequisite for procedures such as VDB migration or upgrade. Disabling a VDB removes all traces of it, including any configuration files, from the target environment to which it was provisioned. However, the VDB itself, as well as the metadata, will still exist on the engine.

When you later enable the VDB again, these configuration files are restored on the target environment.

Procedure

Login to the Delphix Management application.
Select the VDB you want to disable.
From the Actions menu select Disable.
Click Disable to acknowledge the warning.

When you are ready to enable the VDB again, from the Actions menu select Enable, and the VDB will continue to function as it did previously.

Deleting virtual databases

Deleting a VDB will remove it and it's Timeflow and snapshots from the Delphix Engine entirely. This operation is non-reversible.

Prerequisites

You cannot delete a VDB that has dependent VDBs. Before deleting a VDBs, make sure that you have deleted all dependent VDBs. Otherwise, the delete option will be grayed out.

Deleting a VDB is an unrecoverable operation. Proceed only if you want to permanently destroy the unique data that was created in the VDB.

Procedure

Login to the Delphix Management application.
Select the VDB you want to delete.
From the Actions menu select Delete.
Click Delete to confirm.

Force Delete

Deleting a vFiles may fail if it cannot be unmounted successfully from all target environments. You can use the Force Delete option to ignore all failures during unmount.

Using Force Delete

Deleting unused or outdated objects should be a regular part of Delphix Engine administration. This is especially important to prevent low space errors, which can cause the Delphix Engine to stop. The Delphix Engine holds a maximum of 400 objects.

Force Delete can be used when:

The staging host/container is not accessible via Delphix Engine anymore.
The Delphix Connector service is not running (applicable to SQL Server only).
The normal delete is throwing error and not working.
When the target or staging host is decommissioned.

Force Delete should not be used as an alternative to Delete in normal circumstances.

Procedure

Log into the Delphix Management application.
Select Resources > Storage Capacity.
Next to the object you want to delete select the Trash can.
In the Delete dialog select Force Delete. Oracle users will have the option to provide additional credentials.
Click Delete.

Dependencies

If there are dependencies on the snapshot, you will not be able to delete the snapshot free space; the dependencies rely on the data associated with the snapshot. These items are displayed with a lock icon next to the name.

Migrating virtual databases

There may be situations in which you want to migrate a virtual database to a new target environment, for example when upgrading the host on which the VDB resides, or as part of a general data center migration. Using the VDB migrate functionality allows you to change the VDB’s location from one target environment to another.

This is easily accomplished by first disabling the database, then using the Migrate VDB feature to select a new target environment.

Prerequisites

You should have already set up a new target environment that is compatible with the VDB that you want to migrate.
Ensure that the VDB has first been disabled

Procedure

Login to the Delphix Management application.
Select the VDB you want to migrate.
From the Actions menu select Migrate.
Select the new target environment for the VDB, the user for that environment, and the database installation where the VDB will reside.
Select Migrate to confirm your selections.
From the Actions menu select Enable.
Click Enable to confirm.

Within a few minutes, your VDB will re-start in the new environment, and you can continue to work with it as you would any other VDB.

Upgrading virtual databases

Similar to upgrading a data source, upgrading a VDB changes the installation path that is associated with that data platform. While this feature does not upgrade the actual VDB, it is important that Delphix associates this VDB with the correct database version to function properly.

By performing this upgrade, you can update the version of the database installation used for running the VDB.

Procedure

Login to the Delphix Management application.
Select the VDB you want to upgrade.
From the Actions menu select Upgrade.

Virtual to physical (V2P)

Virtual to Physical, or V2P, is the process of exporting a virtual database to a physical one.

After you have created a dSource or a VDB, you can export its contents and log files to a physical database. This process creates a set of directories on the target environment and populates them with the database data, log files, and scripts that are used to recover the physical database.

You can automatically start the physical database recovery process as part of V2P, or you can use the scripts for manual recovery. When the export process completes, the target environment will contain a copy of the database in its unvirtualized size, so before you begin the process, make sure the target directories you specify during V2P have enough capacity to hold the database. For more information, visit the following page: Virtual to physical

Procedure

Login to the Delphix Management application.
Select the dSource or VDB you want to export.
Select the snapshot of the dSource or VDB state you want to export.
From the Actions menu select Virtual to Physical.
Select the target environment, and fill the other input fields depending on the data platform of your object.

Provisioning with replication

This topic describes how to provision from a replicated dSource or virtual database (VDB).

The process for provisioning from replicated objects is the same as the typical VDB provisioning process, except that first you need to select the replica namespace containing the replicated object.

Prerequisites

You must have a dSource or VDB that has been replicated from one Delphix Engine to another, as described in Replication overview
The Delphix Engine containing the replicated dSource or VDB must have a compatible target environment that it can use to provision a VDB from the replicated dSource or VDB.
On the Delphix Engine containing the replicated dSource or VDB, login to the Delphix Management application.

Procedure

From the list of replica namespaces, select the replica namespace that contains the dSource or VDB from which you want to provision.
The provisioning process is now identical to the process for provisioning standard objects.