Provisioning a TDE-enabled vPDB

Overview

Provisioning a Virtual Pluggable Database (vPDB) first involves using the GUI or CLI to specify the vPDB parameters (such as the vPDB name and target container) along with the snapshot to provision from. Once the provision job is started with these parameters, the Delphix Engine does the following:

1. Mounts the snapshot files on the target host.

2. Creates and opens (in mount mode) the auxiliary container database on the target host, using the snapshot files. The auxiliary container database will have both the CDB and PDB data files from the dSource.

3. Completes recovery to bring the auxiliary container database into a consistent state.

4. Finalizes the state of the auxiliary database and unplugs the vPDB datafiles.

5. Plugs the vPDB into the target database, and opens it in read-write mode for general use.

If the dSource is TDE-enabled, then Delphix will need to perform additional operations to complete the provision of a TDE-enabled vPDB to a TDE-enabled target container (indicated in red):

1. Mounts the snapshot files on the target host.

2. Creates and opens (in mount mode) the auxiliary container database on the target host, using the snapshot files. The auxiliary container database will have both the CDB and PDB data files from the dSource.

3. Creates a keystore for the auxiliary container database with the necessary keys to apply encrypted archived log files.

4. Completes recovery to bring the auxiliary container database into a consistent state.

5. Rotates the vPDB and auxiliary CDB master encryption keys by generating new keys that are unique to the vPDB / auxiliary CDB and not associated with the source PDB or CDB.

6. Exports only the newly generated keys to an exported keyfile to enable unplug.

7. Finalizes the state of the auxiliary database and unplugs the vPDB datafiles.

8. Imports the keys from the exported keyfile into the target keystore.

9. When provisioning to a vCDB target, converts the auxiliary CDB into the final vCDB and creates the vCDB keystore from the auxiliary CDB keystore.

10. Plugs the vPDB into the target database, and opens it in read-write mode for general use.

If a plugin violation is detected after plugging the vPDB into the target database, the vPDB provision (and enable) operation will throw a critical error during the SnapSync operation, but it will succeed nonetheless. The vPDB will remain in a running state. You must manually resolve the plugin violations outside of Delphix, close and reopen the vPDB to make the changes effective, and then disable/enable the vPDB within Delphix.

If the initial provision of a TDE-enabled vPDB to a vCDB has either failed or been canceled before step 9, which creates the vCDB Keystore, refreshing that vPDB will fail because there is no target keystore to merge into the auxiliary CDB Keystore for use during recovery. In this case, it will be necessary to delete the failed/canceled vPDB and provision again. This does not apply when provisioning to a linked target CDB, which will already have its own keystore configured.

The following diagram illustrates the provisioning steps.

At each stage of the provisioning process, the keys and exported keyfiles are always on user storage. The exported keyfile is located in the artifact directory, while the auxiliary and target keystores are in the auxiliary keystores directory. Both the artifact directory and auxiliary keystores directory are subdirectories of the TDE keystores root directory, which is either user specified, or if not specified defaults to the toolkit root directory. As for non-TDE-enabled vPDBs, the final vPDB (and vCDB, if applicable) is on Delphix storage while the target linked CDB and its archive logs remain on user storage.

Provisioning a TDE-enabled vPDB

To initiate the provision, Delphix needs the following pieces of information, all of which can be specified in the GUI or CLI:

Adding or editing the target keystore password

The target keystore password is specified in the GUI in the Databases tab under Environments:

1. Login to the Delphix Management application.

2. Click Manage > Environments.

3. Click the Databases tab for your Environment.

4. Next to TDE Keystore Password click on the pencil icon to set or update the password.

   

    

Adding or editing the keystores root directory path

The keystores root directory path is specified in the GUI in the Details tab under Environments:

1. Login to the Delphix Management application.

2. Click Manage → Environments.

3. Click the Details tab for your Environment.

4. Next to Attributes click on the pencil icon to set or update attributes, including the keystores root.

   

Provisioning TDE

The remaining three pieces of information when provisioning to a linked target CDB are specified during the vPDB provision itself by clicking on the Enabled checkbox.

If the target CDB is a virtual CDB, clicking the Enabled checkbox will reveal the two additional necessary fields as well:

Once a TDE-enabled vPDB is provisioned, it can be used the same as a non-TDE-enabled vPDB within Delphix, with the exception of migrate. There are however a few caveats:

• A refresh operation will use the parent keystore for the recovery. If the dSource is rekeyed then the user will need to update the parent keystore with the new keys. Similarly, if the location or password to the parent keystore has changed then they should be updated before the refresh.

• A rewind operation will use the target keystore for the recovery. If the vPDB is rekeyed after it is provisioned, then the rekey will update the target keystore, so it does not need to be updated in Delphix.

• For a single vPDB in a vCDB, if the vCDB keystore location is changed, the new path must be updated in Delphix before refresh or rewind.

• Each disable operation will result in the keys being exported to an exported keyfile in the artifact directory, to be used for a subsequent enable. Refresh and rewind operations will first disable the existing vPDB, so those will also result in a new exported keyfile in the artifact directory.

• Provisioning a second-generation vPDB (vvPDB) from a TDE-enabled vPDB is done in the same manner as a first-generation vPDB, by specifying the TDE parameters during provision. The current keystore for the vPDB can be specified as the parent keystore.

Example illustrating movement of keys

Consider the example illustrated by the vPDB provision above. The vPDB tde_vpdb is provisioned from the dSource CDOMSHSR52CAPDB2 on the VM tde-source18, which is an Oracle database running version 18.11.0. Connecting to this database, we can query v$encryption_keys to determine the current keys in use by each PDB:

SQL> show pdbs
CON_ID CON_NAME               OPEN MODE  RESTRICTED
------ ---------------------- ---------- ----------
     2 PDB$SEED            READ ONLY  NO
     3 CDOMSHSR52CAPDB1        READ WRITE NO
     4 CDOMSHSR52CAPDB2        READ WRITE NO
     5 CDOMSHSR52CAPDB3        READ WRITE NO
SQL> select con_id, key_id from v$encryption_keys order by con_id;

CON_ID KEY_ID
------ ----------------------------------------------------
     1 Ac9MY5kQwU8GvwlYMXImXmMAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
     3 AedrXL3aUk9zv+9t7J8ZsVYAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
     4 AdDdKibLKU9mv6PDAIvVvH0AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
     5 AWdc3ZRaP09Pvw4+2FmLwHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

The v$encryption_keys output for this environment shows that there are 3 PDBs within this CDB, all of which are TDE-enabled. In particular, the PDB used for the dSource has a con_id of 4, and an encryption key starting with AdDdKibL.

The vPDB tde_vdb is provisioned to the CDB CDOMSHTG93CF on the VM tde-target18. Connecting to this database, we can again query v$encryption_keys to determine the keys in use by each PDB:

SQL> show pdbs
CON_ID CON_NAME         OPEN MODE  RESTRICTED
------ ---------------- ---------- ----------
     2 PDB$SEED         READ ONLY  NO
     3 CDOMSHTG93CFPDB1 READ WRITE NO
     4 CDOMSHTG93CFPDB2 READ WRITE NO
     5 CDOMSHTG93CFPDB3 READ WRITE NO
     6 TDE_VPDB         READ WRITE NO


SQL> select con_id, key_id from v$encryption_keys order by con_id;
CON_ID KEY_ID
------ ----------------------------------------------------
     1 AZTc9eKqlk98v8GkQ8/AmaAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
     6 AdDdKibLKU9mv6PDAIvVvH0AAAAAAAAAAAAAAAAAAAAAAAAAAAAA

The key which was originally present in the wallet on the dSource - AdDdKibL - has been imported into the target keystore, and has a con_id of 6, which corresponds to the con_id of the vPDB. There are several things to note about the behavior of Oracle and the v$encryption_keys table:

1. Keys are never deleted from existing keystores by Oracle, only new keys are added. Therefore, if we were to disable the vPDB, which will unmount and unplug it from the CDB, v$encryption_keys will still show the key as present, with its original con_id, even though it has been unplugged:
   

   CODE

   SQL> show pdbs
   CON_ID CON_NAME         OPEN MODE  RESTRICTED
   ------ ---------------- ---------- ----------
        2 PDB$SEED          READ ONLY  NO
        3 CDOMSHTG93CFPDB1  READ WRITE NO
        4 CDOMSHTG93CFPDB2  READ WRITE NO
        5 CDOMSHTG93CFPDB3  READ WRITE NO
   SQL> select con_id, key_id from v$encryption_keys order by con_id;
   CON_ID KEY_ID
   ------ ----------------------------------------------------
        1 AZTc9eKqlk98v8GkQ8/AmaAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
        6 AdDdKibLKU9mv6PDAIvVvH0AAAAAAAAAAAAAAAAAAAAAAAAAAAAA

2. If the wallet is closed for a particular PDB, v$encryption_keys will not show any entries for that PDB. The wallet status can be determined by querying v$encryption_wallet.

3. Querying v$encryption_wallet while the session is attached to CDB$ROOT will return information about the entire CDB, otherwise, only the keys for the current PDB are returned.

TDE keystores root and artifact directory

The artifact directory stores the exported keyfiles used during the workflows for TDE-enabled vPDBs. It is located under the keystores root, in the directory oracle_tde_keystores. Each TDE-enabled vPDB will have its own directory within the oracle_tde_keystores directory, identified by the vPDB name, group name, and a unique identifier, separated by an underscore. If the keystores root directory is not specified, then it defaults to the toolkit directory path.
For example, if the keystores root directory is /work (or keystores root is not specified, and the toolkit directory is /work), the artifact directory for the vPDB tde_vpdb in the group Encrypted could be

/work/oracle_tde_keystores/tde_vpdb_Encrypted_ce7a47e6-8860-4398-bab0-cf0233fc5e3c

Within the artifact directory, there is a subdirectory exported_keys which contains within it the exported keyfiles for each timeflow associated with that vPDB. Each time an export is performed, a new exported keyfile is generated with a timestamp. The contents of the artifact directory may change for future releases, but the path to the artifact directory and the naming convention is not anticipated to change.

As the default keystores root directory root is at the same level as the toolkit directory, it will not be overwritten if a host is refreshed through the Delphix Engine and the toolkit updated. It is the customer's responsibility to maintain the artifact directory and ensure that the contents are not lost, as a disk failure could prevent a TDE-enabled vPDB from being accessed. Thus it is recommended that the keystores root directory be on a disk which is regularly backed up.

If a vPDB is moved to a different host (either through the migrate workflow or an enable after a failover, then the artifact directory will need to be copied to the new target host. See Migrating a TDE-enabled vPDB for details on the manual steps needed for migration.

The artifact directory is not removed when a TDE-enabled vPDB is deleted; the customer can remove it after confirming that the vPDB has been removed (including from any replicated Delphix Engines).