Delphix-initiated single database backup

The following diagram demonstrates the architecture of the Delphix Continuous Data Engine support for the PostgreSQL database with pg_dump logical backup between source and staging database.

Perform the following steps to create a dSource with a Delphix-initiated backup that uses a single database backup to stay in sync with the source.

Prerequisites

Ensure that you meet the following requirements before you begin creating a dSource.

• The environment requirements are met by the source and staging environments as described in Requirements for PostgreSQL Staging and Target environments.

• The environment user or the privileged user must have the following permissions:

  • access to the Postgres commands

  • read+write+execute permissions on the unix_socket_directories configured in postgresql.conf file (The default directories are /var/run/postgresql & /tmp).

• To use privilege elevation,

  • You must meet the Privilege Elevation Requirements. 

  • The high-privilege user must have the following permissions:

    • Write+execute permissions on the PostgreSQL Backup Dump Directory.

For more information, refer to Requirements and Prerequisites. 

Procedure

Perform the following steps to create a dSource using the Delphix-initiated single database backup ingestion.

1. Login to the Delphix Management application.

2. Navigate to the Manage > Environments page.

3. On the Databases tab, choose a repository for your dSource and click the + icon.

4. In the Add Database dialog window, provide the name of your source configuration and click Add.

   

   Note: Starting from PostgreSQL connector version 4.3.0 onwards, a new source config parameter Export Path for Kubernetes VDB on Delphix Engine is added. The value for this field is not to be filled manually and is only used to return the export path for any VDBs created using the Kubernetes provisioning workflow.This field is visible only in the Delphix Continuous Data Engine version below 14.0.0.0.

5. Select your source configuration and click the Add dSource link located on the right.

6. In the Source tab, do the following:

   1. In the NFS Mount Location field, provide a mount location on the staging environment.

   2. If using Privilege Elevation, provide the username of the Privileged OS Account.
      Note: Once the dSource is created, you will not be able to update the privileged OS account. Recreating the dSource account is the only way to make changes.

   3. Select the Delphix Initiated Backup - Single Database Ingestion checkbox. Enabling this checkbox allows Delphix to initiate the logical backup on the source database.

      

   4. Click on the +Add icon next to the Single Database Ingestion method parameter.
      Note: You must click the “Add Button” for External Backup only once as the solution supports a single backup path. Providing multiple paths will result in an error during the linking process.

   5. Provide the values in the following fields.

      • Source Postgres Database Name

      • Source Postgres Database User Name. The default value is postgres.

      • Source Postgres Database User Password

      • Source Postgres DB Host

      • Source Instance Port Number. The default value is 5432.

      • Postgres Backup Dump Directory: Provide the directory on the staging environment on which the backup will be created. This directory should already exist in the staging environment and must have read and write access for the environment user.

      • Number of Backup Jobs: The default value is 2.

      • Number of Restore Jobs: The default value is 2.

      •  (Optional) Staging postgresql.conf config file: Provide the path to a PostgreSQL configuration template file that will be used when starting the dSource.

        

   6. Specify the Staging Instance Port Number. This port number must be unique and the port must be free for a given instance.

   7. Optionally, you can define database configuration parameters during the linking operation in the "Config Settings" section if needed.

      

Note: During dSource creation, you now can configure the parameters and their values in the postgresql.conf file using the "Config Settings" section in the UI. For example, if you wish to disable SSL via UI, you can specify the "ssl" parameter and its value through the UI. This action will update the existing parameter value in the postgresql.conf file with the value provided.
Similarly, if you want to disable a parameter then you can specify the parameter name and then select the Comment Property check box. This action will comment out the parameter in the postgresql.conf configuration file.

h. Click Next.

7. In the dSource Configuration tab, provide a dSource name and then select a group for your dSource. Adding a dSource to a dataset group allows you to manage user permissions for that database and its objects, including snapshots. For more information, refer to the topics under Users and Groups.

8. In the Data Management tab, specify your staging environment, user, and snapshot parameters. Select the Resynchronize dSource checkbox if you want to resynchronize the dSource. This operation will force a non-incremental load of data from the source. This operation is similar to creating a new dSource, but prevents duplication of the storage requirements and maintains timeflow history. Click Next to proceed.

9. In the Policies tab, apply policy details to the dSource if needed, and then click Next to continue.

10. In the Hooks tab, select a Hook Point and then click + to add a script you want to run at that hook point. You can define scripts to run at multiple hook points in the process.

11. In the Summary tab, review the configuration profile for your dSource.

12. Click Submit. The Delphix Continuous Data Engine will initiate two jobs, namely DB_Link and DB_Sync to create the dSource. You can monitor these jobs by selecting System>Jobs or by selecting System >Events in the top menu bar. Upon successful completion of the DB_link job jobs, the database icon will change to a dSource icon on the Environments > Host > Databases screen. The dSource will also be listed under Datasets within its assigned group.

Starting with PostgreSQL connector version 4.0.0 and onwards, support has been added for using pg_dumpall with single database ingestion. This allows you to obtain global objects such as roles and tablespaces during the ingestion process.