Staging push

The Staging Push transfers the responsibility of preparing dSource to the end user. This ingestion method allows you to manually push the source data into the mount path backed by the Delphix storage. It offers an alternative approach to the Staging Pull model where the Delphix Continuous Data Engine takes care of preparing the dSource.

The Staging Push ingestion method has been verified with the following mechanisms:

• Backups + Logs w/ pgBackRest

• Backups + Logs w/ pg_basebackup

• Logical Replication

• pg_dump and pg_restore

• Streaming Replication

Prerequisites

Ensure that you meet the following requirements before you begin creating a dSource.

• The environment requirements are met by the source and staging environments as described in Requirements for PostgreSQL Staging/Target environments.

• To use privilege elevation, you must meet the Privilege Elevation Requirements. 

For more information, refer to Requirements and Prerequisites. 

Procedure

Perform the following steps to create a dSource using the Staging Push ingestion method.

1. Login to the Delphix Management application.

2. Navigate to the Manage > Environments page.

3. On the Databases tab, choose a repository for your dSource and click the icon.

4. In the Add Database dialog window, provide the name of your source configuration and click Add.

   

   Note: Starting from PostgreSQL connector version 4.3.0 onwards, a new source config parameter Export Path for Kubernetes VDB on Delphix Engine is added. The value for this field is not to be filled manually and is only used to return the export path for any VDBs created using the Kubernetes provisioning workflow. This field is visible only in the Delphix Continuous Data Engine version below 14.0.0.0.

5. Select your source configuration and click the Add dSource link located on the right.

6. In the Source tab, do the following:

   1. In the NFS Mount Location field, provide a mount location on the staging environment.

   2. Select the Use Staging Push checkbox.

   3. Specify the Staging Instance Port Number. You must provide a port number that is available for the staging instance.

      

   4. Click Next. 

7. In the dSource Configuration tab, provide a dSource name and then select a group for your dSource. Adding a dSource to a dataset group allows you to manage user permissions for that database and its objects, including snapshots. For more information, refer to the topics under Users and Groups.

8. In the Data Management tab, specify your staging environment, user, and snapshot parameters. Select the Resynchronize dSource checkbox if you want to resynchronize the dSource. This operation will force a non-incremental load of data from the source. This operation is similar to creating a new dSource, but prevents duplication of the storage requirements and maintains timeflow history. Click Next to proceed.

9. In the Policies tab, apply policy details to the dSource if needed, and then click Next to continue.

10. In the Hooks tab, select a Hook Point and then click + to add a script you want to run at that hook point. You can define scripts to run at multiple hook points in the process.

11. In the Summary tab, review the configuration profile for your dSource.

12. Click Submit. The Delphix Continuous Data Engine will initiate two jobs, namely DB_Link and DB_Sync to create the dSource. You can monitor these jobs by selecting System>Jobs or by selecting System >Events in the top menu bar. Upon successful completion of the DB_link job jobs, the database icon will change to a dSource icon on the Environments > Host > Databases screen. The dSource will also be listed under Datasets within its assigned group.