Staging push implementation

The Staging Push feature enables you to push data into the Delphix-provided mount point on your own. It is an alternative Delphix dSource data ingestion approach that enables you to push the data in Delphix, which is different from the conventional pull model where Delphix pulls the data.

There are two mechanisms to create dSource on the staging server.

• Pull architecture: The Postgres plugin pulls the data into the Delphix-provided mount location.

• Staging Push architecture: You push the data into the Delphix-provided mount point on your own.

Staging push implementation details

Staging Push gives you control over some staging database processes. It will give you control of the staging database to ingest data at either the cluster or individual database level. Staging database files will be stored on Delphix storage.

The following ingestion mechanisms are verified:

• Postgres Streaming Replication

• Backups + Logs w/ pg_basebackup

• Backups + Logs w/ pgbackrest

• Logical Replication and

• pg_dump and pg_restore method

The first snapshot created as a part of dSource creation contains a data directory within the Delphix created mount point. This directory will be empty for the first snapshot.

The first snapshot of the dSource will be empty. In this case, dSource snapshot metadata will contain the information whether the snapshot taken is provisionable or not. If you try to create a VDB from the first snapshot, then the Plugin will error out the operation with a proper error message.
The below steps show how to create a dSource using the Staging Push mechanism.

dSource Creation

The first snapshot created as a part of dSource creation contains a data directory within the Delphix created mount point. This directory will be empty for the first snapshot. The following snippet shows how the file system appears inside the Delphix-created mount point.

If you attempt to create a VDB from this snapshot, it will fail with the below error message.

Subsequent Snapshots

The system user needs to follow the steps and commands below before taking any subsequent snapshots.

1. You must recover the physical backup (pg_basebackup or pgBackRest) in the data directory or if you are recovering the backups created through pg_dump then you first need to initialize the Postgres Cluster on the data directory and then you can recover the backup through pg_restore.

2. While recovering the physical backup, you must make sure that all the tablespaces (if exists) should be present under the mount point only.

   The following screenshot represents what the file system will look like once the recovery is complete.

3. 

   You can now proceed to take a snapshot and the VDB creation using the new snapshot. 

   It’s a pre-requisite for the end-user to stop the cluster before taking a snapshot and start the cluster after successful snapshot. If the cluster is not stopped, then the snapshot operation will fail with tan error message.

Disable/Delete Operation

For dSource disable and delete operation, it is a pre-requisite for the end-user to stop the cluster before kicking these operations from the UI. If it's not stopped, then the operations will fail with an error message.

Resync Operation

If the data directory under the mount path contains any file, then the resync operation will fail with an error message. So, for the resync operation, it is a pre-requisite that the data directory should be empty.