vFiles best practices and common pitfalls

Overview

This document is the implementation guide for the best practices of implementing data source integrations using the Delphix vFile functionality provided through the AppData toolkit. Since "unstructured files", or vFiles, typically is implemented for non-DBMS file types, this implementation will require a certain degree of configuration and scripting in order to function. This means that while you do get some of the main functionality of the Delphix Engine, there are also quite a few pitfalls that may deter customers from wanting this type of configuration. Using vFiles and scripts to implement data source ingestion is a workaround that should only be used if you and the customer are fully aware of the best practices and limitations as called out below.

Best practices and implementation

In order to be successful with this workaround, please keep in mind these best practices which will enable you to be successful in implementing a vFile configuration for data sources. 

• Write a script to put the RDBMS into 'Backup' mode to prepare it for the next set of scripts Delphix will perform

• Create tablespaces in the RDBMS which dxtoolkit will take snapshots of while the database is in backup mode

• After any vFile provisioning job, script a recovery of the RDBMS instance

Common use cases

These use cases are the typical scenarios for which you would use this workaround.

Unsupported DBMS configuration

This functionality is most appropriate when a customer is using an unsupported data source and wants a quick MVP to demonstrate the value of Delphix. It is critical to understand the pitfalls of this implementation and what value propositions that this excludes so that the customer is aware of what capabilities our core integrations provide.

Common pitfalls

These pitfalls of functionality should be considered whenever implementing vFile scripted ingestion. Refer to the table for a quick reference of pitfalls.

LogSync and Point in Time provisioning

Since these scripts are taking full backups of the database, there will not be logs available for us to sync with. Therefore, point in time provisioning is not available to any vFile implementation.

Manual operations

Currently, customers using this workaround have to perform the scripts manually. This creates operational overhead for customers, particularly their DBA team. 

Clustering

Using vFiles is not available for clustered instances, and there is currently no workaround for this type of configuration.

Lack of Graphic User Interface (GUI)

Scripts perform all the core functionality that is typically carried out in the Delphix GUI. This implementation precludes the usage of the main GUI and is only accessible via the Command Line Interface (CLI).

Production downtime

As the database enters backup mode during the ingestion process, this workaround causes the production source to go down while Delphix ingests their data.

Understanding and implementing scripts and dxtoolkit

Implementing these scripts requires an understanding of dxtoolkit and the data source which you are trying to integrate with. Typically, we estimate the learning period of this to be about 1 - 2 months to fully understand how to use these tools and how to best implement the scripts for any given source.

Conclusion

Using vFile functionality with scripts is a quick way to demonstrate the value of the Delphix Engine easily. With a little scripting and Delphix knowledge, you can engineer ingestion with dxtoolkit to take snapshots of the entire database. However, there are several pitfalls that every customer should know about and understand before moving forward with this type of implementation. If you have any questions, please reach out to Product Management.