API prerequisite knowledge

JSON

JSON (JavaScript Object Notation) is a minimal, readable format for structuring data. It is a simple format for transmitting data between applications, as an alternative to XML. The Delphix API uses JSON data structure in the format of strings to send and receive data from the API calls, as you will see later in the examples. First, let's look at the JSON fundamentals.

Keys and values

The two primary parts that makeup JSON are keys and values. Together they make key/value pairs, also called name/value pairs.

Key – Always a string enclosed in quotation marks.
Value – Can be a string, number, boolean expression, array, or object.
Key/Value Pair – Follows a specific syntax, with the key followed by a colon followed by the value. Key/value pairs are comma separated.

Let's take a JSON sample string and identify each part of the code.

CODE


{    "foo" : "bar",    "rows" : 100}

The curly brackets start and end the string. The key is "foo" and the value is "bar". A colon ( : ) is the delimiter between them. A comma ( , ) is the delimiter for multiple key/value pairs. The second pair is "rows" and the value is a number of 100.

Types of values

Number	An integer or a decimal number
Boolean	True or false
String	Plain text alphanumeric readable characters
Null	Empty
Array	An associative array of values
Object	An associative array of key/value pairs

Numbers, booleans, and strings.

It is very important to understand the APIs JSON object definitions. Quoted values are treated as strings!

"x" : "1" is treated as a string, while

"x" : 1 is treated as a number

"y" : "true" is treated as a string, while

"y" : true is treated as a boolean true (false)

Null values

CODE


{    "z" :   , "b" : "World"}

Nulls are empty values, but sometimes programmers code "" as a null value.

CODE


{    "z" : ""  , "b" : "World"  }

So always verify how the null values are defined and handled by the application.

Arrays

An array is indicated with the square brackets: [ value1, value2, etc. ]. In this example, we have added a categories key with an array of values.

CODE


..."foo" : {  "bar" : "Hello",  "category" : [ "greetings", "morals" ]}...

Objects

An object is indicated by curly brackets: {"key", "value"}. Everything inside of the curly brackets is part of the object. We already learned that a value could be an object. Therefore, "foo" and the corresponding object are a key/value pair.

CODE


..."foo" : { "bar" : "Hello" }...

The key/value pair "bar" : "Hello" is nested inside the key/value pair "foo" : { ... }. That is an example of a hierarchy (or nested data) within JSON data.

Arrays and Objects can be nested or contained within the same level.

Summary

JSON arrays are [ , , ]

JSON nested objects are , , "x":{ "a":"1", "b":"2" }, ,

JSON data can be passed within the HTTP URL (file or argument), the header, or other handlers.

From within Shell Scripts or Programming Languages, JSON data is typically processed through a "JSON parser." This topic is covered later.

Delphix CLI

Connecting to the Delphix engine CLI

Reference: Connecting to the CLI

There are two user roles accessible, the sysadmin and the delphix_admin.

From a shell environment, you can connect using the ssh command. The IP Address (or Hostname) represents the Delphix Engine (case sensitive):

ssh sysadmin@127.16.160.195

ssh delphix_admin@127.16.160.195

From a putty session, open an ssh connection to the Delphix Engine IP Address or Hostname (case sensitive):

open 127.16.160.195

Login User: sysadmin@SYSTEM

#... or ...

Login User: delphix_admin@DOMAIN

After entering the correct password for the respective user, the menus for that user's role will be different. For example, the sysadmin@SYSTEM user has engine storage, whereas the delphix_admin@DOMAIN user has database provisioning.

You can use the CLI for scripting and configure the connection for ssh passwordless connections.

CLI Cookbook: Configuring Key-Based SSH Authentication for Automation

How to use the CLI to learn the APIs

As stated earlier, a great way to learn how to generate the Delphix RESTFul API calls and the required JSON content is to use the Delphix CLI (Command Line Interface) and turn on the CLI> setopt trace=true option.

Below is an example of how to get the JSON required parameters for a database refresh per the type of refresh performed.

Other types or options may require other JSON parameters, so after changing any parameter, we recommend performing an "ls" command to see if there are any new parameters and/or required values.

CODE


The refresh database example below shows how to use the CLI to identify reference objects for other CLI commands and the respective RESTFul API structure when the  setopt trace=true  option is set.

$ ssh delphix_admin@172.16.160.195 Password: Delphix5030HWv8> ls Childrenaboutaction...connectivitydatabaseenvironment... toolkituser OperationsversionDelphix5030HWv8> database Delphix5030HWv8 database> ls ObjectsNAME            PROVISIONCONTAINER    DESCRIPTIONDPXDEV01        - Vdelphix_demo    delphix_demo        -delphix_demo    - Scripts         - V_2C1            Scripts                -Vvfiles            -                    - Childrentemplate OperationscreateEmptycreateRestorationDatasetexportfileMappinglinkoracleSupportedCharacterSetsprovisionvalidateXppxpp

First, we need to identify the target Delphix virtualized database object to refresh ...

Each Delphix object has a reference that is typically used for parameter values.

CODE


Delphix5030HWv8 database> select Vdelphix_demo Delphix5030HWv8 database 'Vdelphix_demo'> ls Properties    type: MSSqlDatabaseContainer    name: Vdelphix_demo    creationTime: 2016-06-16T14:30:03.033Z    currentTimeflow: 'DB_PROVISION@2016-06-16T10:30:08'    delphixManaged: true    description: (unset)    group: Windows    masked: false    os: Windows    performanceMode: DISABLED    processor: x86    provisionContainer: delphix_demo    reference: MSSQL_DB_CONTAINER-39     restoration: false    runtime:        type: MSSqlDBContainerRuntime        logSyncActive: false    sourcingPolicy:        type: SourcingPolicy        loadFromBackup: false        logsyncEnabled: false    transformation: false Operationsdelete...purgeLogsrefreshremoveLiveSource...Delphix5030HWv8 database 'Vdelphix_demo'> refresh Delphix5030HWv8 database 'Vdelphix_demo' refresh *> ls Properties    type: RefreshParameters    timeflowPointParameters:        type: TimeflowPointSemantic        container: (required)        location: LATEST_POINTDelphix5030HWv8 database 'Vdelphix_demo' refresh *> set timeflowPointParameters.container=delphix_demo Delphix5030HWv8 database 'Vdelphix_demo' refresh *> ls Properties    type: RefreshParameters    timeflowPointParameters:        type: TimeflowPointSemantic        container: delphix_demo (*)         location: LATEST_POINTDelphix5030HWv8 database 'Vdelphix_demo' refresh > *commit     Dispatched job JOB-100    DB_REFRESH job started for "Windows/Vdelphix_demo".    Validating that this dataset is managed by Delphix.    Stopping virtual database.    Unmounting datasets.    Unexporting storage containers.    Metadata for dSource "Vdelphix_demo" successfully deleted.    Starting provisioning of virtual database "Vdelphix_demo".    Creating new TimeFlow.    Generating recovery scripts.    Mounting datasets.    Mounting read-only source logs dataset.    Running user-specified pre-provisioning script.    Recovering virtual database.    The virtual database recovery was successful.    Unmounting read-only source logs dataset.    Running user-specified post-provisioning script.    The virtual database "Vdelphix_demo" was successfully provisioned.    DB_REFRESH job for "Windows/Vdelphix_demo" completed successfully.

Refresh again but this time turn on the setopt trace=true option.

CODE


Delphix5030HWv8 database 'Vdelphix_demo'> refreshDelphix5030HWv8 database 'Vdelphix_demo' refresh *> lsProperties    type: RefreshParameters    timeflowPointParameters:        type: TimeflowPointSemantic        container: (required)        location: LATEST_POINTDelphix5030HWv8 database 'Vdelphix_demo' refresh *> set timeflowPointParameters.container=delphix_demoDelphix5030HWv8 database 'Vdelphix_demo' refresh *> lsProperties    type: RefreshParameters    timeflowPointParameters:        type: TimeflowPointSemantic        container: delphix_demo        location: LATEST_POINTDelphix5030HWv8 database 'Vdelphix_demo' refresh *> setopt trace=trueDelphix5030HWv8 database 'Vdelphix_demo' refresh *> commit=== POST /resources/json/delphix/database/MSSQL_DB_CONTAINER-39/refresh ==={    "type": "RefreshParameters",    "timeflowPointParameters": {        "type": "TimeflowPointSemantic",        "container": "MSSQL_DB_CONTAINER-38"    }}...

The "container" value in the JSON output above is different from the target VDB reference because we are refreshing from the source database container! In this example, the set timeflowPointParameters.container=delphix_demo is represented in JSON output as "container": "MSSQL_DB_CONTAINER-38"

Using the CLI, you can identify the RESTFul API POST and GET commands along with the JSON input data requirements.

CODE


=== POST /resources/json/delphix/database/MSSQL_DB_CONTAINER-39/refresh ==={     "type": "RefreshParameters",     "timeflowPointParameters": {         "type": "TimeflowPointSemantic",         "container": "MSSQL_DB_CONTAINER-38"     } }

So framing the RESTFul URL for a virtual database refresh, the URL will look like

http://<delphix_engine>/resources/json/delphix/database/ MSSQL_DB_CONTAINER-39 /refresh

where the MSSQL_DB_CONTAINER-39 represents the target virtualized database to refresh. We need to POST the JSON data to the URL for processing.

CODE


{    "type": "RefreshParameters",     "timeflowPointParameters": {          "type": "TimeflowPointSemantic",         "container": "MSSQL_DB_CONTAINER-38"     }}

The "timeflowPointParameters" key has 6 "type": "..." options, each of which has its own set of parameters. The type "TimeflowPointSemantic" uses the default LATEST_POINT within the source container, so for simplicity, we will use this type. For more information on timeflowPointParameters 6 types, see the Advanced Section.

If this is a little confusing at this point, do not worry, that's typical. Complete examples will be shown later. The important items to remember are:

Delphix often uses object reference names within the JSON data.
Using the setopt trace=true the option provides the construct for the RESTFul API URLs and the JSON data for POST / GET operations.

HTTP

We use the HTTP protocol every day for web browsing and commercial business. From finding a new restaurant to buying a 1986 Ford Thunderbird Turbo Coupe!

Most people see the HTTP within the URL Address field within the Web Browser window – for example, http://www.google.com

But behind the scenes, HTTP is performing a wide range of functionality. For RESTFul APIs, they use HTTP's GET and POST form functionality to process data. In Delphix's case, the data is also represented as JSON structures.

HTTP GET operation is used to return data only, while HTTP POST operation is used to provide data input in the form of a structured JSON data string or file.

cURL

What is cURL?

The cURL client command is based on a library supporting a number of web protocols, including HTTP. The "curl" command can be called from the command line, while the cURL library is commonly integrated with your favorite programming languages, such as Java, JSP, Python, Perl, PHP, .NET, and PowerShell.

Due to its widespread adoption, we will use cURL for making the Delphix RESTFul API calls within this document. Some operating systems or languages support their own HTTP commands / related libraries, and you can use these instead of cURL. One alternative is the "wget" command described later.

Is cURL installed?

CODE


Operating System Prompt>    curl --versioncurl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz

Get the HTTP output from google.com

Operating System Prompt> curl www.google.com

Wget

An alternative to cURL is Wget, which is typically a native command on all Linux environments. See the Appendix for a complete comparison between Wget and cURL.

dxtoolkit2

Delphix has developed a very robust toolkit, dxtoolkit2, which utilizes the Delphix RESTFul APIs. This toolkit is cross-platform. Its commands are built with the Perl programming language.

We recommend that you review the dxtoolkit2 documentation; you may find a utility that already performs your desired function. For example, the utility dx_get_analytics is absolutely great for dumping analytic data from the Delphix Engine into a .csv (comma-separated value) format, which you can then easily integrate into your enterprise monitoring tools. See the sample "Analytics" use case.

Contact Delphix personnel for the latest download.