Network operations using the Delphix session protocol
This topic describes how the Delphix Engine uses the Delphix Service Protocol (DSP) for network operations, and how this affects features such as replication, V2P, and SnapSync.
Overview
Delphix Session Protocol, or DSP, is a communication protocol that operates at the session and presentation layer in the Open Systems Interconnection (OSI) model.
DSP supports the request-reply pattern for communication between two networked peers. In contrast to the traditional remote procedure call (RPC) models, which focus exclusively on low-level details such as data encoding and wireframing, DSP implements a generic session layer that supports a number of advanced functionalities desired for network communication, including:
Full-duplex remote operation execution and end-to-end cancellation support
Advanced connectivity model with connection trunking and ordered delivery
Fault resilience with automatic connection and session recovery, exactly-once semantics, and optional data digest
High performance with concurrent execution, session flow control, optional data compression, and bandwidth throttling
Built-in security support with pluggable SASL authentication mechanisms and optional TLS encryption
Asynchronous model for session management and remote operation
Most of the features above are essential to the proper operation of a distributed application and yet non-trivial to implement. By offering them in the framework, we can significantly simplify the development of enterprise quality distributed applications.
DSP is officially registered with the Internet Assigned Numbers Authority under the service name dlpx-sp and port number 8415.
Currently, DSP supports Java language binding and provides a java based service framework for distributed applications.
Key concepts
The foundation of DSP is built on top of a few key abstractions, namely, exchange, task, nexus, and service. For an overview of how DSP works and the features it provides, let’s start with these abstractions.
An exchange refers to an application-defined protocol data unit which may be a request or a response. DSP supports the request-response pattern for communication. For each request sent, there is a corresponding response that describes the result of the execution. An application protocol is made up of a set of exchanges.
A nexus (a.k.a., session) refers to a logical conduit between the client and server application. In contrast, a transport connection (a.k.a., connection) refers to a “physical” link. A nexus has a separate naming scheme from the connection, which allows it to be uniquely and persistently identified independently of the physical infrastructure. A nexus has a different lifecycle than the connection. It is first established over a leading connection. After it comes into existence, new connections may be added and existing ones removed. It must have at least one connection to remain operational but may live on even after all connections are lost. Nexus lifecycle management actions, such as create, recover, and destroy, are always initiated by the client with the server remaining passive.
A nexus has dual channels, namely, the fore channel and the backchannel. The fore channel is used for requests initiated from the client to the server, and the backchannel from the server to the client. From a request execution perspective, the nexus is full-duplex and the channels are functionally identical, modulo the operational parameters that may be negotiated independently for each channel. A channel supports a number of features for request processing, such as ordered delivery, concurrent execution, remote cancellation, exactly-once semantics, and throughput throttling.
A service refers to a contract that consists of all exchanges (both the requests and the corresponding responses) defined in an application protocol. Given the full-duplex nature of request execution in DSP, part of the service is fulfilled by the server and the remaining by the client, where the client and server are from the nexus management perspective.
A task implements a workflow that typically involves multiple requests executed in either or both directions over the nexus. A task is a self-contained building block, available in the form of a sharable module including both the protocol exchanges and implementation, that can be easily integrated into other application protocols. A library of tasks may significantly simplify distributed application development by making it more of an assembly experience.
The following is a diagram that illustrates the key abstractions and how they are related to each other.
Security
As a network protocol, DSP is designed with security in mind from the onset. It supports strong authentication as well as data encryption. It follows a session-based authentication model which requires each connection to authenticate before it is allowed to join the session. Authentication is performed using the Simple Authentication and Security Layer (SASL) framework, a standard-based pluggable security framework. The currently supported SASL mechanisms include DIGEST-MD5, PLAIN with TLS, CRAM, and ANONYMOUS. Optionally, TLS encryption may be negotiated between the client and the server for data privacy.
Performance
DSP offers a number of features to enable the support for high-performance network applications. For example, it allows multiple requests to be exchanged in both directions simultaneously, which provides effective pipelining of data transfer to minimize the impact of network latency while ensuring the total ordering at the same time. It supports trunking that can effectively aggregate the throughput across multiple connections, which is crucial for a long fat network (LFN) and 10GigE. It also provides optional compression support which boosts performance over a bandwidth-limited network. We have observed, through both internal benchmarking and in customer environments, DSP-based applications delivering multi GigE in an ideal environment and getting a performance boost of as much as x10 in bandwidth-limited settings.
Resiliency
DSP automatically recovers from transient connection loss without any application involvement. It may also detect random data corruption on the wire and automatically recovers from it. In both cases, outstanding requests are retried once the fault condition is resolved.
DSP offers control over a remotely executing request. Once a request is initiated, the application may cancel it at any time before completion. In the rare event of a session loss, a new session creation request will be held until the old session has been reinstated. It ensures that we never leave any unknown or unwanted activities on the remote side and provides better predictability and consistency guarantees over an otherwise unreliable network.
Diagnosability
Application exceptions encountered during remote execution of a request are communicated back to the initiator through DSP. A standard Java API is used to facilitate the handling of remote exceptions that is in many ways identical to local ones.
DSP provides detailed information and statistics at the session level. The information may be used to examine the state of the session as well as diagnose performance problems. It is currently exposed via an internal support tool called the JMX tool.
Supported applications
Replication is the first feature to take advantage of DSP. It has been rebuilt on top of DSP and shipping in the field since 3.1. In the latest release, a number of host-based applications, such as SnapSync, V2P, and Delphix connector, use DSP as well.