AWS object storage setup

From the Storage setup page, select AWS Object Storage from the Storage Type for Data dropdown menu.

Access method

Choose the preferred method for the engine to access the storage bucket: Role (Instance profile) or Access Key. Role-based access is advocated to mitigate security risks and minimize the maintenance associated with access keys.

Role-based access is recommended to minimize security risks and maintenance related to access keys.

Select region

From the Select region dropdown menu, choose from the available AWS regions. If your desired region is not listed, select Other, which will prompt you to fill out the Endpoint field.

Storage capacity

Input the desired data storage capacity. AWS S3's pricing is based on the actual storage used. For instance, if you provision 10TB but only 1TB is utilized, you will be charged for the 1TB only. Adjusting this number post-setup is straightforward via the sysadmin login or Setup UI, but it cannot be reduced below the current usage by the Delphix engine. The specified number also serves as a quota to limit the object storage from expanding beyond an unforeseen point.

Enter the name of your bucket in the Bucket field and test the connection to verify the VM's access to the bucket.

Block storage for cache

Block devices are utilized as cache, reducing latencies for frequently read data and acting as temporary storage for synchronous writes before the writes are persisted to object storage. The cache should be sized to match the frequently accessed data plus an additional 10% for bookkeeping. Alternatively, begin with a cache sized at 50% of all dSources to be added to the engine.

For specific block storage requirements, refer to the EBS configuration section in the Deployment for AWS EC2 page.

When setting up the disks, ensure they support the throughput of the engine. For example:

gp3 disks are recommended as they offer good performance at a lower cost. At 500 IOPS per 1GiB ratio, a 32 GiB volume can be configured to have the gp3 volume 16K IOPS limit and 1000 MB/s throughput. For reference, r5n.8xlarge instance has a 30K IOPS and 850 MB/s throughput limits so two gp3 devices would be sufficient for the instance.
io2 disks have lower latency at a higher cost. However, the lower latency is not beneficial once the instance IOPS or throughput limit is reached.

Elastic data engines on AWS: Permissions

At a minimum, the following permissions are required for the object storage bucket:

CODE

AWS: [
  "s3:PutObject",
  "s3:GetObject",
  "s3:ListBucket",
  "s3:DeleteObject"
]