Skip to main content
Skip table of contents

Requirements for Db2 hosts and databases

Source Db2 hosts are servers that have Db2 binaries installed and have Db2 instances created on them. The hosts that contain the data that we wish to ingest are referred to as the source environment. Hosts with empty instances (no dbs in instance) are used as either staging or target hosts. This topic describes the requirements for creating connections between the Delphix Engine and Db2 hosts and instances.

Requirements for Db2 source hosts and instances

Each Db2 Source host must have IBM Db2 installed and at least one instance created on the machine. Depending on the way you intend to ingest data into the Staging server, you will also need to ensure that the following requirements are met:

  • For HADR Ingestion: HADR settings for each database to be used with the standby server should be preset before the linking process begins as described in Linking a Db2 dSource

  • For Backup and Log Ingestion, no additional requirements are needed at this point. 

Requirements for Db2 staging and target hosts and instance

  • The staging environment that the Delphix Engine uses must have access to an existing full backup of the source database on disk to create the first full copy. Delphix recommends using compressed backups as that will reduce storage needs and speed up ingest. Once this first full backup is available, subsequent refreshes can be accomplished via HADR or Archive Logs.

  • The staging and target Db2 instances that you wish to use must already exist on the host. We can use the same instance for the dSource and VDB creation and can also have multiple VDBs on the same instance.

  • The available instances on each host can be verified by going to the databases tab for the environment in question.

  • Instance level configuration values such as the bufferpool value will need to be managed by the customer independent of Delphix.

  • The instances used for staging and target environments must be compatible with the source Db2 instance.

Requirements for Db2 DPF staging and target hosts and instances

It is highly recommended that the Database Partitioning Feature (DPF) for Db2 staging and target should be configured on separate hosts due to reason that the DPF environment consumes a lot of resources.

  • The staging and target hosts should be configured with the same number of logical partition nodes as the source.

  • The logical portion configuration should be added in $HOME/sqllib/db2nodes.cfg file where $HOME is the home directory of the instance user.

  • The hostname to be used should be a fully qualified domain name.

  • db2 nodes.cfg file configured for 4 logical partitions should look like:

    CODE 
    0 <FQDN hostname> 0
1 <FQDN hostname> 1
2 <FQDN hostname> 2
3 <FQDN hostname> 3

  • Each logical partition should have a corresponding port entry in /etc/services of hosts. In the example below, db2inst1 is the instance the user and port number from 60000 to 60003 are reserved for inter-partition communication. 

    CODE 
    DB2_db2inst1 60000/tcp
DB2_db2inst1_1 60001/tcp
DB2_db2inst1_2 60002/tcp
DB2_db2inst1_END 60003/tcp

  • Registry variables DB2RSHCMD and DB2COMM should be updated with values as shown below: 

    CODE 
    SSHCMD=$(which ssh)
db2set DB2RSHCMD="$SSHCMD"
db2set DB2COMM=TCPIP

  • After the environment is configured on staging and target, execute db2_all echo hello on both staging and target servers to add host key in the known_hosts file. This will also help to check for any ssh issues on these servers.

  • The backup files (or log directory if using Customer Supplied Archive Logs) should be consistent with the partitions configured on staging.

Additional environment requirements

  • There must be an operating system user (delphix_os) with these privileges:

  • There must be a directory on the staging and target environment where you can install the Delphix Engine Plugin. For example, /var/opt/delphix/toolkit .

    • The delphix_os user must own the toolkit path directory (__/var/opt/delphix/toolkit).

    • If the delphix_os user and instance users (responsible for the Delphix Engine operations such as linking and provisioning) are sharing any common group, then the toolkit path directory (__/var/opt/delphix/toolkit) must have permissions rwxrwx-- (0770), but you can also use more permissive settings. This allows instance users who are part of the same group as the delphix_os user's group to be able to create directories inside the toolkit path directory.

    • If the delphix_os user and instance users (responsible for the Delphix Engine operations such as linking and provisioning) are not sharing any common group, then the toolkit path directory must have -rwxrwxrwx (0777) permissions.

    • The directory should have 1.5GB of available storage: 400MB for the Plugin and 400MB for the set of logs generated by each Db2 instance that runs on the host

    • In Db2 Plugin, plugin directory space will be used as the base location for the mount point

    • When multiple engines access the same host environment, create a separate Db2 plugin directory for each engine that will discover this environment.

  • When configuring staging and target environment for multiple partitions (dpf), the ClientAliveInterval option must be set to 0 or commented out to prevent the SSH connection from getting severed in the midst of command executions.

For Db2 plugin version 4.0.0 and onwards, follow the steps below to change the toolkit directory.

  1. Disable all the datasets related to that environment

  2. The instance of the dataset must have the DIR_CACHE configuration parameter set to No, as shown below.
    db2 update dbm cfg using DIR_CACHE NO

  3. Upload the new vSDK-based plugin. The plugin has upgrade logic implemented in it and will upgrade the existing Lua-based plugin.

  4. Refresh the environment to allow the plugin to create plugin paths and files under the scratch path.

  5. Enable all the datasets and complete the migration process.

  • The Delphix Engine must be able to initiate an SSH connection to the staging and target environment 

  • NFS client services must be running on the staging and target environment

  • Staging and Target host instances must have the same Db2 instance code page as the production instance. The instance owner must have a compatible value defined in LC_ALL or LC_CTYPE for the Db2 code page. Preferably derived from the value of LC_ALL, LC_CTYPE, or LANG of the instance owner on the production host.

  • The primary environment user should share its primary group with the instance user. For example, if the delphix_os is the primary environment user which is used for environment addition and its primary group is also delphix_os then instance users (responsible for the Delphix Engine operations such as linking and provisioning) should share group delphix_os as their secondary group.

Custom configurations in Db2 plugin

DB2 plugin v3.x

The advanced config file is named advanceConfFlag.conf and is created under the <Toolkit directory> during discovery or environment refresh operation if the file does not pre-exist already. This file will have the following configurable parameters:

  • Common group (notCommonGroupFlag):

    • This parameter sets the permission of "mnts", "logs" and "code" directories under the <Toolkit dir> /db2="">location.

    • Set notCommonGroupFlag to false, if the primary group of the primary environment user (user which is used to do discovery) is shared with the instance user.

    • Set notCommonGroupFlag to true, if the primary environment user and the instance user do not share any user group.

    • Once the necessary changes are made, refresh the environment in order to apply the changes.

By default, the notCommonGroupFlag parameter is commented out in the file. This means that the plugin implicitly assumes its value as false. The valid values for this parameter are true or false.

  • Allow Source Database on Same Instance (allowSourceDBonSameInst) By default, the DB2 Plugin restricts the provisioning of a VDB on a DB2 instance which contains a database with the same name as the VDB's source database For example:

    • When provisioning a VDB from a VDB then the latter is treated as the source database.

    • When provisioning a VDB from a dSource then the staging database representing the dSource is treated as the source database.

Set allowSourceDBonSameInst to true, if the user wants to provision a VDB on an instance which contains a database with the same name as the VDB's source database. The valid values for this parameter are true or false.

  • Restore Pipeline limit (restorePipelineLimit): Db2 DPF Plugin performs restoration of all non-catalog partitions in parallel. The parameter restorePipelineLimit allows the user to configure the number of partitions that could be restored in parallel by the plugin. When the new advanceConfFlag.conf file is created during environment refresh or discovery operation, the default value of restorePipelineLimit is set to 10. The user can modify the value of this parameter to a desired value. The valid value for this field is any positive integer.

A default config file created during discovery or environment refresh operation:

CODE 
# If the user is not sharing a common group between primary environment user and instance user.

# Then the user needs to set parameter notCommonGroupFlag as true.

# notCommonGroupFlag=true

# During provision operation check if instance contains a database name which is identical to the source DB name  used for provisioning operation.

# By default plugin will never allow creating a VDB on the instance where source database (or other database which is identical in name with source DB) already exists.

# If the user still wants to create a VDB on the instance which contains a database name which is identical to the source DB name used for provisioning operation,

# then the user needs to set parameter allowSourceDBonSameInst as true.

# allowSourceDBonSameInst=true

# Limit for parallel restores

restorePipelineLimit=10

DB2 plugin v4.x and onwards

DB2 plugin config file is named db2_plugin.conf and is created under the <Toolkit directory>/<Delphix_COMMON>/plugin/DB2_18f4ff11-b758-4bf2-9a37-719a22f5a4b8/ directory during discovery or environment refresh operation if the file does not already exist. This file has two sections represented by the headers - plugin_custom_parameters and plugin_logging_parameters.

It is important for the users to maintain two section headers in the db2_plugin.conf file.
If any of the sections get deleted, the plugin will pick the default value for the parameters under the deleted section.

Plugin custom parameters

Common group (usersHaveCommonGroup)

  • This parameter sets the permission for the "mnts" directory located under <Toolkit dir>/DB2 directory and for the "logs" directory located under the <Toolkit directory>/<Delphix_COMMON>/plugin/DB2_18f4ff11-b758-4bf2-9a37-719a22f5a4b8/ directory.

  • Set the usersHaveCommonGroup parameter as True, if the primary group of the primary environment user (user which is used to do discovery) is shared with the instance user. 

  • Set the usersHaveCommonGroup parameter as False, if the primary environment user and the instance user do not share any user group.

  • Once the necessary changes are made, refresh the environment in order to apply the changes.

 By default, the usersHaveCommonGroup parameter is set to True. The valid values for this parameter are true or false.

Allow Source Database on Same Instance (allowSourceDbOnSameInstance)

By default, the DB2 Plugin restricts the provisioning of a VDB on a DB2 instance which contains a database with the same name as the VDB's source database For example:

  • When provisioning a VDB from a VDB then the latter is treated as the source database.

  • When provisioning a VDB from a dSource then the staging database representing the dSource is treated as the source database.

If the user wants to provision a VDB on an instance which contains a database with the same name as the VDB's source database, then the parameter allowSourceDbOnSameInstance must be set to true. The valid values for this parameter are true or false.

Restore pipeline limit (restorePipelineLimit)

Db2 DPF Plugin performs restoration of all non-catalog partitions in parallel.
The parameter restorePipelineLimit allows the user to configure the number of partitions that could be restored in parallel by the plugin. When the new advanceConfFlag.conf file is created during environment refresh or discovery operation, the default value of restorePipelineLimit is set to 10. The user can modify the value of this parameter to the desired value. The valid value for this field is any positive integer.

Skip archive log validation (skipArchiveLogValidation)

Currently, before rolling forward, the Db2 plugin checks all archive logs to assess the integrity of the archive log files and to decide if the log files are healthy and can be used when rolling forward the database. This process is repeated every time during a resync or when creating a DSource snapshot. 

This option allows the user to set the plugin to skip the validation process and perform a partial validation. By default, the plugin is configured to examine all archive logs and the parameter value is set to False. To be able to use this functionality, the parameter value must be set to True.

Plugin logging parameters

This feature provides three options to configure and control enhanced logging by the plugin:

  1. To set debug log levels.

  2. To set active log file size.

  3. To set log retention levels.

Delphix users can modify the above parameters defined under the plugin_logging_parameters section of this file.

Setting Log Levels (level)

Delphix users can set the logging level of the plugin-generated logs by setting the parameter level under the plugin_logging_parameters header. Higher logging levels will help to expedite debugging issues. There are two levels of logging which are: Info and Debug. The configured logging level will apply to all the objects (both dSources and VDBs) present on a staging/target host. For example; if we have 4 dSources on a host associated with a Delphix Engine then the log level will apply to all the dSources.

Log level description:

  • level=INFO: This level will print only informational logs. This is the default log level.

  • level=DEBUG: This level will print informational logs and debug statements.

Setting maximum size of active log file (logFileSize)

Delphix users can set the maximum size of the active log file by defining the value of the logFileSize parameter under the plugin_logging_parameters header of the db2_plugin.conf file. Once this maximum size limit is reached, the plugin will rotate the log as per the retention property described below. This parameter only takes in a positive integer value. The minimum value of this parameter is 1 MB and the maximum value is 10 MB. The default value is 1 MB.

Setting log retention levels (retention)

Delphix users can set a retention level for plugin-generated logs for each dataset (<Db name>.diag.log) using the parameter retention under the plugin_logging_parameters header. As per this parameter, the log files are moved (archived), renamed or deleted once they reach the value set in the logFileSize parameter. New incoming log data is directed into a new fresh file (at the same location).

By default, the value for this parameter will be set to a minimum value of 2. The user can change this value and set its value within the range 2 and 50. For example; if retention is set to 4, the plugin will have the following log files: <DB Name>.diag.log, <DB Name>.diag.log.1, <DB Name>.diag.log.2, <DB Name>.diag.log.3, <DB Name>.diag.log.4. 

  • File <DB Name>.diag.log is the active log file

  • File <DB Name>.diag.log.1 is the most recent archive log file

  • File <DB Name>.diag.log.4 is the oldest one

The DB2 plugin will ignore values for a parameter if the values do not fit the description above and will continue the operation with the default set of values for that particular parameter.

db2_plugin.conf

CODE 
#
# The parameters must not be left padded with spaces. The plugin will otherwise not accept the specified parameter
# values and will use the default values.
#
 
[plugin_custom_parameters]
#
# If the user has not shared a common group between primary environment user and instance user.
# Then the user needs to set parameter usersHaveCommonGroup as False.
#
 
usersHaveCommonGroup=True
 
#
# During provision operations the plugin checks if the target instance contains a database name which is identical to
# the source DB name from which the VDB is to be created. By default the plugin will never allow creating a VDB on the
# instance where the source database already exists, or on any instance that has a database with the same name. If the
# users still want to create a VDB on an instance which contains a database with the same name as the source DB then
# they need to set parameter allowSourceDbOnSameInstance as True.
#
 
allowSourceDbOnSameInstance=False
 
#
# If the users want to control the number of parallel restores then they can tune the parameter restorePipelineLimit.
# By default the value of this parameter would be 10.
#
 
restorePipelineLimit=10
 
#
# The archive logs validation process can be skipped if the end user so chooses. Instead, a speedier technique can be used,
# in which the archive logs are only validated for the first and last logs and the file sequences in between are confirmed.
#

skipArchiveLogValidation=False
 
[plugin_logging_parameters]
#
# This flag will set the debug level of plugin logs on the remote server (both staging and target). There are two levels:
# Info
# Debug
# The above are the only valid values that can be assigned. If any other value is assigned, the plugin will set the
# default level as Info. the string is case insensitive so info, Info and INFO are acceptable.
#
 
level=INFO
 
#
# This parameter will set the maximum size of the active log file in MB. Once this limit is reached, the plugin will
# rotate the log as per the retention property defined below. The minimum value of this parameter is 1 MB and maximum
# value is 10 MB. This parameter only takes in a positive integer value. The default value is 1 MB
# logFileSize=<positive integer>
#
 
logFileSize=1
 
#
# Whenever the size of <DB Name>.diag.log file exceeds the value provided by logFileSize parameter then the plugin will
# rename the active log file to <DB Name>.diag.log.<number> and a new log file with name <DB Name>.diag.log will be
# generated. For example, if LogRetention is set to 4, the plugin will have the following log files: <DB Name>.diag.log,
# <DB Name>.diag.log.1, <DB Name>.diag.log.2, <DB Name>.diag.log.3, <DB Name>.diag.log.4.
# File <DB Name>.diag.log.4 will be the oldest one.
# File <DB Name>.diag.log.1 will be the most recent archive log file.
# File <DB Name>.diag.log will be the active log file.
# The minimum and default value for retention flag is 2.
# retention=<positive integer>
#
 
retention=2

Instance user requirements

  • The instance owner of each instance you wish to use within staging or a target host must be added as an environment user within the Delphix engine. See Managing Db2 Users and Instance Owners

  • For HADR synced dSources the staging instance owner must be able to "read" the ingested database contents as Delphix will check the validity of the database by querying tables before each dSource snapshot.

If a container is added or deleted, the dSource will have to be resynced.

  • Ensure that the following database configurations are set to default values:

Database Container Requirements

  • All Db2 database container types are fully supported with the exception of Db2 raw containers.

  • It is recommended not to use special characters for source database container paths as this may create problems while parsing such container paths and may also result in failures during dSource and VDB creation.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.

Contact Support Close