Connection concentrator

 

The connection concentrator reduces the resources required on DB2® for OS/390® and z/OS® database servers to support large numbers of workstation and web users. This function can dramatically increase the scalability of your DB2 for OS/390 and z/OS and DB2 Connect™ solution while also providing for fail-safe operation and transaction level load balancing in DB2 for OS/390 and z/OS data sharing environments.

The connection concentrator allows applications to stay connected without any resources being consumed on the DB2 host server. You can have thousands of users active in applications and only have a few threads active on the DB2 host server.

DB2 Connect's connection concentrator technology allows DB2 Connect server products, such as DB2 Connect Enterprise Edition, to provide support to thousands of users simultaneously executing business transactions, while drastically reducing resources required on the S/390® host or System i™ database servers. It accomplishes this goal by concentrating the workload from all applications in a much smaller number of S/390 host or System i database server connections. While this might seem similar to the connection pooling function described above it is in fact a more sophisticated approach to reducing resource consumption for very high volume OLTP (On-line Transaction Processing) applications.

Connection concentrator takes the concept of an agent and splits it into two entities:

When a new application attempts a connection to the host, it is assigned a logical agent. To pass SQL to the database, a coordinating agent is required and is assigned as soon as a new transaction is initiated. The key to this architecture is the fact that the coordinating agent is:

Another key feature is the method of assigning coordinating agents to new transactions in a data sharing environment. DB2 Connect implements a sophisticated scheduling algorithm that uses OS/390 and z/OS Work Load Manager (WLM) information. This information is used to distribute workload across members of a data sharing group according to criteria set up in WLM. WLM is not only aware of the load on each member but also their availability. This allows DB2 Connect to transparently relocate work away from failed or overloaded members to members that are up and underutilized. DB2 Connect connection concentrator is activated when you set the number of maximum logical agents (max_connections) higher than the number of coordinating agents (max_coordagents).

Connection pooling saves the cost of establishing a connection when one is no longer needed by a terminating application. In other words, one application has to disconnect before another one can reuse a pooled connection.

Alternatively the connection concentrator allows DB2 Connect to make a connection available to an application as soon as another application has finished a transaction and does not require that other application to disconnect. In essence, a database server connection and its associated host and DB2 Connect resources are used by an application only while it has an active transaction. As soon as the transaction completes, the connection and associated resources are available for use by any other application that needs to have a transaction executed.

In previous versions of DB2 Connect, every active application had an Engine Dispatchable Unit (EDU) which managed the database connection as well as any application requests. This EDU was typically referred to as the coordinator agent. Each coordinator agent tracked the state, or context of the application and EDU. Each EDU takes a significant amount of memory when the number of connections increases, and context switching between agents results in additional overhead.

In the above architecture, there is a one-to-one relationship between connections and EDUs. The connection concentrator, however, permits a many-to-one relationship between connections and EDUs. That is, the relationship of connections (X) to EDUs (Y) is now X >= Y.

The connection concentrator splits the agent into two entities, a logical agent and a worker agent. Logical agents represent an application, but without reference to a particular EDU. The logical agent contains all the information and control blocks required by an application. If there are n applications connected to the server, there will be n logical agents on the server. Worker agents are physical EDUs that execute application requests, but which have no permanent attachment to any given application. Worker agents associate with logical agents to perform transactions, and at the transaction boundary end the association and return to the available pool.

An entity known as the dispatcher assigns worker agents to logical agents. Limitations in the number of open file handles on certain computing platforms might result in more than one scheduler instance.

Restrictions for the connection concentrator

There are a number of important restrictions to the use of the DB2 Connect server concentrator. Review the following information in its entirety before attempting to use the connection concentrator on your system.

General restrictions:

When working with DB2 Version 9 or Version 8 FixPak 13 (or higher), to enable DB2 Connect concentrator support requires System i Version 5 Release 4 (PTF SI23726). Otherwise, only the XA portion of the connection concentrator is supported.

Activating the connection concentrator

The database manager configuration parameter max_coordagents sets the maximum number of logical agents. You can activate the concentrator feature by setting the value of max_connections to any number greater than the default. The default value for max_connections is equivalent to the value of max_coordagents. Because each application will have one logical agent, max_connections actually controls the number of applications that can be connected to the database instance, while max_coordagents controls the number of inbound connections that can be active at any time. max_connections will take a numeric range from max_coordagents up to 64,000. The default number of logical agents is equal to max_coordagents.

Both max_connections and max_coordagents can be set to AUTOMATIC. If max_connections is set to AUTOMATIC, the number of connections can be increased beyond the base configured value. If both max_connections and max_coordagentsare set to AUTOMATIC, max_connections can be increased beyond the base value, and max_coordagents is automatically increased to maintain the concentration ratio between connections and the coordinator agents.

Several existing configuration parameters are used to configure agents. These parameters are as follows:

max_coordagents

Maximum number of active coordinator agents.

num_poolagents

Agents pool size. The agent pool includes inactive agents and idle agents. For improved performance, num_poolagents should be configured to equal the average number of clients.

num_initagents

Initial number of worker agents in the pool. These will be idle agents.

XA transaction support

The architecture of the connection concentrator allows DB2 Connect to provide tightly coupled XA transaction support to DB2 for OS/390 and z/OS and DB2 for System i. The concentrator will associate a worker agent with a particular XA transaction (single XID) as it would for any other transaction. However, if the XA transaction is ended by xa_end() (branch boundary), the worker agent will not release itself into the general pool. Instead, the worker remains associated with that particular XA transaction. When another application joins the same XA transaction, the worker agent will be attached to that application.

Any transaction boundary call will return the agent to the pool. For instance, xa_prepare() with read only, xa_rollback(), xa_recover(), xa_forget(), xa_commit(), or any XA error that causes rollback will return the agent to the normal pool. Xa_end() itself only ends the transaction branch, and this is not sufficient to end its association with the XID.

Examples of XA transaction support

  1. Consider an environment where 4,000 or more concurrent connections are needed. A web server that uses CGI applications, or an office system with many desktop users can both exceed this requirement. In these cases, efficiency will usually require that DB2 Connect operate as a stand-alone gateway; that is, the database and the DB2 Connect system are on separate machines. The DB2 Connect server system might not be able to maintain 4,000 simultaneous open connections to the database machine. In most cases, the number of transactions occurring at any given moment will be considerably less than the number of concurrent connections. The system administrator could then maximize the efficiency of the system by setting the database configuration parameters as follows:

         MAX_CONNECTIONS =  4,000      MAX_COORDAGENTS =  1,000      NUM_POOLAGENTS  =  1,000
    The concentrator will keep open up to 4,000 concurrent sessions, even though the gateway is only managing 1,000 transactions at a time.

  2. In the above example, worker agents will constantly form and break associations to logical agents. Those agents that are not idle might maintain a connection to the database but are not participating in any particular transaction, hence they are available to any logical agent (application) that requests a connection.

    The case of XA transactions is somewhat different. For this example, assume that a TP Monitor is being used with a DB2 Connect gateway and an zSeries® or System i database. When an application requests a connection, the concentrator will either turn an inactive agent over to serve that request, or create a new worker agent. Assume that the application requests an XA transaction. An XID is created for this transaction and the worker agent is associated with it.

    When the application's request has been serviced, it issues an xa_end() and detaches from the worker agent. The worker agent remains associated with the XID of the transaction. It can now only service requests for transactions with its associated XID.

    At this time, another application might make a request for a non-XA transaction. Even if there are no other available worker agents, the agent associated with the XID will not be made available to the second application. It is considered active. The second application will have a new worker agent created for it. When that second application completes its transaction, its worker agent is released into the available pool.

    Meanwhile, other applications requesting the transaction associated with the first agent's XID can attach and detach from that agent, which executes its dedicated XA transaction for them. Any application requesting that particular transaction will be sent to this worker agent if it is free.

    The worker agent will not be released back into the general pool until an application issues a transaction boundary call (not xa_end()). For instance, an application might end the transaction with xa_commit(), at which point the worker agent drops its association with the XID and returns to the available pool. At this point any requesting application can use it for either another XA, or a non-XA, transaction.