Transactional high availability
The high availability of the transaction service enables any server in a cluster to recover the transactional work for any other server in the same cluster. This facility forms part of the overall WebSphere Application Server high availability (HA) strategy.
(ZOS) This feature is in addition to the support for peer restart and recovery, which enables us to restart on a peer system in the sysplex.
As a vital part of providing recovery for transactions, the transaction service logs information about active transactional work in the transaction recovery log. The transaction recovery log stores the information in a persistent form, which means any transactional work in progress at the time of a server failure can be resolved when the server is restarted. This activity is known as transaction recovery processing. In addition to completing outstanding transactions, this processing also ensures any locks held in the associated resource managers are released.
Peer recovery processing
The standard recovery process performed when an application server restarts is for the server to retrieve and process the logged transaction information, recover transactional work and complete indoubt transactions. Completion of the transactional work (and hence the release of any database locks held by the transactions) takes place after the server successfully restarts and processes its transaction logs. If the server is slow to recover or requires manual intervention, the transactional work cannot be completed and access to associated databases is disrupted.
To minimize such disruption to transactional work and the associated databases, WAS provides a high availability strategy known as transaction peer recovery.
Peer recovery is provided within a server cluster. A peer server (another cluster member) can process the recovery logs of a failed server while the peer continues to manage its own transactional workload. We do not have to wait for the failed server to restart, or start a new application server specifically to recover the failed server.
Figure 1. Peer recovery
The peer recovery process is the logical equivalent to restarting the failed server, but does not constitute a complete restart of the failed server within the peer server. The peer recovery process provides an opportunity to complete outstanding work; it cannot start new work beyond recovery processing. No forward processing is possible for the failed server.
Peer recovery moves the high availability requirements away from individual servers and onto the server cluster. After such failures, the management system of the cluster dispatches new work onto the remaining servers; the only difference is the potential drop in overall system throughput. If a server fails, all required is to complete work that was active on the failed server and redirect requests to an alternate server.
By default, peer recovery is disabled until we enable failover of transaction log recovery in the cluster configuration, and restart the cluster members. After enabling transaction log recovery, WAS supports two styles for the initiation of transaction peer recovery: automated and manual. You determine which style is more appropriate, based on the deployment, and specify that style by configuring the appropriate high availability policy. This high availability policy is referred to elsewhere in these topics as the policy for the transaction service.
- Automated peer recovery
- This style is the default for peer recovery initiation. If an application server fails, WAS automatically selects a server to undertake peer recovery processing on its behalf, and passes recovery back to the failed server when it restarts. To use this model, enable transaction log recovery and configure the recovery log location for each cluster member.
- Manual peer recovery
- We must explicitly configure this style of peer recovery. If an application server fails, we use the administrative console to select a server to perform recovery processing on its behalf.
In a HA environment, configure the compensation logs as well as the transaction logs. For each server in the cluster, use the compensation service settings to configure a unique compensation log location, and ensure that all cluster members can access those compensation logs.
Peer recovery example
The following diagrams illustrate the peer recovery process that takes place if a single server fails. Figure 2 shows three stable servers running in a WebSphere Application Server cluster. The workload is balanced between these servers, which results in locks held by the back-end database on behalf of each server.
Figure 2. Server cluster up and running, just before server failure
Figure 3 shows the state of the system after server 1 fails without clearing locks from the database. Servers 2 and 3 can run their existing transactions to completion and release existing locks in the back-end database, but further access might be impaired because of the locks still held on behalf of server 1. In practice, some level of access by servers 2 and 3 is still possible, assuming appropriately configured lock granularity, but for this example assume that servers 2 and 3 attempt to access locked records and become blocked.
Figure 3. Server 1 fails. Servers 2 and 3 become blocked as a result
Figure 4 shows a peer recovery process for server 1 running inside server 3. The transaction service portion of the recovery process retrieves the information stored by server 1, and uses that information to complete any indoubt transactions. In this figure, the peer recovery process is partially complete as some locks are still held by the database on behalf of server 1.
Figure 4. Peer recovery process started in server 3
Figure 5 shows the state of the server cluster when the peer recovery process is complete. The system is in a stable state with just two servers, between which the workload is balanced. Server 1 can be restarted, and will have no recovery processing of its own to perform.
Figure 5. Server cluster stable again with just two servers: server 2 and server 3
Subtopics
- Deployment for transactional high availability
- High availability policies for the transaction service
Related: