Disable file locking
If we use Network File System Version 3 (NFSv3) for storing transaction recovery logs, and we want to use automated peer recovery, first disable file locking.
To complete this task first configure the system to prevent system overloading and network partitioning, as described in the topic about how to choose between automated and manual transaction peer recovery. These situations can lead to the initiation of a peer recovery process for an active server.
If we do not take this precautionary step, data corruption can occur.
The following list contains some actions that we can take to prevent system overloading and network partitioning:
- Modify the core group heartbeat settings to change the amount of time after which WebSphere Application Server considers a server failed. See topic about the high availability manager.
- Ensure that the network is safe from network partitioning by, for example, installing backup network adapters.
- Modify the workload management throttling so that the server cannot be overloaded.
WAS obtains an exclusive lock on the physical recovery log files whenever it is instructed to undertake recovery processing, and releases this lock when it is instructed to pass ownership of the logs to another server. Access to a recovery log is performed only when the exclusive lock is held.
NFSv3 supports exclusive file locks, but holds them on behalf of a failed host until that host can restart. In this context, the host is the physical machine running the application server that requests the lock and it is the restart of the host, not the application server, that eventually triggers the locks to release. See the topic about how to choose between automated and manual transaction peer recovery for more information.
To provide a more appropriate failover behavior, we can either use manual failover, and configure the system as described in Configure manual peer recovery for the transaction service, or we can disable the use of exclusive file locking.
Tasks
- In the administrative console, click...
Servers > Server Types > WebSphere application servers > server > [Container Settings] Container Services > Transaction Service.
- Clear the Enable file locking check box.
- Click Apply or OK.
- Save change to the master configuration.
- Repeat the previous steps for every server in the cluster.
- Restart the servers in the cluster for the changes to take effect.
Exclusive file locking is disabled for all the servers in the cluster.
What to do next
Having taken steps to mitigate the risk to recovery log integrity when locking is disabled, we can tune the heartbeating parameters of the WAS high availability (HA) framework to change the conditions under which a server is considered failed. By considering the characteristics of applications, network, and peak workloads, determine an acceptable period of time after which the likelihood of an incorrectly diagnosed server failure is acceptably small.
A trade-off exists between reducing the risk of an incorrect diagnosis of server failure and increasing the time for automated failover and peer recovery to occur. By default, a server is considered failed after 20 heartbeats, with a 10-second frequency, are missed. These defaults are custom properties of the core group that we can modify.
Related:
Transactional high availability High availability manager How to choose between automated and manual transaction peer recovery Configure manual peer recovery for the transaction service