Stopping a cluster member containing active sessions
When using sessions, the impact of stopping or killing a cluster member is that session information in that cluster member is lost. This means that persistent or replicated sessions need to be used to allow for the failover of the session to another cluster member. Follow this procedure to discover what happens:
Cycle through to make sure that all the cluster members are available. There is no need to repeat the changes to the workload management policy.
Note: If you use the same browser for the session management test and the workload management test, you will find that all your BeenThere requests return to the same server. This is because the cookie for your session management test is still valid. |
http://<yourWebServer>/hitcount
Click the Increment button a few times and remember the session count you have reached.
http://<yourWebServer>/wlm/beenthere
http://<yourWebServer>/hitcount
What is happening?
Upon choosing to increment the session counter, WebSphere adds a cookie to your browser. Within this cookie is the session identifier and a clone ID of where this session is stored.
At the end of each request, the session is written to the database or distributed between all appservers using memory-to-memory replication. When the cluster member running the session is stopped, the plug-in chooses another cluster member for the request to go to instead. In the case of using database persistence, this cluster member, finding that it does not have the session cached locally, searches the database. In the case of using memory-to-memory replication, the application server will find the session in its own cache. So in both cases, the other application server finds the session and uses that one instead of creating a new one. This is why the session counter is incremented, not reset.
The WebSphere session manager changes the cookie and appends the cluster clone ID of the new cluster member to it. The cookie now has two cluster clone IDs on it, the original (stopped) cluster member and the failover cluster member.
The plug-in now tries to connect to the stopped cluster member first, and finding it marked as down, will try the failover cluster member instead.
Starting up the stopped cluster member means that the plug-in now returns the session requests to the original cluster member after the retry interval has passed.
This can be seen in the trace listing in Example 5-19.
Example 5-19 Plug-in trace with session failover
...
### We have a session with clone id 'v544d031' for PluginMember1
TRACE: ws_server: serverCreate: Creating the server object
TRACE: ws_server: serverSetName: Setting name was1node_PluginMember1
TRACE: ws_server: serverSetCloneID: Setting clone id v544d031
### We have a session with clone id 'v544d0o0' for PluginMember2
TRACE: ws_server: serverCreate: Creating the server object
TRACE: ws_server: serverSetName: Setting name was2Node_PluginMember2
TRACE: ws_server: serverSetCloneID: Setting clone id v544d0o0
...
### When a request comes, the plugin dispatches it to PluginMember1
TRACE: ws_common: websphereHandleSessionAffinity: Checking the cookie affinity: JSESSIONID
TRACE: lib_htrequest: htrequestGetCookieValue: Looking for cookie: 'JSESSIONID'
TRACE: lib_htrequest: htrequestGetCookieValue: name='JSESSIONID', value='0001IWuUT_zhR-gFYB-pOAk75Q5:v544d031'
TRACE: ws_common: websphereParseCloneID: Parsing clone ids from '0001IWuUT_zhR-gFYB-pOAk75Q5:v544d031'
TRACE: ws_common: websphereParseCloneID: Adding clone id 'v544d031'
TRACE: ws_common: websphereParseCloneID: Returning list of clone ids
TRACE: ws_server_group: serverGroupFindClone: Looking for clone
TRACE: ws_server_group: serverGroupGetFirstPrimaryServer: getting the first primary server
TRACE: ws_server_group: serverGroupFindClone: Comparing curCloneID 'v544d031' to server clone id 'v544d031'
TRACE: ws_server_group: serverGroupFindClone: Match for clone 'was1node_PluginMember1'
...
### When PluginMember1 goes down a connection to the next cluster member in the the primary server list will be attempted. Note that the plugin will first try to connect to PluginMember1, when it can't connect to it, it will restart the process of checking for cluster members starting at the first cluster member in the primary server list.
.
TRACE: ws_transport: transportStreamDequeue: Checking for existing stream from the queue
ERROR: ws_common: websphereGetStream: Failed to connect to app server on host 'app1.itso.ibm.com', OS err=10061
ERROR: ws_common: websphereExecute: Failed to create the stream
ERROR: ws_server: serverSetFailoverStatus: Marking was1node_PluginMember1 down
STATS: ws_server: serverSetFailoverStatus: Server was1node_PluginMember1 : pendingConnections 0 failedConnections 1 affinityConnections 2 totalConnections 2.
ERROR: ws_common: websphereHandleRequest: Failed to execute the transaction to 'was1node_PluginMember1'on host 'app1.itso.ibm.com'; will try another one
...
### Then it iterates again decending down the primary server list.
TRACE: lib_htrequest: htrequestGetCookieValue: Looking for cookie: 'JSESSIONID'
TRACE: lib_htrequest: htrequestGetCookieValue: name='JSESSIONID', value='0001IWuUT_zhR-gFYB-pOAk75Q5:v544d031'
TRACE: ws_common: websphereParseCloneID: Parsing clone ids from '0001IWuUT_zhR-gFYB-pOAk75Q5:v544d031'
TRACE: ws_common: websphereParseCloneID: Adding clone id 'v544d031'
TRACE: ws_common: websphereParseCloneID: Returning list of clone ids
TRACE: ws_server_group: serverGroupFindClone: Looking for clone
TRACE: ws_server_group: serverGroupGetFirstPrimaryServer: getting the first primary server
TRACE: ws_server_group: serverGroupFindClone: Comparing curCloneID 'v544d031' to server clone id 'v544d031'
TRACE: ws_server_group: serverGroupFindClone: Match for clone 'was1node_PluginMember1'
TRACE: ws_server: serverHasReachedMaxConnections: currentConnectionsCount 0, maxConnectionsCount -1.
STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of was1node_PluginMember1, ignoreWeights 1, markedDown 1, retryNow 0, wlbAllows 0 reachedMaxConnectionsLimit 0
TRACE: ws_server: serverHasReachedMaxConnections: currentConnectionsCount 0, maxConnectionsCount -1.
TRACE: ws_server_group: serverGroupCheckServerStatus: Server was1node_PluginMember1 is marked down; retry in 60
TRACE: ws_server_group: serverGroupGetNextPrimaryServer: getting the next primary server
TRACE: ws_server_group: serverGroupFindClone: Comparing curCloneID 'v544d031' to server clone id 'v544d0o0'
TRACE: ws_server_group: serverGroupGetNextPrimaryServer: getting the next primary server
TRACE: ws_server_group: serverGroupGetNextUpPrimaryServer: Getting the next up primary server
TRACE: ws_server_group: serverGroupGetNextPrimaryServer: getting the next primary server
TRACE: ws_server: serverHasReachedMaxConnections: currentConnectionsCount 0, maxConnectionsCount -1.
STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of was2Node_PluginMember2, ignoreWeights 1, markedDown 0, retryNow 0, wlbAllows 0 reachedMaxConnectionsLimit 0
TRACE: ws_server: serverHasReachedMaxConnections: currentConnectionsCount 0, maxConnectionsCount -1.
TRACE: ws_server_group: serverGroupIncrementConnectionCount: Server was2Node_PluginMember2 picked, pendingConnectionCount 1 totalConnectionsCount 2.
TRACE: ws_server_group: serverGroupFindClone: Returning next up server was2Node_PluginMember2
TRACE: ws_common: websphereHandleSessionAffinity: Setting server to was2Node_PluginMember2
...
### On the the response back from PluginMember2 a new clone id is appended to the session cookie
TRACE: ws_common: websphereExecute: Wrote the request; reading the response
TRACE: lib_htresponse: htresponseRead: Reading the response: b2c384
TRACE: HTTP/1.1 200 OK
TRACE: Server: WAS/5.1
TRACE: Cache-Control: no-cache
TRACE: Set-Cookie: JSESSIONID=0002IWuUT_zhR-gFYB-pOAk75Q5:v544d031:v544d0o0;Path=/
...
### On the the next request we now have a second clone id on the session cookie.
TRACE: ws_common: websphereHandleSessionAffinity: Checking the cookie affinity: JSESSIONID
TRACE: lib_htrequest: htrequestGetCookieValue: Looking for cookie: 'JSESSIONID'
TRACE: lib_htrequest: htrequestGetCookieValue: name='JSESSIONID', value='0002IWuUT_zhR-gFYB-pOAk75Q5:v544d031:v544d0o0'
TRACE: ws_common: websphereParseCloneID: Parsing clone ids from '0002IWuUT_zhR-gFYB-pOAk75Q5:v544d031:v544d0o0'
TRACE: ws_common: websphereParseCloneID: Adding clone id 'v544d031'
TRACE: ws_common: websphereParseCloneID: Adding clone id 'v544d0o0'
TRACE: ws_common: websphereParseCloneID: Returning list of clone ids
TRACE: ws_server_group: serverGroupFindClone: Looking for clone
TRACE: ws_server_group: serverGroupGetFirstPrimaryServer: getting the first primary server
TRACE: ws_server_group: serverGroupFindClone: Comparing curCloneID 'v544d031' to server clone id 'v544d031'
TRACE: ws_server_group: serverGroupFindClone: Match for clone 'was1node_PluginMember1'
TRACE: ws_server: serverHasReachedMaxConnections: currentConnectionsCount 0, maxConnectionsCount -1.
STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of was1node_PluginMember1, ignoreWeights 1, markedDown 1, retryNow 0, wlbAllows 0 reachedMaxConnectionsLimit 0
TRACE: ws_server: serverHasReachedMaxConnections: currentConnectionsCount 0, maxConnectionsCount -1.
TRACE: ws_server_group: serverGroupCheckServerStatus: Server was1node_PluginMember1 is marked down; retry in 55
TRACE: ws_server_group: serverGroupGetNextPrimaryServer: getting the next primary server
TRACE: ws_server_group: serverGroupFindClone: Comparing curCloneID 'v544d031' to server clone id 'v544d0o0'
TRACE: ws_server_group: serverGroupGetNextPrimaryServer: getting the next primary server
TRACE: ws_server_group: serverGroupGetFirstPrimaryServer: getting the first primary server
TRACE: ws_server_group: serverGroupFindClone: Comparing curCloneID 'v544d0o0' to server clone id 'v544d031'
TRACE: ws_server_group: serverGroupGetNextPrimaryServer: getting the next primary server
TRACE: ws_server_group: serverGroupFindClone: Comparing curCloneID 'v544d0o0' to server clone id 'v544d0o0'
TRACE: ws_server_group: serverGroupFindClone: Match for clone 'was2Node_PluginMember2'
...
### When PluginMember1 comes back up after the retry interval our session requests go back to it
TRACE: ws_common: websphereHandleSessionAffinity: Checking the cookie affinity: JSESSIONID
TRACE: lib_htrequest: htrequestGetCookieValue: Looking for cookie: 'JSESSIONID'
TRACE: lib_htrequest: htrequestGetCookieValue: name='JSESSIONID', value='0002IWuUT_zhR-gFYB-pOAk75Q5:v544d031:v544d0o0'
TRACE: ws_common: websphereParseCloneID: Parsing clone ids from '0002IWuUT_zhR-gFYB-pOAk75Q5:v544d031:v544d0o0'
TRACE: ws_common: websphereParseCloneID: Adding clone id 'v544d031'
TRACE: ws_common: websphereParseCloneID: Adding clone id 'v544d0o0'
TRACE: ws_common: websphereParseCloneID: Returning list of clone ids
TRACE: ws_server_group: serverGroupFindClone: Looking for clone
TRACE: ws_server_group: serverGroupGetFirstPrimaryServer: getting the first primary server
TRACE: ws_server_group: serverGroupFindClone: Comparing curCloneID 'v544d031' to server clone id 'v544d031'
TRACE: ws_server_group: serverGroupFindClone: Match for clone 'was1node_PluginMember1'
TRACE: ws_server: serverHasReachedMaxConnections: currentConnectionsCount 0, maxConnectionsCount -1.
STATS: ws_server_group: serverGroupCheckServerStatus: Checking status of was1node_PluginMember1, ignoreWeights 1, markedDown 1, retryNow 1, wlbAllows 0 reachedMaxConnectionsLimit 0
TRACE: ws_server: serverHasReachedMaxConnections: currentConnectionsCount 0, maxConnectionsCount -1.
TRACE: ws_server_group: serverGroupCheckServerStatus: Time to retry server was1node_PluginMember1
TRACE: ws_server_group: serverGroupIncrementConnectionCount: Server was1node_PluginMember1 picked, pendingConnectionCount 1 totalConnectionsCount 3.
TRACE: ws_common: websphereHandleSessionAffinity: Setting server to was1node_PluginMember1
TRACE: ws_server_group: assureWeightsValid: group PluginCluster
TRACE: ws_server_group: serverGroupGetFirstPrimaryServer: getting the first primary server
TRACE: ws_server_group: weights_need_reset: was1node_PluginMember1: 4 max, 0 cur.
TRACE: ws_server_group: serverGroupGetNextPrimaryServer: getting the next primary server
TRACE: ws_server_group: weights_need_reset: was2Node_PluginMember2: 1 max, 0 cur.
TRACE: ws_server_group: serverGroupGetNextPrimaryServer: getting the next primary server
TRACE: ws_server_group: weights_need_reset: Time to reset the weights
TRACE: ws_server_group: serverGroupGetFirstServer: getting the first server
TRACE: ws_server_group: serverGroupGetNextServer: getting the next server
TRACE: ws_server_group: serverGroupGetNextServer: getting the next server
TRACE: ws_server_group: assureWeightsValid: max multiplication factor 1.
TRACE: ws_server_group: serverGroupGetFirstServer: getting the first server
TRACE: ws_server_group: assureWeightsValid: Server was2Node_PluginMember2: 1 max, 1 cur.
TRACE: ws_server_group: serverGroupGetNextServer: getting the next server
TRACE: ws_server_group: serverGroupGetNextServer: getting the next server
TRACE: ws_server_group: lockedServerGroupUseServer: Server was1node_PluginMember1 picked, weight -1.
TRACE: ws_common: websphereFindTransport: Finding the transport
...
WebSphere is a trademark of the IBM Corporation in the United States, other countries, or both.
IBM is a trademark of the IBM Corporation in the United States, other countries, or both.