Asynchronous request dispatcher application design considerations

Asynchronous request dispatcher application design considerations

Asynchronous request dispatcher (ARD) is not a one-size-fits-all solution to servlet programming. We must evaluate the needs of the application and the caveats of using ARD. Switching all includes to start asynchronously is not the solution for every scenario, but when used wisely, ARD can improve response time.

Asynchronous request dispatcher client-side implementation

JavaScript is dynamically written to the response output.

This JavaScript results in Ajax requests back to a server-side results provider.

Because of the asynchronous input/output (AIO) features of the channel, the Ajax request does not tie up a thread and instead is notified for completion through an include callback.

The client makes only one request at a time for the asynchronous includes because of browser limitations in the number of connections.

Original connection must be valid for the lifetime of the includes. It cannot be reused for the Ajax requests.

Comment nodes, such as following,


...are placed in the browser object model since comment nodes have no effect on the page layout.
Whenever a complete fragment exists, a response can be sent to the client and the comment node with the same ID is replaced. Requests are made until all the fragments are retrieved.
Verify applications on all supported browsers when using client-side aggregation. Object-oriented JavaScript principles are used so that applications only need avoid using the method name getDynamicDataIBMARD. Any previously specified window.onload is started before the ARD onload method.

Asynchronous request dispatcher channel results service

Requests for include data from the asynchronous JavaScript code are sent to known Uniform Resource Identifiers, URIs also known as URLs, that the ARD channel can intercept to prevent traveling through web container request handling. These URIs are unique for each server restart.

For example, /IBMARD01234567/asyncInclude.js is the URI for the JavaScript that forces the retrieval of the results, and /IBMARD01234567/IBMARDQueryStringEntries?=12000 is used to retrieve the results for the entry with ID 12000.

To prevent unauthorized results access, unique IDs are generated for the service URI and for the ARD entries. A common ID generator is shared among the session and ARD, so uniqueness is configurable through session configuration. Session IDs are considered secure, but they are not as secure as using a LTPA token.

Custom client-side aggregation

To perform our own client-side aggregation, the isUseDefaultJavascript method must return as false. The isUseDefaultJavascript method is part of the AsyncRequestDispatcherConfig method, which is set on the AsyncRequestDispatcher or for the AsyncRequestDispatcherConfigImpl.getRef method. The AsyncRequestDispatcherConfigImpl.getRef method is the global configuration object. We might want to perform our own client-side aggregation if the back button functionality is problematic. Remove the results from the generic results service to prevent memory leaks so that multiple requests with the same response results through an XMLHttpRequest fail. To facilitate proper location of position, placeholders are still written in the code as

where x is the order of the includes. The endpoint to retrieve results are retrieved from the request attribute com.ibm.websphere.webcontainer.ard.endpointURI.
When making a request to the endpoint, ARD sends as many response fragments as possible when the request is made. Therefore, the client needs to rerequest if all fragments are not initially returned. Trying to display the results directly in a browser without using an XMLHttpRequest can result in errors related to non well-formed XML. The response data is returned in the following format with a content type of text/xml:
<div id="2"><BR>Servlet 3--dispatcher3 requesting Servlet3 to sleep for 0 seconds at: 1187967704265 
<BR> Servlet 3--Okay, all done!  This should print pop up: third at: 1187967704281 </div>
For additional information about the AsyncRequestDispatcherConfig and the AsyncRequestDispatcher interfaces, review the com.ibm.websphere.webcontainer.async package in the API documentation. The generated API documentation is available in the documentation table of contents from the path Reference > APIs - Application Programming Interfaces.

Server-side aggregation

Like client-side aggregation, server-side aggregation uses the ARD channel as a results service. The ARD channel knows which asynchronous includes have occurred for certain set of buffers. Those buffers can then be searched for an include placeholder. Because of the issues of JSP buffering, the placeholder for the include might not be in the searched buffers. If this occurs, the next set of buffers must also look for any include placeholders missed in the previous set. ARD attempts to iteratively aggregate as includes return so that response content can be sent to the client as soon as possible.

Concurrency

A work manager is used to start the includes. If the number of currently requested includes is greater than the work manager maximum thread pool size and this size is not growable, it starts the work on the current thread and skips the placeholder write. Using Concurrency Utilities for Java EE enables propagation of the Java EE context of the original thread including work areas, internationalization, application profile, z/OS operation system work load management, security, transaction, and connection context.

Timer

A single timer is used for ARD and timer tasks are created for all the timeout types of ARD requests. Tasks that are registered with the timer might not run at the exact time specified because the timer runs on a single thread, therefore one timeout might have to wait for the other timeout actions to complete. The timer is used as a last resort.

Remote request dispatcher

Optionally, ARD can be used in concert with the remote request dispatcher. The remote request dispatcher runs the include on a different application server in a core group by serializing the request context into a SOAP message and using web services to call the remote server. This is useful when the expense of creating and sending a SOAP message through web services is outweighed by issuing the request locally.

Exceptions

In the case of an exception in an included servlet, the web container goes through the error page definitions mapped to exception types. So an error page defined in the deployment descriptor shows up as a portion of the aggregated page. Insert logic into the error page itself if behavior is different for an include. Because the include runs asynchronously, the top-level servlet might not still be in service, therefore the exception is not propagated back from an asynchronous include like a normal include. Other includes finish so that partial pages can be displayed.

If the ARD work manager runs out of worker threads, the include is processed like a synchronous include. This is the default setting, but the work manager can also grow such that it does not result in this condition. This change in processing is invisible to the user during processing but is noted once in the system logs as a warning message and the rest of the time in the trace logs when enabled. Other states that can trigger the include to occur synchronously are reaching the maximum percentage of expired requests over a time interval and reaching the maximum size of the results store.

Exceptions might happen outside of the scope of normal error page handling. For example, work can be rejected by the work manager. A timer can expire waiting for an include response to return. The ARD channel, acting as a generic service to retrieve the results, might receive an ID that is not valid. In these cases, there is no path to the error page handling because the context is missing, such as ServletRequest, ServletResponse, and ServletContext, for the request to work. To mitigate these issues, we can use the AsyncRequestDispatcherConfig interface to provide custom error messages. Defaults are provided and internationalized as needed.

Exceptions can also occur outside the scope of the request the custom configuration was set on, such as on the subsequent client-side XMLHttpRequests. In this case, the global configuration must be altered. This can be retrieved through com.ibm.wsspi.ard.AsyncRequestDispatcherConfigImpl.getRef().

Include start

The work manager provides a timeout for how long to wait for an include to start. Because this typically happens immediately, there is not a programmatic way to enable this. However, this is configurable in the work manager settings. By default, we do not encounter this because of the maximum thread check before scheduling the work. Work can be retried if setRetriable(true) is called on the in use AsyncRequestDispatcherConfig.

Include finish

The initiated timeout starts after the work is accepted. It can be configured through the console or programmatically through the AsyncRequestDispatcherConfig.setExecutionTimeoutOverride method; The default is 60000 ms, or one minute. In place of the include results, the message from the AsyncRequestDispatcherConfig.setExecutionTimeoutMessage is sent. If this initiated timeout is reached, but the actual include results are ready when the data can be flushed, preference is given to the actual results. Also, this does not apply to insertFragmentBlocking calls that always wait until the include is completed.

Expiration of results

Because the client-side must hold the results in a service to send for the Ajax request, we need a way to expire the results if the client goes down and never retrieves the entry. The default of a minute is sufficient for a typical request because the Ajax request would come in immediately after sending the response. The timer can be configured programmatically using the setExpirationTimeoutOverride method of AsyncRequestDispatcherConfig. The message from the getOutputRetrievalFailureMessage method of AsyncRequestDispatcherConfig is displayed when someone tries to access an entry that has expired and been removed from cache. This message is the same message sent to someone requesting a result with an ID that never existed.

Includes or fragments

Consider which operations can be done asynchronously and when they can start. Ideally, all the includes are completed when the getFragment calls are made at the beginning of the request so that the includes can have more time to complete, and upon inserting the fragments, there is less extra buffering and aggregating if they have completed. However, calling an asynchronous include is easier because it follows the same pattern as a normal request dispatcher include.

Web container

ServletContext

When doing cross-context includes, the context that is a target of the include must also have ARD enabled because the web application must have been initialized for ARD for its servlet context to have valid methods to retrieve an AsyncRequestDispatcher. The aggregation type is determined by the original context's configuration because we cannot mix aggregation types.

ServletRequest

We must clone the request for each include. Otherwise, conflicts between threads might occur. Because applications can wrap the default request objects, your wrappers must implement the com.ibm.wsspi.webcontainer.servlet.IServletRequest interface, which has one method, the public Object clone method, which creates the CloneNotSupportedException.

Unwrapping occurs until a request wrapper that implements this interface is found. Non-implementing wrappers are lost; however, a servlet filter configured for the include can rewrap the response.

Changes made to the ServletRequest are not propagated back to the top-level servlet unless transferState on the AsyncRequestDispatcherConfig is enabled and insertFragmentBlocking is called.

ServletResponse

A wrapped response that extends com.ibm.websphere.servlet.response.StoredResponse is created by ARD and sent to the includes because the response output must be retrievable beyond the lifecycle of the original response.

Internal headers set in asynchronous includes are not supported due to lifecycle restrictions unless transferState on the AsyncRequestDispatcher config is enabled and insertFragmentBlocking is called. Normal headers are not supported in a synchronous include as specified by the servlet specification.

Include filters can rewrap the new response and must flush upon completion.

ServletInputStream

An application reading parameters using getParameter is not problematic. Parsing of parameters is forced before the first asynchronous include to prevent concurrent access to the input stream.

HttpSession

Initial getSession calls that result in a Set-Cookie header must be called from the top-level servlet because it is unpredictable when the includes are started and if the headers have already been flushed. The exception is when transferState on the AsyncRequestDispatcherConfig is enabled and an insertFragmentBlocking is called. This normally creates an exception when we add the header.

Filters

If there is a filter for an include, the filter is issued on the asynchronous thread.

Nested asynchronous includes

Nested asynchronous includes are not supported because they complicate aggregation. However, an asynchronous include can have nested synchronous includes. Any attempt to perform a nested asynchronous include reverts to a synchronous include.

Transactions

Every task that is submitted to the work manager is called using its own transaction, much like container-managed transactions in typical enterprise beans. The run time starts a local transaction before starting the method. The task can start its own global transaction if this transaction is possible for the calling Java EE component.

If the task creates an exception, any local transactions are rolled back. If the method returns normally, any incomplete local transactions are completed according to the unresolved action policy configured for the bean. If the task starts its own global transaction and does not commit this global transaction, the transaction is rolled back when the method returns.

Connection management

A task that is submitted to the work manager can use the data sources and connection factories that its creating servlet obtained using java:comp resource references. However, the bean method must access those connections using a get, use, or close pattern. There is no connection caching between method calls on a task. The data sources or connection factories can be cached, but the connections must be retrieved on every method call, used, and then closed. While the task can look up connection factories using a global JNDI name, this is not recommended for the following reasons:

The JNDI name is hardcoded in the application, for example, as a property or string literal.
The connection factories are not shared because there is no way to specify a sharing scope.
Evaluate high load scenarios because asynchronous includes might increase the number of threads waiting on the connection.

Performance

Because includes are completed asynchronously, the total performance data for a request must consider the performance of the asynchronous includes. The total time of the request might previously be understood by the time for the top-level servlet to complete, but now that servlet is exiting before the includes are completed. The top-level servlet still accounts for much of the additional setup time required for each include.

Therefore, a new ARD performance metric was added to the Performance Monitoring Infrastructure to measure the time for a complete request through the ARD channel. The granularity of these metrics is at the request URI level.

Because ARD is an optional feature that must be enabled, no performance decline is seen when not using ARD. However, non-ARD applications that reside on an ARD-enabled application server would suffer from the extra layer of the ARDChannel. The channel layer does not know to which application it is going so it is either on or off for all applications in a channel chain. These are defined per virtual host.

Security

Security is not invoked on synchronous include dispatches according to the servlet specification. However, security context is passed along through Concurrency Utilities for Java EE to support programmatic usage of the isUserInRole and getUserPrincipal methods on the ServletRequest. This security context can also be propagated across to a remote request dispatch using Web Services Security.

Related:

Asynchronous request dispatcher
Developing servlet applications using asynchronous request dispatcher
http://www.ibm.com/developerworks/websphere/techjournal/0608_hesmer/0608_hesmer.html