Test strategies

Test strategies

Defining a test strategy helps you focus test resources on the right portions of code at the right time.

When developing medium to large applications, the first question that comes up is "Where do I start?", and then "What next?". Guidance is provided in the form of metrics that help you analyze the complexity and reliability of the methods, classes, and components-under-test (CUT) and decide your strategy.

Component testing, as opposed to functional testing, concentrates on individual software components: classes or methods as well as Enterprise JavaBeans (EJB) and Web service components. The point of component testing is to isolate each component in order to ensure that it functions properly. If necessary, you can use stubs to replace other components, thus conditioning the test results only to the behavior of the component-under-test. Once a reliable test suite has been created, you can rerun it on a regular basis to ensure non-regression of the CUT.

You can concentrate your component tests on the following areas:
Multiple classes or methods integrated as a subsystem.
A single class
A single method
A single Java interface, abstract class, or superclass
EJBs
Web services

Subsystem-level testing

The scope of subsystem-level testing (also called integration testing) is to test a set of integrated classes that must cooperate to provide some service. In a multi-tier environment, a subsystem might be one of the tiers. The focus of subsystem-level testing is to verify the interfaces between the component-under-test and other subsystems. These tests exercise the interactions between the different objects of the subsystem.

Although subsystem-level testing should play a significant role in your overall test effort, be sure to supplement subsystem-level testing with method-level and class-level testing. Focusing only on subsystem-level testing would provide superficial test coverage because it does not give you enough control over the individual classes.

Two common types of integration testing are often referred to as top down and bottom up.
Top-down testing starts at the top of the program hierarchy and travels down its branches. This can be done in either from the shortest path down to the deepest level, or across the hierarchy, before proceeding to the next level. The main advantage of this type of testing is that the global architecture of the subsystem can be seen and tested early. The main disadvantage is the need to use stubs until the lower-level components are written, which may not necessarily provide reliable test results for the top-level components.
Bottom-up testing typically starts by testing the lowest-level components first on an individual basis and then in clusters. This ensures that each component is fully tested before it is used by its calling component. This method has the advantage of detecting errors in critical components earlier in the development process. The main disadvantage is that most or many components must be built before a working program can be presented. Object-oriented languages lend themselves well to bottom-up testing.
A third option is to closely follow the design strategy. This means that each component is tested as it is built. Test definition is part of the design of the component. As part of this strategy, you can take either a top-down or bottom-up approach.

Class-level testing
Class-level testing exercises the interactions between the methods in a class or a small cluster of classes. The focus is to verify that the class supports all of the use cases required by its consumers and is robust enough to handle unexpected sequences of methods. For the following reasons, class-level testing is generally considered the most effective way to test object-oriented software:
Testing at the class level is extremely effective in terms of code coverage. With just a few tests, you can reach 60% to 70% code coverage.
Class-level testing reflects the services that the class provides to its consumers. The tests are easy to write and can be used as documentation for the class.
Class-level testing is the optimal level of testing for finding bugs early in the design process.

Method-level testing

Method-level testing exercises the different conditions defined in the method code in isolation from any other methods. The focus is generally on ensuring that the method correctly processes all of its possible inputs. In many cases, testing a series of methods in isolation provides incomplete test coverage, and some methods, such as private methods, cannot be tested in isolation. (Private methods can have a major impact on the state of an object and can change the behavior of other method invocations.)

Interactions between the methods of a class are a common source of problems, and the best way to discover these problems is to exercise the methods in various scenarios. Therefore, you should always combine method-level testing with class or subsystem-level testing, or both. In some cases, such as testing stateless classes, you can focus exclusively on method-level testing, but in other cases you might decide to avoid method-level testing and focus only on class-level testing.

The key to successful method-level testing is determining which methods to actually test. The following table provides answers to some frequently asked questions about method-level testing:

Question Answer
What if the method is inherited from a superclass? If the method is inherited from a superclass and has already been tested as part of the superclass, you do not need to do method-level testing of this method in the context of the inherited object. You should still use this method in the context of class-level testing of the inherited object.
What if the method overrides the method of a parent class? If the method has been tested in the context of a parent class, you need to retest the overriding method.
What if the method is overloaded or overridden? A method is overloaded when multiple methods in the same class have the same name but different parameters. A method is overridden when it is declared in a class and it's implementation is defined or altered in a subclass. Overridden methods allow an object type to be polymorphic in nature. In either case, be sure to test each implementation of these class methods individually.

Interface, abstract class, and superclass testing

You can use abstract tests to test Java interfaces, abstract classes, and superclasses. Although abstract classes cannot actually be run on their own, you can apply an abstract test to any class that implements an interface, realizes an abstract class, or inherits from a superclass.

EJB testing

Testing an Enterprise Java Bean (EJB) generally consists of verifying the EJB's business logic and the success or failure of its lifecycle methods. This is done by testing the methods defined in the bean class by calling the methods through their various interfaces. Tests can reside in the same container as the EJB or outside the application server.

The best way to test an EJB is to deploy it on an application server and run it in the context of its container. If the EJB has a local interface, test it from within the same application server, for example, using another EJB. If the EJB has a remote interface, you can test it with a Java client running outside the application server (at the expense of increased network traffic). Either way, you will need to stub out components called by the EJB-under-test so that you can isolate the behavior of the EJB.

You can test session beans and entity beans built according to either the Enterprise JavaBeans 1.1 or 2.0 specification. You can test both stateful and stateless session beans and entity beans with either container-managed or bean-managed persistence (CMP or BMP). To be tested, all beans must have either a remote or a local interface. Testing of message-driven beans is not currently supported.

Several test patterns simplify the process of testing EJBs.

Web service testing

When testing distributed components such as Web services, the approach is basically the same as for testing any other component. The goal of the test is to stimulate a Web service running on an application server and to verify that server responses match the expected return values.

The Web service interface, as described in a Web service description document (WSDL), is a required input to automatically generate a component test for a Web service.

Related concepts
Java subsystems
State-based testing techniques

Related tasks
Testing Java methods
Testing Java classes

Question	Answer
What if the method is inherited from a superclass?	If the method is inherited from a superclass and has already been tested as part of the superclass, you do not need to do method-level testing of this method in the context of the inherited object. You should still use this method in the context of class-level testing of the inherited object.
What if the method overrides the method of a parent class?	If the method has been tested in the context of a parent class, you need to retest the overriding method.
What if the method is overloaded or overridden?	A method is overloaded when multiple methods in the same class have the same name but different parameters. A method is overridden when it is declared in a class and it's implementation is defined or altered in a subclass. Overridden methods allow an object type to be polymorphic in nature. In either case, be sure to test each implementation of these class methods individually.