Test run results

Running a component test produces test run results at each level of the test hierarchy, from the test suite to the individual test. Test run results are displayed in the Test Navigator under the Run folder and in the test data comparator.

 

Run folder

The Test Navigator lists test run results in a tree view, as shown in the following screen capture:

Within a test case run, one data set run is produced for each data set specified in the test data table.

Within each test data set, the Test Navigator lists an individual test for each possible combination of input values specified in that data set. If the data set specifies a single input value for each parameter, then one individual test run is generated. If the data set specifies one or several sets or ranges, then one individual test run is produced for each combination of input values in the data set. For example, if a data set specifies two input parameters, where one parameter is specified as a set of two values and the other is specified as a range of three values, then a total of 6 individual tests will be generated.

 

Verdict of a test run

The test verdict tells you whether a test has succeeded or failed. The verdict of any level of component test is displayed in the Test Navigator as an icon overlay. The verdict status can be any of the following:

Icon	Status	Description
	Passed	The actual returned values matched the expected values specified in the test data table.
	Failed	The actual returned values did not match the expected values.
	Inconclusive	The test verdict could not be evaluated; for example, it may be that no validation action was performed or the test was interrupted.
	Error	An error occurred during test execution. This may be due to a problem with the test behavior script or the component-under-test. You can also manually insert an error action in the test behavior script.

In the Test Navigator, each level of the test hierarchy inherits the worst verdict from its children. For example, if a test case contains several data sets of which only one is Failed and all others are Passed, then the verdict of the test case is Failed.

 

Test run data

Once you know the verdict of a test run, you can investigate a problem by looking at the actual data used to determine the verdict. The test data is displayed in the test data comparator, as shown in the following screen capture:

For each data set and individual test, the test data comparator displays the following information:

• In: The input value of the individual test (single value or element of a set or array).

• Expected: The acceptable output expected from the component-under-test.

• Actual value: The value actually returned by the component-under-test during the test.

 

Related tasks

Browsing test results using the Test Navigator
Viewing component test results
Generating a component test report

Related reference
Test Navigator reference