Concepts: Runtime
Observation & Analysis
Topics
Introduction
The observation and subsequent analysis of the runtime behavior of a software component is an important practice in the successful debugging of software. Understanding the runtime behavior of software involves two key practices:
- Observation of the software during runtime execution.
- Analysis of the captured observations.
Runtime observation and analysis techniques are themselves not dependent on testing in so far as runtime behavior can be observed and analyzed without the requirement for predefined test inputs or the use of testing techniques to stimulate the application behavior. However, testing can be successfully combined with runtime observation techniques and tools: for example, tools that automate runtime observation can be executed during test execution, improving the visibility into the runtime behavior of the component that occurs in response to the test.
Observing runtime behavior
While it is important to carefully observe all of the behavior that occurs during runtime execution of the software, there are usually significant observation points that are useful to specifically monitor. These significant observation points are often:
- Decision points at which the software logic path is about to or has just branched.
- Completion points at which an important logic path has completed, typically resulting in a state change within the software environment.
- An interface point between two separate application components.
- An interface point between the software and it's execution environment, including any interfaces to hardware components.
These observation points may also align with control points at which it may be desirable to alter either the application state or the flow of control through the logic paths. These concerns are often referred to as Points of Control and Observation (PCO).
Runtime observation excludes static observation methods as a primary approach, such as review of the static software source code or of the relationships between the software building blocks captured in visual models and so forth. Rather, it requires an executable software component and offers valuable information not available through other debugging techniques about how the developed component behaves when it runs, either in the test environment, or in the final deployment environment. The observations captured from the runtime behavior may subsequently be related to static elements to provide additional insight.
Analyzing runtime observations
Software runtime analysis is simply the practice of understanding the behavior of a software component by analyzing data collected during runtime execution of the component. During the development of the component by the Implementer, runtime observation and analysis is one aspect of the debugging activities the Implementer undertakes.
Automated tool support
Because of the potential volume of low-level information that can be captured from runtime behavior, the speed at which that information is generated, and the subsequent difficulty in understanding the potentially vast amount of information, automated tool support is a key factor in making this practice feasible. There are various approaches that can be taken to provide tool support yourself, and a wealth of tools available commercially that will save you the time, effort and cost of creating your own.
