IBM Tivoli Monitoring > Version 6.3 Fix Pack 2 > Installation Guides > Installation Guide > Set up event forwarding to Netcool/OMNIbus > Architecture overview
IBM Tivoli Monitoring, Version 6.3 Fix Pack 2
Event behavior
How events are handled depends on several criteria, including the architecture type (unidirectional versus bidirectional), and the event type (pure versus sampled).
Table 1 describes the behavior for events sent from the Hub monitoring server to Netcool/OMNIbus. Table 2 describes the behavior for events sent directly from the monitoring agents to Netcool/OMNIbus.
Behavior of events originating from a hub monitoring server
Action Event type Unidirectional behavior Bidirectional behavior Situation condition becomes true. Pure and sampled events. Summary: A new event is opened in the hub monitoring server and in the Netcool/OMNIbus ObjectServer if it does not deduplicate an existing event.
Details: The hub monitoring server opens a new situation event and sends an event with the Open status to Netcool/OMNIbus ObjectServer using flows 1 to 4 as shown in Figure 1 and a new event is opened in Netcool/OMNIbus if it does not deduplicate an existing event.
The hub monitoring server sends an event with open status to Netcool/OMNIbus each time a condition is true for a pure situation. For sampled situations, an event with open status is sent when the situation condition transitions from not true to true. Until the sampled situation condition becomes false, another event is not sent unless the status of the event is changed, for example to acknowledged.
Same as unidirectional behavior. Sampled event situation condition is no longer true or a pure sampled situation UNTIL modifier condition becomes true.
Pure and sampled events. Summary: Event is closed in the hub monitoring server and cleared in the Netcool/OMNIbus ObjectServer.
Details: After the event is closed in the hub monitoring server, a closed status update event is sent to Netcool/OMNIbus ObjectServer by using flows 1 to 4 as shown in Figure 1. When the IBM Tivoli Monitoring triggers process the status update event they clear the event in the Netcool/OMNIbus ObjectServer.
For more information about creating a situation UNTIL modifier and details on when sampled events are closed if they have an UNTIL modifier, see the IBM Tivoli Monitoring Tivoli Enterprise Portal User's Guide.
Same as unidirectional behavior. Event acknowledged using the Netcool/OMNIbus Event List UI. Pure and sampled events. Summary: Event status is changed to acknowledged in the Netcool/OMNIbus ObjectServer. However, the event status is not updated in the hub monitoring server and Tivoli Enterprise Portal.
Summary: The event status is changed to acknowledged in the Netcool/OMNIbus ObjectServer, the hub monitoring server, and Tivoli Enterprise Portal.
Details: After the event status is changed to acknowledged in the Netcool/OMNIbus ObjectServer, the hub monitoring server is notified about the status change by flows 5 through 8 in Figure 1. In flow 7, the IBM Tivoli Monitoring Situation Update Forwarder sends a CT_Acknowledge SOAP request to the hub monitoring server. The hub monitoring server changes the event status to Acknowledged when it processes the SOAP request and sends a status update event back to OMNIbus using flows 9 through 11.
Event cleared or deleted using the Netcool/OMNIbus Event List UI. Pure events. Summary: The pure event is cleared or deleted in the Netcool/OMNIbus ObjectServer. However, the event remains open in the hub monitoring server and Tivoli Enterprise Portal until the UNTIL modifier condition of the situation becomes true.
Summary: The pure event is cleared or deleted in the Netcool/OMNIbus ObjectServer and closed in the hub monitoring server and Tivoli Enterprise Portal.
Details: After the event is cleared or deleted in the Netcool/OMNIbus ObjectServer, the hub monitoring server is notified about the status change by flows 5 through 8 in Figure 1. In flow 7, the IBM Tivoli Monitoring Situation Update Forwarder sends a CT_Reset SOAP request to the hub monitoring server. The hub monitoring server closes the event when it processes the SOAP request and sends a status update event back to OMNIbus using flows 9 through 11.
Event cleared or deleted using the Netcool/OMNIbus Event List UI. Sampled events. Summary: The sampled event is cleared or deleted in the Netcool/OMNIbus ObjectServer. However, the event remains open in the hub monitoring server and Tivoli Enterprise Portal until the situation condition is no longer true. No further status updates are sent to Netcool/OMNIbus until the situation condition becomes false and then true again. Therefore, the Netcool/OMNIbus operator is not notified that the event condition has not been resolved.
Summary: The sampled event is cleared or deleted in the Netcool/OMNIbus ObjectServer but its status is changed to Acknowledged in the hub monitoring server and Tivoli Enterprise Portal for a specified time. If the situation condition is still true after the specified time, a status update event is sent to Netcool/OMNIbus and an event is opened. This status update notifies the Netcool/OMNIbus operator that the event condition is not resolved.
Details: When the sampled event is cleared or deleted, the event data is cached by the ObjectServer in an IBM Tivoli Monitoring table. Then the hub monitoring server is notified of the status change by flows 5 through 8 in Figure 1. In flow 7, the IBM Tivoli Monitoring Situation Update Forwarder sends a CT_Acknowledge SOAP request with a configurable timeout to the hub monitoring server. The hub monitoring server changes the event status to Acknowledged and starts an expiration timer. A status update event is sent back to OMNIbus by using flows 9 through 11. The events are marked as Acknowledged in the hub monitoring server and Tivoli Enterprise Portal because a sampled event cannot be closed unless the situation condition is no longer true. By leaving the situation as Acknowledged, Netcool/OMNIbus is notified if the situation condition is still true after the timeout expires.If the situation condition is still true when the timeout expires, the hub monitoring server sends an Acknowledgement Expired status update event to Netcool/OMNIbus ObjectServer by using flows 1 to 4 as shown in Figure 1. If the event has already been removed from the Netcool/OMNIbus ObjectServer alerts.status table, a new event is opened in the ObjectServer.
Event cleared or deleted using the Netcool/OMNIbus Event List UI (continued) Details: (continued) Because status update events contain only base ITM EIF slots and no agent-specific slots, the event is re-opened using data that was cached when the event was cleared or deleted from the Netcool/OMNIbus Event List UI. However, if the event is still in the Netcool/OMNIbus ObjectServer alerts.status table when the status update event is processed, the event is deduplicated by the ITM triggers. The event is then reopened and contains event attribute settings from the original event. Event deacknowledged using the Netcool/OMNIbus Event List UI. Pure and sampled events. Summary: The event status is changed to deacknowledged in the Netcool/OMNIbus ObjectServer. However, the event status is not updated in the hub monitoring server and Tivoli Enterprise Portal.
Summary: The event status is changed to deacknowledged in the Netcool/OMNIbus ObjectServer and to resurfaced in the hub monitoring server and Tivoli Enterprise Portal.
Details: After the event status is changed to deacknowledged in the Netcool/OMNIbus ObjectServer, the hub monitoring server is notified about the status change by flows 5 through 8 in Figure 1. In flow 7, the IBM Tivoli Monitoring Situation Update Forwarder sends a CT_Resurface request to the hub monitoring server. The hub monitoring server changes the event status to Resurfaced when it processes the SOAP request and sends a status update event back to OMNIbus using flows 9 through 11.
Event acknowledged without a timeout using the Tivoli Enterprise Portal Situation Event Console. Pure and sampled events. Tivoli Enterprise Portal operators should not change the event status when the unidirectional architecture is being used. Summary: The event status is changed to acknowledged in both the hub monitoring server and Netcool/OMNIbus ObectServer.
Details: After the event status is changed to acknowledged in the hub monitoring server, an Acknowledged status update event is sent to Netcool/OMNIbus using flows 1 to 4 as shown in Figure 1 and the event status is changed to Acknowledged in the Netcool/OMNIbus ObjectServer and Netcool/OMNIbus Event List UI.
Event acknowledged with timeout using the Tivoli Enterprise Portal Situation Event Console. Pure and sampled events. Tivoli Enterprise Portal operators should not change the event status when the unidirectional architecture is being used. Summary: The event status is changed to acknowledged in both the hub monitoring server and Netcool/OMNIbus ObjectServer and you can configure how Netcool/OMNIbus handles the timeout notification event. See the detailed description for more information.
Details: After the event status is changed to acknowledged in the hub monitoring server, an Acknowledged status update event is sent to Netcool/OMNIbus using flows 1 to 4 as shown in Figure 1 and the event status is changed to Acknowledged in the Netcool/OMNIbus ObjectServer and Netcool/OMNIbus Event List UI. When the timeout expires and the situation event is still open in the hub monitoring server, the hub monitoring server sets the event status to Acknowledgement Expired and sends an Acknowledgement Expired status update event to Netcool/OMNIbus using flows 1 to 4 as shown in Figure 1.
By default, the IBM Tivoli Monitoring triggers reject the Acknowledgement Expired status update event and use flows 5 to 8 to send a request to the hub monitoring server to set the event status to Acknowledged. (In flow 7, the Situation Update Forwarder sends a CT_Acknowledge request to the hub monitoring server.) The hub monitoring server sets the event status to Acknowledged when it processes the SOAP request and sends a status update event back to OMNIbus using flows 9 through 11.
You can override the default IBM Tivoli Monitoring trigger behavior by setting the sit_ack_expired_def_action variable to ACCEPT using the procedure described in Customize how the IBM Tivoli Monitoring OMNIbus triggers handle event status updates from the monitoring servers. If you set the variable to ACCEPT, the IBM Tivoli Monitoring triggers deacknowledge the event in the Netcool/OMNIbus ObjectServer but the event still has the acknowledgment expired status in the hub monitoring server.
Event deacknowledged using the Tivoli Enterprise Portal Situation Event Console.
Pure and sampled events. Tivoli Enterprise Portal operators should not change the event status when the unidirectional architecture is being used. Summary: Behavior is configurable. See the detailed description for the two types of behaviors that are supported.
Details: After the event status is changed to Resurfaced in the hub monitoring server, a Resurfaced status update event is sent to Netcool/OMNIbus using flows 1 to 4 as shown in Figure 1. By default the IBM Tivoli Monitoring triggers accept the Resurfaced status update event and deacknowledge the event in the Netcool/OMNIbus ObjectServer.
You can override the default trigger behavior by setting the sit_resurface_def_action variable to REJECT using the procedure described in Customize how the IBM Tivoli Monitoring OMNIbus triggers handle event status updates from the monitoring servers. If you set the variable to REJECT then the IBM Tivoli Monitoring triggers use flows 5 to 8 in Figure 1 to send a CT_ACKNOWLEDGE SOAP request to the hub monitoring server. The hub monitoring server sets the event status to Acknowledged when it processes the SOAP request and sends a status update event back to OMNIbus using flows 9 through 11.
Situation is stopped. Situations are stopped if an operator initiates the situation stop action from the Tivoli Enterprise Portal or modifies the situation definition. However, a situation is not stopped if its distribution list is modified.
Pure events. Summary: The hub monitoring server closes all pure events for the situation. These same events are cleared in the Netcool/OMNIbus ObjectServer unless you configure a different behavior in the IBM Tivoli Monitoring triggers.
Details: As the hub monitoring server closes all pure events for the situation, it sends a situation stop event to Netcool/OMNIbus for each remote monitoring server that was monitoring the situation. The situation stop event specifies the remote monitoring server in the situation_thrunode EIF slot. (This slot is mapped to the ITMThruNodeOMNIbus attribute by the Tivoli Netcool/OMNIbus EIF Probe.)
When the IBM Tivoli Monitoring triggers in Netcool/OMNIbus process a situation stop event, they clear all events for the situation that are detected by the remote monitoring server specified by the ITMThruNode OMNIbus attribute. However, you can configure the IBM Tivoli Monitoring triggers to ignore situation stop events for pure events. See Customize how the IBM Tivoli Monitoring OMNIbus triggers handle event status updates from the monitoring servers.
Same as unidirectional behavior. Situation is stopped. Situations are stopped if an operator initiates the situation stop action from the Tivoli Enterprise Portal or modifies the situation definition. However, a situation is not stopped if its distribution list is modified.
Sampled events. Summary: The hub monitoring server closes all sampled events for the situation. These same events are cleared in the Netcool/OMNIbus ObjectServer.
Details: As the hub monitoring server closes all sampled events for the situation, it sends a situation stop event to Netcool/OMNIbus for each remote monitoring server that was monitoring the situation. The situation stop event specifies the remote monitoring server in the situation_thrunode EIF slot. (This slot is mapped to the ITMThruNode OMNIbus attribute by the Tivoli Netcool/OMNIbus EIF Probe.)
When the IBM Tivoli Monitoring triggers in Netcool/OMNIbus process a situation stop event, they clear all events for the situation that are detected by the remote monitoring server specified by the ITMThruNode OMNIbus attribute.
Same as unidirectional behavior. Agent stopped. Pure events. Summary: Stopping the agent has no effect on pure situation event status. An MS_Offline situation event is also sent to Netcool/OMNIbus to indicate that either the monitoring agent has been stopped or it has lost connectivity to the monitoring server.
Same as unidirectional behavior. Agent stopped. Sampled events. Summary: The sampled events from the agent are closed in the hub monitoring server if the agent is stopped for long enough. Its events are also cleared in the Netcool/OMNIbus ObjectServer. An MS_Offline situation event is also sent to Netcool/OMNIbus to indicate that either the monitoring agent has been stopped or it has lost connectivity to the monitoring server.
Details: After the monitoring server of the agent detects that the agent has not responded for three situation sampling intervals, the situation's event is closed in the hub monitoring server. A Closed status update event is sent to Netcool/OMNIbus ObjectServer by using flows 1 to 4 as shown in Figure 1. When the IBM Tivoli Monitoring triggers process the status update event, the triggers clear the event in the Netcool/OMNIbus ObjectServer.
If you do not want events to be closed in Netcool/OMNIbus after an agent is stopped, you can customize this behavior. See Customize event status processing behavior when agent switching is used or the agent goes offline.
Same as unidirectional behavior. Agent loses connectivity to the primary monitoring server and switches to the secondary monitoring server. Pure and sampled events. When an agent has a situation that has triggered and the situation is true, and the agent subsequently loses connectivity to the Tivoli Enterprise Monitoring Server and switches to a different monitoring server, the Hub monitoring server sends an EIF event with class type ITM_ControlSignal to the Netcool/OMNIbus ObjectServer for each situation event that is still open for the agent. When the IBM Tivoli Monitoring triggers process this event, they update the ITMThruNode attribute to specify the new monitoring server for the agent. However, the event might be closed by the original monitoring server if it detects the agent is no longer responding to it before the new monitoring server determines that the situation event is still true and the Hub monitoring server sends the ITM_ControlSignal event. There are environment variables that can be added to the monitoring server environment file to customize the behavior of event status processing when agent switching occurs to help ensure that events are not closed by the original monitoring server. For a complete description of these variables, see Customize event status processing behavior when agent switching is used or the agent goes offline.
If an agent switches to another monitoring server because the original monitoring server was stopped, all sampled events for the agent will be closed by the original monitoring server. This behavior is not configurable. However, you can configure whether pure events are closed in this scenario. See Customize how the IBM Tivoli Monitoring OMNIbus triggers handle event status updates from the monitoring servers.
Same as unidirectional behavior. Hub monitoring server stopped. Pure and sampled events. Summary: No flows occur between the hub monitoring server and Netcool/OMNIbus when the hub monitoring server is stopped. Same as unidirectional behavior. Hub monitoring server started. Pure events. Summary: The pure events in the Netcool/OMNIbus ObjectServer are unaffected. However, the hub monitoring server and Netcool/OMNIbus might not have the same status for pure events.
Details: When the hub monitoring server is started, it sends a master_reset event to Netcool/OMNIbus using flows 1 to 4 as shown in Figure 1. The IBM Tivoli Monitoring triggers in the Netcool/OMNIbus ObjectServer do not update the status of pure events when the master reset event is processed.
However, Netcool/OMNIbus and the hub monitoring server (and Tivoli Enterprise Portal) might have a different status for the pure events after the hub is restarted.
- The hub monitoring server does not have any event status for pure events that were opened or acknowledged prior to the hub restart if the events were for agents connected directly to the hub monitoring server. These events might still be open or acknowledged in the Netcool/OMNIbus ObjectServer.
- The hub monitoring server has a status of open for pure events that were open or acknowledged prior to the hub restart if the events were for agents connected to the remote Tivoli Enterprise Monitoring Server. However, these events may be acknowledged, cleared, or deleted in Netcool/OMNIbus.
Same as unidirectional behavior. Hub monitoring server started. Sampled events. Summary: The sampled events in the Netcool/OMNIbus ObjectServer from this hub monitoring event are cleared.
Details: When the hub monitoring server is started, it sends a master_reset event to Netcool/OMNIbus using flows 1 to 4 as shown in Figure 1. The IBM Tivoli Monitoring triggers in the Netcool/OMNIbus ObjectServer clear all sampled events from this hub monitoring server when the master reset event is processed. The master reset handling ensures that events are cleared in Netcool/OMNIbus if the situation condition became false while the hub monitoring server was stopped. The events whose situation conditions are still true will be reopened in Netcool/OMNIbus after the master reset event is sent.
Same as unidirectional behavior. Remote monitoring server stopped. Pure events. Summary: The hub monitoring server closes all of the pure events for agents connected to the remote monitoring server. The same events are cleared in Netcool/OMNIbus Object unless you configure a different behavior. An MS_Offline situation event is sent to Netcool/OMNIbus for the remote monitoring server to indicate that these managed systems are not being monitored. You will not see an MS_Offline message for each of the monitoring agents when the remote monitoring server goes offline.
Details: As the hub monitoring server closes the pure events for each situation being monitored by the remote monitoring server, it sends a situation stop event to Netcool/OMNIbus and specifies the remote monitoring server in the situation_thrunode EIF slot. (This slot is mapped to the ITMThruNode OMNIbus attribute by the Tivoli Netcool/OMNIbus EIF Probe.)
When the IBM Tivoli Monitoring triggers in Netcool/OMNIbus process a situation stop event, they clear all events for the situation that are detected by the remote monitoring server specified by the ITMThruNode OMNIbus attribute. However, you can configure the IBM Tivoli Monitoring triggers to ignore situation stop events for pure events. See Customize how the IBM Tivoli Monitoring OMNIbus triggers handle event status updates from the monitoring servers.
Same as unidirectional behavior. Remote monitoring server stopped. Sampled events. Summary: The hub monitoring server closes all of the sampled events for agents connected to the remote monitoring server. The same events are cleared in Netcool/OMNIbus Object. An MS_Offline situation event is sent to Netcool/OMNIbus for the remote monitoring server and each of the monitoring agents connected to the monitoring server to indicate that these managed systems are not being monitored. You will not see an MS_Offline message for each of the monitoring agents when the remote monitoring server goes offline.
Details: As the hub monitoring server closes the sampled events for each situation being monitored by the remote monitoring server, it sends a situation stop event to Netcool/OMNIbus and specifies the remote Tivoli Enterprise Monitoring Server in the situation_thrunode EIF slot. (This slot is mapped to the ITMThruNode OMNIbus attribute by the Tivoli Netcool/OMNIbus EIF Probe.)
When the IBM Tivoli Monitoring triggers in Netcool/OMNIbus process a situation stop event, they clear all events for the situation that are detected by the remote monitoring server specified by the ITMThruNode OMNIbus attribute.
Same as unidirectional behavior. Remote monitoring server started. Pure and sampled events. Same as unidirectional behavior when a remote monitoring server is stopped. Same as bidirectional behavior when a remote monitoring server is stopped.
Behavior of events originating from IBM Tivoli Monitoring agents
Action Event Type Behavior Situation condition becomes true. Pure and sampled events. A new event is opened in the agent and in the Netcool/OMNIbus ObjectServer if it does not deduplicate an existing event.
An event with open status is sent from the agent to Netcool/OMNIbus each time the situation condition is true, if the following conditions are met:
- The situation generates pure events, or
- The situation generates sampled events, the events are sent to Netcool/OMNIbus using SNMP, and the situation mode is set to Rising Continuous.
Sampled event situation condition is no longer true. Sampled events. Event is closed in the agent and cleared in the Netcool/OMNIbus ObjectServer. Event acknowledged or deacknowledged using Netcool/OMNIbus Event List UI in OMNIbus. Pure and sampled events. The event status is changed to acknowledged or deacknowledged in Netcool/OMNIbus but the event status maintained by the agent is unaffected. Event cleared or deleted using Netcool/OMNIbus Event List UI. Pure events. The event is cleared or deleted in the Netcool/OMNIbus ObjectServer. However, the event status maintained by the agent is not affected. Event cleared or deleted using Netcool/OMNIbus Event List UI. Sampled events. The event is cleared or deleted in the Netcool/OMNIbus ObjectServer. However, the event status maintained by the agent is unaffected and the Netcool/OMNIbus operator is not notified if the event condition has not been resolved, unless the following conditions are met:
- The agent has been configured to send SNMP events to Netcool/OMNIbus, and
- The situation mode is set to Rising Continuous so that an event is sent to Netcool/OMNIbus each sampling interval that the situation event evaluates to true. With this mode, the event is reopened in Netcool/OMNIbus if the event condition is still true.
Situation is stopped using the Agent Service Interface. Pure and sampled events. If the agent is configured to send lifecycle events when a situation is stopped, a EE_SIT_STOPPED event is sent to Netcool/OMNIbus.
For agents that send SNMP events to Netcool/OMNIbus, the events in Netcool/OMNIbus are not affected by this lifecycle event.
For agents that send EIF events to Netcool/OMNIbus, the events from the agent for the stopped situation are cleared in Netcool/OMNIbus.
Agent is stopped. Pure and sampled events. No events are sent by the agent when it is stopped. Agent is started. Pure and sampled events. If the agent is configured to send SNMP events to Netcool/OMNIbus, no events other than lifecycle traps are sent by the agent when it is started.
If the agent is configured to send EIF events to Netcool/OMNIbus, by default the agent does not send any events (other than lifecycle events) to Netcool/OMNIbus when it is started. However, you can change this behavior and configure the agent to send a master reset event to Netcool/OMNIbus when the agent is started. When the IBM Tivoli Monitoring triggers in Netcool/OMNIbus process this event, they clear all events for the agent. This ensures that events are cleared in Netcool/OMNIbus if the situation condition became false while the agent was stopped. The events whose situation conditions are still true will be reopened in Netcool/OMNIbus after the master reset event is sent.
The agent does not maintain event status across restarts.
Agents can also send lifecycle and heartbeat events to Netcool/OMNIbus. For more details on lifecycle and heartbeat events, see the Agent Autonomy chapter in the IBM Tivoli Monitoring Administrator's Guide.
Parent topic:
Architecture overview