IBM Tivoli Monitoring > Version 6.3 Fix Pack 2 > Installation Guides > Installation Guide > Performance tuning > Tivoli Data Warehouse
IBM Tivoli Monitoring, Version 6.3 Fix Pack 2
Historical data collection
This section contains recommendations related to historical data collection and controlling the volume of data that is generated and loaded into the Tivoli Data Warehouse.
- Do not start collecting historical data for an attribute group without first estimating the data volume that will be generated and the disk space requirements on both the agent and the warehouse database. The Warehouse load projections spreadsheet helps you make these calculations. You can find this spreadsheet by searching for "warehouse load projections" or the navigation code "1TW10TM1Y" in IBM Integrated Service Management Library.
- Collect historical data only for attribute groups where there is a planned use for the information. Do not collect data that will not be used.
- The number of rows per day generated by a monitoring agent collecting historical data for an attribute group can be calculated with the following formula:
( 60 / collection interval) * 24 * (# instances at each interval)
Where:
- 60 Represents the 60 minutes in an hour.
- collection interval The data collection interval, in minutes. This value can be 1, 5, 15, 30, 60, or 1440 (1 day).
- 24 Represents 24 hours in one day.
- # instances at each interval The number of instances recorded at each interval.
The two variables in this formula are the collection interval and the number of instances recorded at each interval.
- The collection interval is a configuration parameter specified in the Historical Data Collection configuration dialog.
- The number of instances recorded at each interval depends on the nature of the attribute group and the managed system being monitored. Some attribute groups, such as NT_Memory, generate a single row of data per collection interval. Most attribute groups, however, generate multiple rows of data, one row for each monitoring instance (for example, one per CPU, one per disk, and so on). Certain attribute groups can be instance-intense, generating dozens or hundreds of rows of data per collection interval. Examples of instance-intense attribute groups would be those reporting information at the process level, thread level, disk level (for file servers), or network connection level.
For attribute groups that return multiple rows, the number of instances recorded at each interval is configuration-dependent, and can be different from one monitoring environment to another. The Warehouse load projections spreadsheet requires the user to specify the number of instances recorded at each interval. There are several approaches that you can use to come up with this number.
- Use the portal client, build a table view for the monitoring agent and define a query to obtain data from the required attribute group. The number of rows shown in the table view is the number of instances that would be recorded at each interval. For details on how to define table views in the portal client, see the IBM Tivoli Monitoring Tivoli Enterprise Portal User's Guide.
- Issue a SOAP call to collect data for this attribute group from the monitoring agent. The number of data rows returned by the SOAP call is the number of instances that would be recorded at each interval. For details on how to issue SOAP calls, see Appendix A "Tivoli Enterprise Monitoring Web services" in the IBM Tivoli Monitoring Administrator's Guide.
- If you have a test environment, you can create a monitoring server to use in historical data collection testing. Enable historical data collection for the required attribute group under this remote monitoring server, and configure a representative agent to connect to this monitoring server. When the agent uploads data to the Warehouse Proxy Agent, you can query the WAREHOUSELOG table to see how many rows were written by the agent for the attribute group.
To minimize the data volume (rows per day) generated by a monitoring agent for an attribute group, consider the following two recommendations:
- Use the longest data collection interval (1, 5, 15, 30, 60 or 1440 minutes) that will provide the required level of information.
- Avoid or minimize data collection for instance-intense attribute groups, which can generate many rows per data collection interval.
- When configuring historical data collection for an attribute group, you specify which monitoring servers collects the data. Since historical data collection is configured at the monitoring server level, restrict the number of agents collecting data for an attribute group, if possible, by enabling historical data collection on a subset of the monitoring servers.
- To minimize the performance impact on the monitoring server, configure historical data to keep short-term history files at the agent, if possible, rather than at the monitoring server.
- Enable warehouse collection only for attribute groups where there is a planned use for the information. For attribute groups with historical data collection enabled but not configured for warehouse collection, schedule regular tasks to prune the short-term history files (the supplied programs are described in the IBM Tivoli Monitoring Administrator's Guide).
- To spread the warehouse collection load across the day, configure warehouse collection to occur hourly rather than daily.
Parent topic:
Tivoli Data Warehouse