IBM Tivoli Monitoring > Version 6.3 Fix Pack 2 > Administrator's Guide > Manage historical data

IBM Tivoli Monitoring, Version 6.3 Fix Pack 2

Conversion process for using delimited flat files

If you chose not to warehouse your data, you must convert your collected data to delimited flat files. Data can be scheduled for conversion either manually or automatically. If you choose to continue to convert data to delimited flat files, schedule data conversion to be automatic. Perform data conversion on a regular basis even if you are collecting historical data only to support short-term history displayed in product reports.

If the KHD_TOTAL_HIST_MAXSIZE environment variable is used, the agent can no longer write any historical data to the short-term history files once the limit is reached. This variable is a limit for the agents.

Data conversion programs

The conversion of short-term history files to delimited flat files is done by running a data rolloff program:

  • krarloff

  • KPDXTRA

Columns added to history data files and to meta description files

Four columns are automatically added to the history data files and to the meta description files:

  • TMZDIFF. The time zone difference from Universal Time (GMT). This value is shown in seconds.
  • WRITETIME. The CT time stamp when the record was written. This is a 16-character value in the format, where c is the century; yymmdd is the year, month, and day; and hhmmssttt is hours, minutes, seconds, and milliseconds: cyymmddhhmmssttt
  • SAMPLES. The SAMPLES column increments for every value collected during the same sample and then reset to its starting value again. Rows collected on the same sample have different SAMPLES column values.
  • INTERVAL. The time between samples, shown in milliseconds.

The data warehousing process adds only two columns, TMZDIFF and WRITETIME, to the Tivoli Data Warehouse database.

Meta description files

A meta description file describes the format of the data in the source files. Meta description files are generated at the start of the historical data collection process.

The various operating system environments use different file naming conventions. Here are the rules for some operating system environments:

  • IBM i and HP NonStop Kernel: Description files use the name of the data file as the base. The last character of the name is 'M'. For example, for table QMLHB, the history data file name is QMLHB and the description file name is QMLHBM.

  • z/OS: Description records are stored in the PDS facility, along with the data.
  • UNIX and Linux: Uses the *.hdr file naming convention.

  • Windows: Uses the *.hdr file naming convention.

Sample *.hdr meta description file

    TMZDIFF(int,0,4) WRITETIME(char,4,16) 
QM_APAL.ORIGINNODE(char,20,128) QM_APAL.QMNAME(char,148,48) 
QM_APAL.APPLID(char,196,12) QM_APAL.APPLTYPE(int,208,4) 
QM_APAL.SDATE_TIME(char,212,16) 
QM_APAL.HOST_NAME(char,228,48) 
QM_APAL.CNTTRANPGM(int,276,4) QM_APAL.MSGSPUT(int,280,4) 
QM_APAL.MSGSREAD(int,284,4) QM_APAL.MSGSBROWSD(int,288,4) 
QM_APAL.INSIZEAVG(int,292,4) QM_APAL.OUTSIZEAVG(int,296,4) 
QM_APAL.AVGMQTIME(int,300,4) QM_APAL.AVGAPPTIME(int,304,4) 
QM_APAL.COUNTOFQS(int,308,4) QM_APAL.AVGMQGTIME(int,312,4) 
QM_APAL.AVGMQPTIME(int,316,4) QM_APAL.DEFSTATE(int,320,4) 
QM_APAL.INT_TIME(int,324,4) QM_APAL.INT_TIMEC(char,328,8) 
QM_APAL.CNTTASKID(int,336,4) SAMPLES(int,340,4) 
INTERVAL(int,344,4)

For example, an entry can have the form, where int identifies the data as an integer, 75 is the byte offset in the data file, and 20 is the length of the field for this attribute in the file: 

    attribute_name(int,75,20)


Parent topic:

Manage historical data

+

Search Tips   |   Advanced Search