$('a[name]').remove(); $('#ic-homepage__footer').before('

'); $("#tabs").tabs({ selected: 1 }); $("#ic-homepage__ic-tips").append( quickTipHTML() ); unhideOneProductTip(); $("#ic-homepage__product-tips").wrapInner('

'); $("#ic-homepage__feed-tips").wrapInner('

'); });

IBM Tivoli Monitoring > Version 6.3 > User's Guides > Log File Agent User's Guide IBM Tivoli Monitoring, Version 6.3

Configuration file

The Tivoli Log File Agent uses a configuration file that contains configuration options and filters. You create this configuration file and configure the agent instance to use it.

The configuration file is read by the agent when it starts, and is monitored for changes to its timestamp every 60 seconds thereafter. If the timestamp of the file changes, the agent reinitializes its configuration dynamically, without requiring a restart. See Change the agent configuration and format files.

The .conf file for the Log File agent accepts these options:

DupDetectionKeyAttributes

A comma-separated list of IBM Tivoli Monitoring attributes that are used to determine which events are duplicates and which are not. For more information about LogfileEvents attributes, see LogfileEvents attribute group. If all the named attributes are the same in two events, then those two events are considered duplicates. This option applies only to IBM Tivoli Monitoring events. See Event filtering and summarization.

  1. The attributes names are case-sensitive and so you must enter the names exactly as described.

  2. If you do not provide a list of attributes, the values are defaulted to Class and Logname.

ENFORCE_STRICT_TEC_COMPATIBILITY

Set this parameter to Y to have the Log File agent treat white space characters in the same way as the Tivoli Enterprise Console Log File Adapter. This means that all white space characters in the log data are respected. For example, when you use a format such as "%s %s" to extract information from log messages, the Log File agent matches not only a literal space but also any other white space characters that are present such as tabs and carriage returns.

When this parameter is not set the default behavior of the Log File agent when it matches a Tivoli Enterprise Console style format string is to match as much of the input text as it can, while it processes the format from left-to-right.

For example, take the format string %s:%s and the input string one:two:three. The Log File agent by default assigns one.two to the first parameter (corresponding to the first %s), and it assigns three to the second parameter. The behavior of the Tivoli Enterprise Console Log File Adapter was the opposite. Setting the ENFORCE_STRICT_TEC_COMPATIBILITY parameter to Y causes the Log File agent to behave like the Tivoli Enterprise Console Log File Adapter in the way it matches formats.

  1. This parameter applies only to format statements that use the Tivoli Enterprise Console log file adapter syntax. It does not apply to format statements that use the regular expression syntax.

  2. Set this parameter has a performance impact. To give greater control over the behavior and performance of matching, avoid setting this parameter, and use regular expressions instead.

EventSummaryInterval

Specifies the number of seconds during which the agent searches for duplicate events to suppress. Set to a positive integer. This option applies only to IBM Tivoli Monitoring events. See Event filtering and summarization.

EventFloodThreshold

Specifies which events are sent when duplicate events are detected. Set to send_none, send_all, send_first, or a positive integer. This option applies only to IBM Tivoli Monitoring events. See Event filtering and summarization.

EventMaxSize

Specifies in bytes, the maximum size of a generated event. If specified, this parameter is used in two places:

  1. The parameter can be used by the agent to set the size of a buffer that is used to process events. If not set, this buffer defaults to a size of 16384 bytes. If the buffer is set too small, events are truncated and can be discarded.

  2. The parameter can be used by the EIF sender to set the size of a buffer that is used to send events to an EIF receiver, such as the OMNIbus EIF probe. If not set, this buffer defaults to a size of 4096 bytes. If the buffer is set too small, events are discarded.

FileComparisonMode

Specifies which log files are monitored when more than one matches a wildcard pattern. The following values are available:

CompareByAllMatches

This value is the default behavior. All files that match the wildcard pattern that is specified in LogSources are monitored.

CompareByLastUpdate

Of the files that match the wildcard pattern that is specified in LogSources, the file with the most recent last update timestamp is monitored.

CompareBySize

Of the two or more files that match the file name pattern criteria, the bigger file is selected for monitoring. Do not use CompareBySize with multiple matching files that are being updated at the same time and increasing their file sizes. If the largest file is subject to frequent change, monitoring might continually restart at the beginning of the newly selected file. Instead, use CompareBySize when there is a set of matching files, but only one is active and being updated at any specific time.

CompareByCreationTime

Of the files that match the wildcard pattern that is specified in LogSources, the file with the most recent creation timestamp is monitored.

Tip: The CompareByLastUpdate, CompareBySize, and CompareByCreationTime values can all be used for rolling log files. CompareByLastUpdate is normally used for these files.

Restriction: The CompareByCreationTime value:

  • Is applicable only to Windows operating systems as UNIX and Linux operating systems do not store a true creation time for files.

  • Is not supported for remote files that you monitor using the Secure Shell (SSH) File Transfer Protocol.

FQDomain

Specifies how and if the agent sets a domain name.

  • If set to yes, the agent determines the system domain name itself.

  • If set to no, the agent does not set a domain name. The fqhostname attribute is assigned a blank string.

  • If set so that it does not contain a yes or no value, the domain name is accepted as the value and it is appended to the host name.

See Format file.

IncludeEIFEventAttr

The agent includes a large attribute that is called EIFEvent, which is a representation of the event that would be sent through the Event Integration Facility if that feature is enabled. The information that is contained in the EIFEvent attribute can also be found in other attributes. Its large size made it problematic, thus it was disabled in version 6.2.3 and no value is shown if viewed on the Tivoli Enterprise Portal. Setting this value to y, re-enables the EIFEvent attribute in Tivoli Monitoring.

Using this attribute might cause situations to fail if you have large events. A large event in this context is an event where the total number of bytes necessary to contain all of the values, for all attributes, and their names, results in a string longer than 3600 bytes.

LognameIsBasename

When set to y, the value of the Logname attribute in IBM Tivoli Monitoring is the base name of the log file in which the event was found. This option applies only to IBM Tivoli Monitoring events. The path is removed. For example, /data/logs/mylog.log becomes mylog.log. If this value is set to n, then you get the full path. However, because the attribute is limited to 64 characters, setting it to n means that the name is truncated if it is longer. For this reason, the default value is y. To see the full path name in a longer attribute, you can specify it in the mappings section of a format in the .fmt file, for example, filename FILENAME CustomSlot1. The mapping completes the slot named filename with the full path of the file in which the event was found and maps it into CustomSlot1 that is 256 characters long.

LogSources

Specifies the text log files to poll for messages. The complete path to each file must be specified, and file names must be separated by commas. Within each file name, you can also use an asterisk (*) to represent any sequence of characters, or a question mark (?) to represent any single character. For example, mylog* results in polling all log files whose names begin with mylog, whereas mylog??? results in polling all log files whose names consist of mylog followed by exactly three characters. These wildcard characters are supported only within the file name; the path must be explicitly specified.

To use regular expressions or pattern matching in the path, see the RegexLogSources description.

A log file source is not required to exist when the agent is started; the log file is polled when it is created.

NewFilePollInterval

Specifies the frequency, in seconds, that the agent checks for new files to monitor. For example, if a file name specified by the LogSources or RegexLogSources configuration file settings does not yet exist, when the agent starts it checks again for the files existence after this interval.

NumEventsToCatchUp

Specifies the event in the log that the agent starts with. This option provides some flexibility if the source that is being monitored is new or the agent is stopped for an extended time. Valid values are as follows:

For text files, values 0 and -1 apply. For Windows event log, values 0, -1, and n apply.

0

Start with the next event in the logs. This value is the default.

-1

When set to -1, the agent saves its place in the file that is being monitored. It saves its place so that when the agent is stopped and later restarted, it can process any events that are written to the log while it was stopped. The agent otherwise ignores events that arrived while it was stopped and restarts from the end of the file. This setting does not apply to pipes, or syslog monitoring on UNIX and Linux.

n

Set to a positive integer. Starts with the nth event from the most current event in the logs; that is, start n events back from the most current event in the logs. If n is greater than the number of events that are available, all the events that are available are processed.

You can use the n value only for Windows Event Log. The n value is ignored when UseNewEventLogAPI is set to y.

PollInterval

Specifies the frequency, in seconds, to poll each log file that is listed in the LogSources option for new messages. The default value is 5 seconds

If you upgraded a Windows event log adapter from a previous release and you have a value that is set for PollingInterval in the Windows registry, specify the PollInterval option in the agent configuration file with the same value used in the Windows registry. This rule applies only if you are replacing a Tivoli Enterprise Console log file agent that had values in the registry.

ProcessPriorityClass

Specifies the process priority for the agent. You can adjust this value to improve system performance if the agent processes large volumes of events and is using too many processor resources. The possible values are:

  • A - Very low priority

  • B - Low priority

  • C - Typical priority

  • D - Above typical priority

  • E - High priority

  • F - Very high priority

  • USE_CONF_FILE_VALUE - Use the value specified in the configuration file (This value is the default)

The value of this setting is global. If you implement monitoring profiles by using subnodes this value applies to all monitoring profiles. Its value cannot be different in different configuration files for the same agent.

RegexLogSources

Specifies the text log files to poll for messages. It differs from the LogSources option in that regular expression meta characters can be used in the base name portion of the file name and at most one subdirectory of the file name. This difference provides greater flexibility to describe multiple files to monitor in multiple directories than the LogSources option.

For example, specifying /var/log/mylog* for the LogSources statement is identical to using the dot (.) meta character followed by an asterisk (*) meta character to form /var/log/mylog.* in the RegexLogSources statement. This type of qualifier results in polling all log files in the /var/log directory whose base names begin with mylog and are followed by zero or more characters. A /var/log/mylog.+ qualifier results in polling all log files in the /var/log directory whose names begin with mylog and are followed by one or more characters.

Similar to LogSources, the complete path to each file must be specified and the file names must be separated by commas. However, the comma is also a valid character inside a regular expression. In order to distinguish between a comma that is used as part of a regular expression and one used to separate file names, commas that are used as part of a regular expression must be escaped with the backslash (\) character.

For example, if you want to search for logs that match either of the following regular expressions: /logs/.*\.log and /other/logs/[a-z]{0,3}\.log, you must escape the comma in the {0,3} clause of the second expression, so that the agent does not mistake it for the beginning of a new expression: RegexLogSources=/logs/.*\.log,/other/logs/[a-z]{0\,3}\.log

If meta characters are used in the path name, the meta characters can be used in only one subdirectory of the path. For example, you can specify /var/log/[0-9\.]*/mylog.* to have meta characters in one subdirectory. The [0-9\.]* results in matching any subdirectory of /var/log that consists solely of numbers and dots (.). The mylog.* results in matching any file names in those/var/log subdirectories that begin with mylog and are followed by zero or more characters.

Because some operating systems use the backslash (\) as a directory separator it can be confused with a regular expression escape meta character. Because of this confusion forward slashes must always be used to indicate directories. For example, Windows files that are specified as C:\temp\mylog.* might mean the \t is a shorthand tab character. Therefore, always use forward slashes (/) on all operating systems for directory separators. The C:/temp/mylog.* example represents all files in the C:/temp directory that start with mylog.

If more than one subdirectory contains meta characters, a trace message is also issued. For example, c:/[0-9\.]*/temp.files/mylog.* has two subdirectories with meta characters. [0-9\.]* is the first subdirectory with meta characters and temp.files is the second subdirectory that used a dot (.) meta character. In this case, the agent assumes that the first subdirectory with the meta character is used and the subsequent directories with meta characters are ignored. For more information about troubleshooting issues with regular expressions, see Agent troubleshooting.

SubnodeName

A string value that can be used to override the default name that is assigned to a monitoring profile subnode. By default the subnode name that is assigned to a monitoring profile corresponds to the base name of the configuration file that is used for that profile. Using this setting a different subnode name can be assigned.

SubnodeDescription

A string value that can be used to assign a value to the Subnode Description attribute of the LFAProfiles workspace.

UnmatchLog

Specifies a file to log discarded events that cannot be parsed into an event class by the agent. The discarded events can then be analyzed to determine whether modifications are needed to the agent format file. Events that match a pattern that uses *DISCARD* do not appear in the unmatch log because they did match a pattern. This option is generally used in a test environment to validate the filters in the format file. This option fills up your file system if you leave it on for extended periods


Options for remote log file monitoring by using SSH

Other than SshHostList, which is a list, all options can have only one value, which is applied to all remote hosts specified in SshHostList.

Only text log files are supported. AIX error report, syslog, and Windows event log are not supported.

Tip: You can set up syslog to write its output to a text log file and then remotely monitor that text file with log file agent.

SshAuthType

Must be set to either PASSWORD or PUBLICKEY. If set to PASSWORD, the value of SshPassword is treated as the password to be used for SSH authentication with all remote systems. If set to PUBLICKEY, the value of SshPassword is treated as the pass phrase that controls access to the private key file. If set to PUBLICKEY, SshPrivKeyfile and SshPubKeyfile must also be specified.

SshHostList

A comma-separated list of remote hosts to monitor. All log files that are specified in the LogSources or RegexLogSources statements are monitored on each host that is listed here. If one of the host names specified is localhost, the agent monitors the same set of files directly on the local system. When you specify localhost, SSH is not used to access the files on the local system, the log files are read directly.

SshPassword

When the value of SshAuthType is PASSWORD, this value is the account password of the user that is specified in SshUserid. You can supply the account password in clear text, or you can supply a password that is encrypted with the itmpwdsnmp command. For more information about how to encrypt a password using theitmpwdsnmp command, see Encrypting a password or pass phrase.

When the value of SshAuthType is PUBLICKEY, this value is the pass phrase that decrypts the private key that is specified by the SshPrivKeyfile parameter. You can supply the pass phrase in clear text, or you can supply a pass phrase that is encrypted with the itmpwdsnmp command. For more information about how to encrypt a password using theitmpwdsnmp command, see Encrypting a password or pass phrase.

If the value of SshAuthType is PUBLICKEY, and you configured SSH not to require a pass phrase, SshPassword must be set to null. To set SshPassword to null, the entry in the configuration file is: 

    SshPassword=

SshPort

A TCP port to connect to for SSH. If not set, defaults to 22.

SshPrivKeyfile

If SshAuthType is set to PUBLICKEY, this value must be the full path to the file that contains the private key of the user that is specified in SshUserid and SshPubKeyfile must also be set. If SshAuthType is not set to PUBLICKEY, this value is not required and is ignored.

SshPubKeyfile

If SshAuthType is set to PUBLICKEY, this value must be the full path to the file that contains the public key of the user that is specified in SshUserid and SshPrivKeyfile must also be set. If SshAuthType is not set to PUBLICKEY, this value is not required and is ignored.

SshUserid

The user name on the remote systems which the agent uses for SSH authentication.


The following option is supported on UNIX and Linux only:

AutoInitSyslog

If this option is set to Yes, the agent automatically configures the syslog facility to write a standard set of events to a pipe that the agent monitors. By enabling this setting, you can monitor syslog events without maintaining and rolling-over log files. If not set in the configuration file, it is the same as being set to No.

Restriction: This option is not supported for remote log file monitoring.


The following options are supported on Windows only:

NTEventLogMaxReadBytes

If you are using the older NT Event Log interface (UseNewEventLogAPI is not set to y) to read Event Log data on a Windows system, the agent reads up to this number of bytes each time it checks the Event Log for new data. Setting the value to 0 causes the agent to attempt to read all new data, as it did in earlier releases. This activity can occupy the agent for a considerable amount of time on a system with many events. The default value is 655360. When set, the agent might not stop at exactly the value specified, but rather at the nearest multiple of an internal buffer size to this value.

PreFilter

Specifies how events in a Windows Event Log are filtered before agent processing. PreFilter statements are used by PreFilterMode when determining which events are sent from an event log to the agent. An event matches a PreFilter statement when each attribute=value specification in the PreFilter statement matches an event in the event log. A PreFilter statement must contain at least the log specification and can contain up to three further specifications, which are all optional: event ID, event type, and event source. The order of the attributes in the statement does not matter.

The basic format of the PreFilter statement is as follows: 

    PreFilter:Log=log_name;EventId=value; EventType=value;Source=value;

You can specify multiple values for each attribute by separating each with a comma.

Each PreFilter statement must be on a single line.

PreFilter is not mandatory. All Windows log events are sent to the agent if prefilters are not specified and PreFilterMode=OUT.

PreFilterMode

This option applies only to Windows Event Log. Specifies whether Windows log events that match a PreFilter statement are sent (PreFilterMode=IN) or ignored (PreFilterMode=OUT). Valid values are IN, in, OUT, or out. The default value is OUT.

PreFilterMode is optional; if PreFilterMode is not specified, only events that do not match any PreFilter statements are sent to the agent.

If you set PreFilterMode=IN, you must also define the PreFilter statements.

SpaceReplacement

Set to TRUE by default for Windows Event Log (Windows Server 2008 only) but not for previous versions of event log. When SpaceReplacement is TRUE, any spaces in the security ID, subsource, Level, and keywords fields of the event log messages are replaced with underscores (_). When SpaceReplacement is FALSE, any spaces in the security ID, subsource, Level, and keywords fields of the event log messages remain unchanged. For more information about this option, see Windows 2008 event log.

UseNewEventLogAPI

When set to y on Windows systems, uses the new Windows Event Log interface for event logs. The option is supported only on Windows 2008 and later. The option is needed to access many of the new event logs that debuted in Windows 2008 and the applications that run on it. The option is ignored on earlier versions of Windows and on UNIX and Linux. For more information about this option, see Windows 2008 event log.

WINEVENTLOGS

Controls which Windows event logs are monitored.

The WINEVENTLOGS statement is a comma-delimited list with no spaces. See Windows 2008 event log.

Any carriage returns, tabs, or newlines in Windows events are replaced by spaces.


The following option is supported on AIX only:

AIXErrptCmd

An errpt (error report) command string can be supplied here, which the agent runs. The command output is fed into the stream of log data that is being monitored.

For example, the following command causes the agent to search for the mmddhhmmyy string and replace it with the actual date and time on startup. Only the first occurrence of the string is replaced. 

    AIXErrptCmd=errpt -c -smmddhhmmyy

Although you can supply your own errpt command, the Log File agent enforces the same constraints on this command that the Monitoring Agent for UNIX Logs does. In particular, use the -c (concurrent mode) option so that the command runs continuously. You cannot use the -t option or the following options that result in detailed output: -a, -A, or -g. The data stream is the standard output from the errpt command, so regular expressions in the.fmt file must be written to match. For example, the data output might be: 

    IDENTIFIER TIMESTAMP  T C RESOURCE_NAME  DESCRIPTION
F7FA22C9   0723182911 I O SYSJ2      UNABLE TO ALLOCATE SPACE IN FILE SYSTEM
2B4F5CAB   1006152710 U U ffdc       UNDETERMINED ERROR
2B4F5CAB   1006152610 U U ffdc       UNDETERMINED ERROR

A sample format that picks up the data rows, but not the header, would be: 

    REGEX GenericErrpt
^([A-F0-9]{8}) +([0-9]{10}) ([A-Z]) ([A-Z]) (\S+) +(.*)$
Identifier $1 CustomSlot1
Timestamp  $2 CustomSlot2
T          $3 CustomSlot3
C          $4 CustomSlot4
Resource   $5 CustomSlot5
msg        $6
END

see also the Agent Builder User's Guide under Monitoring an AIX Binary Log.


The following options apply only where events are being forwarded to EIF:

BufferEvents

Specifies how event buffering is enabled. The possible values are:

  • YES - Stores events in the file that is specified by the BufEvtPath option (This value is the default).

  • MEMORY_ONLY - Buffers events in memory.

  • NO - Does not store or buffer events.

BufEvtPath

Specifies the full path name of the agent cache file. If this path is not ecified the default is:

  • UNIX systems: For more information/etc/Tivoli/tec/cache

  • Windows systems: \etc\Tivoli\tec\cache

If events are being forwarded to more than one server, a BufEvtPath value must be specified for each forwarding channel. An index number is appended to the BufEvtPath name for each additional entry. For example, use BufEvtPath1 to indicate the path name of the agent cache file for forwarding to the first additional server. The value that is set in each BufEvtPath must be unique.

BufEvtMaxSize

Specifies the maximum size, in KB, of the agent cache file. The default value is 64. The cache file stores events on disk when the BufferEvents option is set to Yes. The minimum size for the file is 8 KB. File sizes specified less that this level are ignored, and 8 KB is used. There is no upper limit to the value you specify for the maximum file size.

If the cache file exists, you must delete the file for option changes to take effect.

NO_UTF8_CONVERSION

Specifies whether Tivoli Event Integration Facility encodes event data in UTF-8. When this option is set to YES, Tivoli Event Integration Facility does not encode event data in UTF-8. The data is assumed to already be in UTF-8 encoding when passed to Tivoli Event Integration Facility. A prefix is however added to the flag to indicate that the data is in UTF-8 encoding (if the flag does not exist at the beginning of the event data). The default value is NO.

MaxEventQueueDepth

This value indicates the maximum number of events that can be queued for forwarding. When the limit is reached each new event that is placed on the queue bumps the oldest event from the queue. If not specified the default value is 1000. This setting applies to all forwarding channels if NumAdditionalServers is used.

NumAdditionalServers

This entry is required if you want to forward events to more than one Netcool OMNIbus ObjectServer. Its value is used to indicate the number of servers that events are forwarded to. Valid values are 1-8.

ServerLocation

Specifies the name of the host on which the event server is installed. Specify host_name or IP_address. Use the dotted format for IP_address. You can specify failover values for the server locations if you want to, such as: ServerLocation1=2.3.4.5,2.3.4.6., If you specify failover values for ServerLocation you must also specify an additional ServerPort value for each ServerLocation.

If events are being forwarded to more than one server, a ServerLocation value must be specified for each server. An index number is appended to the ServerLocation name for each additional entry. For example, use ServerLocation1 to specify the name of the host on which the first additional server is installed.

ServerPort

Specifies the port number on which the EIF receiver listens for events. The ServerPort option can contain up to eight values, which are separated by commas. If failover values are specified for ServerLocation set an equivalent ServerPort value. The ServerPort is not used when the TransportList option is specified.

If events are being forwarded to more than one server, a ServerPort value must be specified for each server. An index number is appended to the ServerPort name for each additional entry. For example, use ServerPort1 to specify the port number on which the EIF receiver listens for events for the first additional server.

TransportList

Specifies the user-supplied names of the transport mechanisms, which are separated by commas. When a transport mechanism fails for sender applications, the API uses the following transport mechanisms in the order specified in the list. For receiving applications, the API creates and uses all the transport mechanisms. The transport type and channel for each type_namemust be specified using the Type and Channels keywords:

type_nameType

Specifies the transport type for the transport mechanism that is specified by the TransportList option. SOCKET is the only supported transport type.

The server and port for each channel_name are specified by the ServerLocation and ServerPort options.

type_nameChannels

channel_namePort

Specifies the port number on which the transport mechanisms server listens for the specified channel (set by the Channel option). When this keyword is set to zero, the portmapper is used. This keyword is required.

channel_namePortMapper

Enables the portmapper for the specified channel.

channel_namePortMapperName

If the portmapper is enabled, specifies the name of the portmapper.

channel_namePortMapperNumber

Specifies the ID registered by the remote procedure call.

channel_namePortMapperVersion

If the portmapper is enabled, specifies the version of the portmapper

channel_nameServerLocation

Specifies the name of the event server and region on which the server for transport mechanisms is located for the specified channel. The channel is set by the Channel option. This keyword is required.


Unsupported configuration file options:

The following .conf file options that are supported by the Tivoli Enterprise Console Log File Adapter are ignored by the Log File agent.

Other than the previously listed configuration file options, the Log File agent supports all of the other configuration file options that are documented in the IBM Tivoli Enterprise Console Adapters Guide.

The configuration file accepts generic EIF options when used directly with OMNIbus. These options operate only over an EIF connection to OMNIbus. They do not affect events that are sent to IBM Tivoli Monitoring. For more information about these EIF options, see EIF keywords.

Related concepts:

Event filtering and summarization

Related reference:

LogfileEvents attribute group
Change the agent configuration and format files
Configuration Files

+

Search Tips   |   Advanced Search