Parameters for file log agents

We can define the following parameters for file log agents:

buffer_size

Reduce memory fragmentation and improve the performance of writing to a file by:

• Not queuing many small events individually to the file log agent.
• Buffering events into blocks of a nominated size before queuing for writing.

The buffer_size parameter specifies the maximum size message the program attempts to construct by combining smaller events into a large buffer.

Buffers consist of only an integral number of events; events are not split across buffers. If any individual event exceeds that maximum configured size, the large event is recorded in a buffer of its own, exceeding the configured value. The default buffer size for logging to a file is 0 bytes. This value prevents buffering and each event is handled individually.

If a value is specified for the buffer_size parameter, events are packed into buffers of that size before queuing to the file log agent. For example, around 10 events are packed into each buffer that is written to the file when:

• The value for the buffer_size parameter is set to 2 KB.
• Events are assumed to be about 256 bytes.

This process reduces the number of disk input/outputs (I/Os) that are made while logging to 10 percent of the equivalent non-buffering case. A default queue size of 200 also consumes around 10 times the memory of a default configuration that did no buffering if:

• The buffer size was 2 KB.
• The event size was around 200 bytes.

This size is because the maximum queue size value has not been changed. However, the size of events being queued has increased tenfold.

flush_interval

The flush_interval parameter is a multiuse parameter.

Ensure that stream buffers are flushed to disk regularly. Configure the frequency with which the server asynchronously forces a flush of the file stream to disk. To configure this frequency, use the flush_interval parameter. The value that is defined for this parameter is 0, < 0, or the flush interval in seconds.

Specify a value of 0 results in the flushing of the buffer every 600 seconds.

Specify a value of < 0 results in the absolute value used as the asynchronous flush frequency. However, a stream flush is also forced synchronously after each record is written.

Events are consolidated into large buffers based on the value of the buffer_size parameter. However, the flush_interval parameter also might affect the size of buffer written. When a flush is scheduled, an in-memory, partially filled buffer is also queued for writing before it completes the buffer fill.

The event queue is triggered for processing at the flush interval rate. The trigger enables processing of events that were waiting for longer than the scheduled flush time. Such processing applies to a scenario when the queue does not reach the high water mark between scheduled flushes.

hi_water

Processing of the event queue is scheduled regularly at the configured flush interval. It also is triggered asynchronously by the queue size that reaches a high water mark on the event queue.

The default value is two-thirds of the maximum configured queue size. If the maximum queue size is zero, the high water mark is set to a default of 100.

The transaction rates and the values of these options determine the maximum amount of memory that is consumed by enabling event logging to file.

If the event queue high water mark is set to 1, every event queued is relayed to the log agent as soon as possible. This setting is not optimal. Use it to ensure that events get to disk as fast as possible. Doing so adversely impacts overall performance.

log_id

An open log file is associated with a short name identifier to facilitate the recording of events from different categories to the same file. Use the log_id parameter to set the log file identifier (ID) explicitly; otherwise, it is given a default value. If the path parameter is specified, the default value is the configured path name. If the path parameter is not specified, the log ID defaults to the domain component of the event category being captured. For example:

logcfg = audit.azn:file

implies

log_id=audit

To capture events to a common file, set the log file ID to a suitable value in a fully optioned file configuration. Then, use the shorthand configuration variant to capture events from additional categories as shown:

[aznapi-configuration]
logcfg = audit.azn:file path=audit.log,
     rollover_size=-1,flush_interval=20,log_id=audit,
     ...
logcfg = audit.authn:file log_id=audit

Because of the default rules, this configuration is also equivalent to the following specification:

[aznapi-configuration]
logcfg = audit.azn:file path=audit.log,
     rollover_size=-1,
     ...
logcfg = audit.authn:file

If you construct a configuration where the log ID value does not match any open log file, no events are captured. For example, the following configuration does not record any events because the configuration line that initializes the log file was commented out:

[ivacld]
#logcfg = audit.azn:file path=azn.log,log_id=azn,...
logcfg = audit.authn:file log_id=azn

mode

Configure the mode parameter to open a file in either text or binary mode. For example:

[aznapi-configuration]
logcfg = audit.azn:file
...
mode={text|binary},
...

Text mode is deprecated on AIX, Linux, and Solaris operating systems. Binary mode on a Windows operating system writes the log file in an AIX®, Linux®, or Solaris-compatible format.

path

The path specifies the name and location of a log file. There is no default value, because the value of the log_id parameter takes precedence. An example for the WebSEAL audit trail file on AIX, Linux, and Solaris operating systems is as follows:

[aznapi-configuration]
logcfg = category:file path=audit.log

The directory portion of this path must exist. The log file is created if it does not exist.

queue_size

There is a delay between events being placed on the queue and the file log agent removing them. The queue_size parameter specifies the maximum size to which the queue is allowed to grow.

Consider that a new event is ready to be placed on the queue. Then, if the queue reaches the maximum size, the requesting thread is blocked until space is available in the queue. This process causes the performance of the event propagation thread to slow down to that of the file logging thread.

Limiting the queue size for the log agent must be configured with setting the queue size for the central event propagation queue. Unless the event propagation defined by the queue_size parameter is constrained appropriately, memory usage can still grow without bounds.

[aznapi-configuration]
logcfg = audit.azn:file
...
queue_size=number_events,
...

Default is 0. Specifying a value of 0 indicates that no limit is enforced on the growth of the unprocessed event queue. Correspondingly, the event propagation thread is not constrained by the speed of the logging thread. The unrecorded event queue can grow to an unmanageable size if:

• You are using the default.
• Events are being generated faster than they can be recorded to file.

rollover_size

Configure the rollover_size parameter to specify the maximum size to which a log file can grow. The default value is 2000000 bytes.

When the size of a log file reaches the specified rollover threshold, the existing file is backed up. The back-up happens to a file of the same name with the current date and time stamp appended. A new log file is then started. The possible rollover size values are interpreted as follows:

• If the rollover_size value is less than zero, a new log file is created:
  • With each invocation of the process, and
  • Every 24 hours since that instance.

• If the rollover_size value is equal to zero, the log file grows until it reaches 2 GB and then rolls over. If a log file exists at startup, new data is appended to it.

• If the rollover_size value is greater than zero, the log file grows until it reaches the lesser of the following values and then rolls over:
  • The specified value
  • 2 GB

  If a log file exists at startup, new data is appended to it.

max_rollover_files

Configure the max_rollover_files parameter to specify the maximum number of rollover files to be kept on disk.

When the number of rollover log files reaches the specified threshold, the oldest log file is deleted.

The value of this configuration parameter is interpreted as follows:

• If the max_rollover_files value is blank or not specified, then no rollover files are deleted.

• If the max_rollover_files value is equal to zero, then only the current log file is kept, and all rollover log files are deleted.

• If the max_rollover_files value is greater than zero, then only that number of rollover log files are kept. When the number of rollover log files exceeds max_rollover_files, the oldest log file is deleted.

Parent topic: Configure file log agents