Certificate expiration monitoring

The certificate expiration monitor admin task is a scheduled task that cycles through all the keystores in the security configuration and reports on any certificates that are expired, certificates that fall within the expiration threshold, and certificates that fall within the pre-notification period.

Certificate expiration monitoring relies on the following definitions:

Expired certificates

Certificates are created with a finite life span. Self-signed or chained certificates that have reached the end of their life span are reported and replaced, if possible. Certificate authority (CA) signed certificates cannot be replaced but will be reported. Replacing CA-signed certificates is the responsibility of the administrator.

Certificates within the expiration threshold

There is a period of time before a certificate expires. A certificate in this period of time is one within the expiration threshold. The server replaces certificates within the expiration threshold so that the certificate does not expire and cause outages. By default the expiration threshold is 60 days, but can be configured as required.

Pre-notification period

Before a certificate falls within the expiration threshold there are warnings issued that indicate that the certificate will be replaced, when the expiration threshold date is reached. The period of time prior to the expiration threshold date is called the pre-notification period and is set at 90 days for the certificate.

The certificate expiration monitor performs the following:

  1. Clears out the NodeDefaultDeletedStore or DmgrDefaultDeletedStore.

    This operation is performed silently without reporting that the certificates are deleted.

  2. Checks the root key stores, DmgrDefaultRootStore or NodeDefaultRootStore and the DmgrRSATokenRootStore or NodeRSATokenRootStore. If any root certificates are expired, falls in the threshold period, or the pre-notification period, then the certificate is noted in the report.

  3. If there are any root certificates that are expired or fall in the threshold period that root certificate is recreated using all the information used to create the original one. Any signer certificates from the original root certificate are replaced with the signers from the new root certificate.

  4. If a root certificate is replaced, then all the keystores are checked to see if there are any chained certificates signed with the original root certificate. If there are, then the chain certificate is renewed (recreated with the new root certificate). Any signer certificate from the original certificate is replaced with the signer from the recreated certificate.

  5. After all root keystores are processed, the rest of the keystores are checked for expired certificates, certificates in the expiration threshold, or certificates in the pre-notification period. Any certificate falling in any one of these categories is noted in the report.

  6. If there are any expired certificates or certificates in the expiration threshold period and these certificates are self-signed certificates or chained certificates created by WebSphere, then they are replaced. If the chained certificates root is not in the root key store then it will be recreated as a default root certificate. Any signer certificates from the original certificate are replaced with the signer from the new certificate."

  7. A report is generated and returned, written to a log file, or mailed.

The server default certificate is a chained certificate with a 365 day life span. It is signed with the default root certificate which has a 15 year life span.

Configure this monitor task to run according to a particular schedule. The schedule produces the next start date that persists in the configuration and, when the date is reached, WAS starts the monitor to check all of the keystores for certificates that meet the expiration threshold. We can start the task manually to run at any time.

The following security.xml configuration object specifies when the monitor task starts, determines the certificate expiration threshold, and indicates whether we are notified in an e-mail using Simple Mail Transfer Protocol (SMTP) or in a message log.

<wsCertificateExpirationMonitor xmi:id="WSCertificateExpirationMonitor_1" name="Certificate Expiration Monitor" daysBeforeNotification="30" isEnabled="true" autoReplace="true" deleteOld="true" wsNotification="WSNotification_1" wsSchedule="WSSchedule_2" nextStartDate="1134358204849"/>

The expiration monitor replaces self-signed certificates and chained personal certificates that are signed by a root certificate in DmgrDefaultRootStore or NodeDefaultRootStore. Self-signed certificates are renewed using all the information that was used to create the original self-signed certificate. A chained certificate is renewed using the same root certificate that was used to sign the original certificate.

The expiration monitor automatically replaces only self-signed certificates and chained certificates that are expired or that meet the expiration threshold criteria. To replace all of the signers from the old certificate with the signer that belongs to the new certificate in all the keystores in the configuration for that cell, set the autoReplace attribute to true. When the deleteOld attribute is true, the old personal certificate and old signers also are deleted from the keystores. The isEnabled attribute determines whether the expiration monitor task runs based upon the nextStartDate attribute that is derived from the schedule. The nextStartDate attribute is derived from the schedule in milliseconds since 1970, and is identical to the System.currentTimeMillis(). If the nextStartDate has already passed when an expiration monitor process begins, and the expiration monitor is enabled, the task is started, but a new nextStartDate value is established based on the schedule.

The following sample the schedule object shows the frequency attribute as the number of days between each run of the certificate monitor.

<wsSchedules xmi:id="WSSchedule_2" name="ExpirationMonitorSchedule"  frequency="30" dayOfWeek="1" hour="21" minute="30"/>

The dayOfWeek attribute adjusts the schedule to run on a specified day of the week, which is always the same day regardless of whether the frequency is set to 30 or 31 days. Based on 24-hour clock, the hour and minute attributes determine when the expiration monitor is started on the specified day.

The following sample code of the notification object shows the notification configuration, which notifies you after the expiration monitor runs.

<>wsNotifications xmi:id="WSNotification_1" name="MessageLog" logToSystemOut="true" emailList=""/
For expiration monitor notifications, we can select message log, e-mail using SMTP server, or both methods of notification. When you configure the e-mail option, use the format user@domain@smtpserver. If we do not specify an SMTP server, WAS defaults to the same domain as the e-mail address. For example, if we configure joeuser@ibm.com, WAS attempts to call smtp-server.ibm.com. To specify multiple e-mail addresses using scripting, add a pipe (|) character between entries. When specify the logToSystemOut attribute, the expiration monitor results are sent to the message log for the environment, which is typically SystemOut.log.

The expiration monitor clears out the deleted certificates keystore. The monitor first clears out the deleted keystore. Due to the nature of the PKCS12 keystore, there must be at lease one entry in the keystore so the signer certificates from the dummy key store will remain in the deleted keystore. There is no reporting on the certificate being deleted from the deleted keystore.

When the expiration monitor replaces certificates, this can dynamically affect the runtime when the following configuration option is enabled:

Security > SSL certificate and key management. Under settings, check the checkbox for Dynamically update the run time when SSL configuration changes occur.

When enabled, any certificates that are replaced causes the client SSL runtime to begin using the new certificates immediately, which in turn, flushes SSL and keystore caches and causes some ports using SSLServerSockets (RMI/IIOP on distributed and Admin SOAP) to restart. Restarting ports breaks existing connections. These connections can be reconnected after the port restart is completed. Endpoints using the channel framework (HTTP, BUS, RMI/IIOP on z/OS) leave existing connections unaffected but still use the new certificates for new connections.

When the dynamic change property is disabled and before the new certificates become effective, the administrator needs to recycle all processes in the entire cell after each node has the synchronized configuration. Regardless of which method is chosen, you should always check the health of the cell after the certificate expiration monitor has run (based on the schedule specified). The schedule should be set to run the certificate expiration monitor during a maintenance period so that if a restart is required after the certificate replacement, it will not cause unexpected outages.


Related concepts

Default chained certificate configuration in SSL