IBM MQ and IBM MQ Appliance on premises considerations for GDPR readiness
For PID(s):
- 5724-H72 IBM MQ
- 5655-AV9 IBM MQ Advanced for z/OSĀ®
- 5655-AV1 IBM MQ Advanced for z/OS, Value Unit Edition
- 5655-AM9 IBM MQ Advanced Message Security for z/OS
- 5725-S14 IBM MQ Appliance M2000
- 5725-Z09 IBM MQ Appliance M2001
- 5655-MQ9 IBM MQ for z/OS
- 5655-VU9 IBM MQ for z/OS Value Unit Edition
- 5639-L92 IBM MQ Internet Pass-Thru
- 5655-MF9 IBM MQ Managed File Transfer for z/OS
- 5655-ADV IBM WebSphere MQ Advanced for z/OS
- 5655-AMS IBM WebSphere MQ Advanced Message Security for z/OS
- 5724-R10 IBM WebSphere MQ File Transfer Edition for Multiplatforms
- 5724-A39 IBM WebSphere MQ for HP NonStop Server
- 5724-A38 IBM WebSphere MQ for HP OpenVMS
- 5655-W97 IBM WebSphere MQ for z/OS
- 5655-VU8 IBM WebSphere MQ for z/OS Value Unit Edition
- 5655-VUE IBM WebSphere MQ for z/OS Value Unit Edition
- 5725-C79 IBM WebSphere MQ Hypervisor Edition for Red Hat Enterprise Linux for x86
- 5725-F22 IBM WebSphere MQ Hypervisor for AIX
- 5655-MFT IBM WebSphere MQ Managed File Transfer for z/OS
Notice:
This document is intended to help you in your preparations for GDPR readiness. It provides information about features of IBM MQ that we can configure, and aspects of the product's use, that you should consider to help your organization with GDPR readiness. This information is not an exhaustive list, due to the many ways that clients can choose and configure features, and the large variety of ways that the product can be used in itself and with third-party applications and systems.
Clients are responsible for ensuring their own compliance with various laws and regulations, including the European Union General Data Protection Regulation. Clients are solely responsible for obtaining advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulations that may affect the clients' business and any actions the clients may need to take to comply with such laws and regulations.
The products, services, and other capabilities described herein are not suitable for all client situations and may have restricted availability. IBM does not provide legal, accounting, or auditing advice or represent or warrant that its services or products will ensure that clients are in compliance with any law or regulation.
Table of Contents
- GDPR
- Product Configuration for GDPR
- Data Life Cycle
- Data Collection
- Data Storage
- Data Access
- Data Processing
- Data Deletion
- Data Monitoring
- Capability for Restricting Use of Personal Data
GDPR
General Data Protection Regulation (GDPR) has been adopted by the European Union ("EU") and applies from May 25, 2018.
Why is GDPR important?
GDPR establishes a stronger data protection regulatory framework for processing of personal data of individuals. GDPR brings:
- New and enhanced rights for individuals
- Widened definition of personal data
- New obligations for processors
- Potential for significant financial penalties for non-compliance
- Compulsory data breach notification
Read more about GDPR:
Product Configuration - considerations for GDPR Readiness
The following sections provide considerations for configuring IBM MQ to help your organization with GDPR readiness.
Data Life Cycle
IBM MQ is a transactional message oriented middleware product that enables applications to asynchronously exchange application provided data. IBM MQ supports a range of messaging APIs, protocols and bridges for the purpose of connecting applications. As such, IBM MQ may be used to exchange many forms of data, some of which could potentially be subject to GDPR. There are several third-party products with which IBM MQ might exchange data. Some of these are IBM-owned, but many others are provided by other technology suppliers. The Software Product Compatibility Reports website provides lists of the associated software. For considerations regarding the GDPR readiness of a third-party product, you should consult that product's documentation. IBM MQ administrators control the way in which IBM MQ interacts with data passing through it, by the definition of queues, topics and subscriptions.
What types of data flow through IBM MQ?
As IBM MQ provides asynchronous messaging service for application data, there is no one definitive answer to this question because use cases vary through application deployment. Application message data is persisted in queue files (pagesets or the Coupling Facility on z/OS), logs and archives and the message may itself contain data that is governed by GDPR. Application provided message data may also be included in files collected for problem determination purposes such as error logs, trace files and FFSTs. On z/OS application provided message data may also be included in address space or Coupling Facility dumps.
The following are some typical examples of personal data that may be exchanged using IBM MQ:
- Employees of the customer (for example; IBM MQ might be used to connect the customer's payroll or HR systems)
- The customer's own clients' personal data (for example; IBM MQ might be used by a customer to exchange data between applications that relates to their clients, such as taking sales leads and storing data inside their CRM system).
- The customer's own clients' sensitive personal data (for example; IBM MQ might be used within industry contexts that require personal data to be exchanged, such as HL7-based healthcare records when integrating clinical applications).
In addition to application provided message data, IBM MQ processes the following types of data:
- Authentication Credentials (such as username and passwords, API keys, etc.)
- Technically Identifiable Personal Information (such as device IDs, usage based identifiers, IP address, etc. - when linked to an individual)
Personal data used for online contact with IBM
IBM MQ clients can submit online comments/feedback/requests to contact IBM about IBM MQ subjects in a variety of ways, primarily:
- Public comments area on pages in the IBM MQ area on IBM Developer
- Public comments area on pages of IBM MQ product documentation in IBM Knowledge Center
- Public comments in the IBM Support Forums
- Public comments in the IBM RFE Community on IBM Developer
Typically, only the client name and email address are used, to enable personal replies for the subject of the contact, and the use of personal data conforms to the IBM Online Privacy Statement.
Data Collection
IBM MQ can be used to collect personal data. When assessing your use of IBM MQ and your needs to meet with the demands of GDPR, you should consider the types of personal data which in your circumstances are passing through IBM MQ. You may wish to consider aspects such as:
- How does data arrive into your queue managers? (Across which protocols? Is the data encrypted? Is the data signed?)
- How is data sent out from your queue managers? (Across which protocols? Is the data encrypted? Is the data signed?)
- How is data stored as it passes through a queue manager? (Any messaging application has the potential to write message data to stateful media, even if a message is non-persistent. Are you aware of how messaging features could potentially expose aspects of the application message data passing through the product?)
- How are credentials collected and stored where needed by IBM MQ to access third-party applications?
IBM MQ may need to communicate with other systems and services which require authentication, for example LDAP. Where needed, authentication data (userids, passwords) is configured and stored by IBM MQ for its use in such communications. Wherever possible, you should avoid using personal credentials for IBM MQ authentication. Consider the protection of the storage used for authentication data. (See Data Storage below.)
Data Storage
When message data travels through queue managers, IBM MQ will persist (perhaps multiple copies of) that data directly to stateful media. IBM MQ users may want to consider securing message data whilst it is at rest.
The following items highlight areas where IBM MQ persists application provided data, which users may wish to consider when ensuring compliance with GDPR.
- Application Message Queues:
IBM MQ provides message queues to allow asynchronous data exchange between applications. Non-persistent and persistent messages stored on a queue are written to stateful media.
- File Transfer Agent Queues:
IBM MQ Managed File Transfer utilizes message queues to co-ordinate the reliable transfer of file data, files that contain personal data and records of transfers are stored on these queues.
- Transmission Queues:
To transfer messages reliably between queue managers, messages are stored temporarily on transmission queues.
- Dead-Letter Queues:
There are some circumstances where messages cannot be put to a destination queue and are stored on a dead-letter queue if the queue manager has one configured.
- Backout Queues:
JMS and XMS messaging interfaces provide a capability that allows poison messages to be moved to a backout queue after a number of backouts have occured to allow other valid messages to be processed.
- AMS Error Queue:
IBM MQ Advanced Message Security will move messages that don't comply with a security policy to the SYSTEM.PROTECTION.ERROR.QUEUE error queue in a similar way to dead-letter queuing.
- Retained Publications:
IBM MQ provides a retained publication feature to allow subscribing applications to recall a previous publication.
Read more:
- Logging: Making sure that messages are not lost
- MFT Agent queue settings
- Defining a transmission queue
- Use the dead-letter queue
- Handling poison messages in IBM MQ classes for JMS
- AMS error handling
- Retained publications
The following items highlight areas where IBM MQ may indirectly persist application provided data which users may also wish to consider when ensuring compliance with GDPR.
- Trace route messaging:
IBM MQ provides trace route capabilities, which record the route a message takes between applications. The event messages generated may include technically identifiable personal information such as IP addresses.
- Application activity trace:
IBM MQ provides application activity trace, which record the messaging API activities of applications and channels, application activity trace can record the contents of application provided message data to event messages.
- Service trace:
IBM MQ provides service trace features, which records the internal code paths through which message data flows. As part of these features, IBM MQ can record the contents of application provided message data to trace files stored on disk.
- Queue manager events:
IBM MQ can generate event messages that could include personal data, such as authority, command and configuration events.
Read more:
To protect access to copies of the application provided message data consider the following actions:
- Restrict privileged user access to IBM MQ data in the filesystem, for example restricting user membership of the 'mqm' group on UNIX platforms.
- Restrict application access to IBM MQ data via dedicated queues and access control. Where appropriate avoid unnecessary sharing of resources such as queues between applications and provide granular access control to queue and topic resources.
- Use IBM MQ Advanced Message Security to provide end-to-end signing and/or encryption of message data.
- Use file- or volume-level encryption to protect the contents of the directory used to store trace logs.
- After uploading service trace to IBM, we can delete your service trace files and FFST data if you are concerned about the contents potentially containing personal data.
Read more:
An IBM MQ administrator may configure a queue manager with credentials (username and password, API keys, etc.) for 3rd party services such as LDAP, Salesforce, etc. This data is generally stored in the queue manager data directory protected through file system permissions.
When an IBM MQ queue manager is created, the data directory is set up with group-based access control such that IBM MQ can read the configuration files and use the credentials to connect to these systems. IBM MQ administrators are considered privileged users and are members of this group so have read access to the files. Some files are obfuscated but they are not encrypted. For this reason, to fully protect access to credentials, you should consider the following actions:
- Restrict privileged user access to IBM MQ data, for example restricting membership of the 'mqm' group on UNIX platforms.
- Use file- or volume-level encryption to protect the contents of the queue manager data directory.
- Encrypt backups of the production configuration directory and store them with appropriate access controls.
- Consider providing audit trails for authentication failure, access control and configuration changes with security, command and configuration events.
Read more:
Data Access
IBM MQ queue manager data can be accessed through the following product interfaces, some of which are designed for access through a remote connection, and others for access through a local connection.
- IBM MQ Console [Only Remote]
- IBM MQ REST API [Only Remote]
- MQI [Local and Remote]
- JMS [Local and Remote]
- XMS [Local and Remote]
- IBM MQ Telemetry (MQTT) [Only Remote]
- IBM MQ Light (AMQP) [Only Remote]
- IBM MQ IMS bridge [Only Local]
- IBM MQ CICS bridge [Only Local]
- IBM MQ bridge for HTTP [Only Remote]
- IBM MQ MFT Protocol bridges [Only Remote]
- IBM MQ Connect:Direct bridges [Only Remote]
- IBM MQ Bridge to Salesforce [Only Remote]
- IBM MQ Bridge to Blockchain [Only Remote]
- IBM MQ MQAI [Local and Remote]
- IBM MQ PCF commands [Local and Remote]
- IBM MQ MQSC commands [Local and Remote]
- IBM MQ Explorer [Local and Remote]
The interfaces are designed to allow users to make changes to an IBM MQ queue manager and messages stored on it. Administration and messaging operations are secured such that there are three stages involved when a request is made;
- Authentication
- Role mapping
- Authorization
Authentication:
If the message or administrative operation was requested from a local connection, the source of this connection is a running process on the same system. The user running the process must have passed any authentication steps provided by the operating system. The user name of the owner of the process from which the connection was made is asserted as the identity. For example, this could be the name of the user running the shell from which an application has been started. The possible forms of authentication for local connections are:
- Asserted user name (local OS)
- Optional username and password (OS, LDAP or custom 3rd party repositories)
If the administrative action was requested from a remote connection, then communications with IBM MQ are made through a network interface. The following forms of identity may be presented for authentication via network connections;
- Asserted user name (from remote OS)
- Username and password (OS, LDAP or custom 3rd party repositories)
- Source network address (such as IP address)
- X.509 Digital Certificate (mutual SSL/TLS authentication)
- Security tokens (such as LTPA2 token)
- Other custom security (capability provided by 3rd party exits)
Role mapping:
In the role mapping stage, the credentials that were provided in the authentication stage may be mapped to an alternate user identifier. Provided the mapped user identifier is permitted to proceed (for example administrative users may be blocked by channel authentication rules), then the mapped user id is carried forward into the final stage when authorizing activities against IBM MQ resources.
Authorization:
IBM MQ provides the ability for different users to have different authorities against different messaging resources such as queues, topics and other queue manager objects.
Logging activity:
Some users of IBM MQ may need to create an audit record of access to MQ resources. Examples of desirable audit logs might include configuration changes that contain information about the change in addition to who requested it.
The following sources of information are available to implement this requirement:
- An IBM MQ queue manager can be configured to produce command events when an admin command has been run successfully.
- An IBM MQ queue manager can be configured to produce configuration events when a queue manager resource is created, altered or deleted.
- An IBM MQ queue manager can be configured to produce an authority event when an authorization check fails for a resource.
- Error messages indicating failed authorization checks are written to the queue manager error logs.
- The IBM MQ Console will write audit messages to its logs when authentication, authorization checks fail or when queue managers are created, started, stopped or deleted.
When considering these kind of solutions, IBM MQ users might want to give consideration to the following points:
- Event messages are non-persistent so when a queue manager restarts the information is lost. Any event monitors should be configured to constantly consume any available messages and transfer the content to persistent media.
- IBM MQ privileged users have sufficient privileges to disabled events, clear logs or delete queue managers.
For more information about securing access to IBM MQ data and providing an audit trail refer to the following topics:
Data Processing
Encryption using a Public Key Infrastructure:
We can secure network connections to IBM MQ to use TLS, which can also provide mutual authentication of the initiating side of the connection.
Use the PKI security facilities that are provided by transport mechanisms is the first step towards securing data processing with IBM MQ. However, without enabling further security features, the behavior of a consuming application is to process all messages delivered to it without validating the origin of the message or whether it was altered whilst in transit.
Users of IBM MQ that are licensed to use Advanced Message Security (AMS) capabilities can control the way in which applications process personal data held in messages, through the definition and configuration of security policies. Security policies allow digital signing and/or encryption to be applied to message data between applications.
It is possible to use security policies to require and validate a digital signature when consuming messages to ensure messages are authentic. AMS encryption provides a method by which message data is converted from a readable form to an encoded version that can only be decoded by another application if it is the intended recipient or the message and has access to the correct decryption key.
For more information about using SSL and certificates to secure your network connections, refer to the following topics in the IBM MQ V9 product documentation:
Data Deletion
IBM MQ provides commands and user interface actions to delete data which has been provided to the product. This enables users of IBM MQ with facilities to delete data which relates to particular individuals, should this be required.
- Areas of IBM MQ behavior to consider for complying with GDPR Client Data deletion
- Delete message data stored on an application queue by:
- Removing individual messages using messaging API or tooling or by using message expiry.
- Specifying that messages are non-persistent, held on a queue where non-persistent message class is normal and restarting the queue manager.
- Administratively clearing the queue.
- Delete the queue.
- Delete retained publication data stored on a topic by:
- Specifying that messages are non-persistent and restarting the queue manager.
- Replacing the retained data with new data or by using message expiry.
- Administratively clearing the topic string.
- Delete data stored on a queue manager by deleting the whole queue manager.
- Delete data stored by the Service trace commands by deleting the files in the trace directory.
- Delete FFST data stored by deleting the files in the errors directory.
- Delete address space and Coupling Facility dumps (on z/OS).
- Delete archive, backup or other copies of such data.
- Areas of IBM MQ behavior to consider for complying with GDPR Account Data deletion
- We can delete account data and preferences stored by IBM MQ for connecting to queue managers and 3rd party services by deleting (including archive, backup or otherwise replicated copies thereof):
- Queue manager authentication information objects that store credentials.
- Queue manager authority records that reference user identifiers.
- Queue manager channel authentication rules that map or block specific IP addresses, certificate DN's or user identifiers.
- Credentials files used by IBM MQ Managed File Transfer Agent, Logger and MQ Explorer MFT Plugin for authentication with queue manager and file servers.
- X.509 digital certificates that represent or contain information about an individual from keystores which may be used by SSL/TLS connections or IBM MQ Advanced Message Security (AMS).
- Individual user accounts from IBM MQ Appliance, including reference to those accounts in system log files.
- IBM MQ Explorer workspace metadata and Eclipse settings.
- IBM MQ Explorer password store as specified in the Password Preferences.
- IBM MQ Console and mqweb server configuration files.
- Salesforce connection data configuration files.
- blockchain connection data configuration files.
Read more:
- Configure the IBM MQ Bridge to Salesforce
- Configure IBM MQ for use with blockchain
- MFT and IBM MQ connection authentication
- Mapping credentials for a file server by using the ProtocolBridgeCredentials.xml file
- Configure IBM MQ Console users and roles
Data Monitoring
IBM MQ provides a range of monitoring features that users can exploit to gain a better understanding of how applications and queue managers are performing.
IBM MQ also provides a number a features that help manage queue manager error logs.
Read more:
Capability for Restricting Use of Personal Data
Use the facilities summarized in this document, IBM MQ enables an end-user to restrict usage of their personal data.
IBM MQ message queues should not be used as a permanent data store in the same way as a database, which is particularly true when handling application data that is subject to GDPR.
Unlike a database where data may be found through a search query, it can be difficult to find message data unless you know the queue, message and correlation identifiers of a message.
Provided messages containing an individual's data can be readily identified and located, it is possible using standard IBM MQ messaging features to access or modify message data.