This is the first Component of the IdML suite - a set of TDI Function Components, a Connector and a Parser designed to create and read IdML files (or books). The particular use of this Function Component is summed up in two points:
Since only the Open IdML FC can share books, it is also the one to free them from the static map, when they are no longer needed. In order to prevent data corruption (if two Open IdML FCs attempt to work with the same book) this Component places an exclusive lock on the associated book, thus preventing other Open IdML FCs to use it, until it is freed.
<?xml version="1.0" encoding="UTF-8"?> <idml:idml xmlns:idml="http://www.ibm.com/xmlns/swg/idml" xmlns:cdm="http://www.ibm.com/xmlns/swg/cdm" xmlns:xsi"http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/swg/idml idml.xsd"> <idml:source.IdMLSchemaVersion="0.8"> <cdm:process.ManagementSoftwareSystem id="id_value" CDMSchemaVersion="2.9.3" > <cdm:MSSName>ibm-cdm:///CDMMSS/identifier</cdm:MSSName> <cdm:Label>label_value</cdm:Label> <cdm:ProductName>product_value</cdm:ProductName> <cdm:ManufacturerName>manufacturer_value</cdm:ManufacturerName> </cdm:process.ManagementSoftwareSystem> </idml:source> <idml:operationSet opid="1"> OPERATIONS ... ... </idml:operationSet> </idml:idml>Each IdML book has two parts - one idml:source (also referred as heading) and one or more idml:operationSet-s.
The source part provides identifying information for the MSS (ManagementSoftwareSystem) that manages the resources listed in the IdML (in the operationSet part), while the operationSet(s) determine what should be done with these resources. The Open IdML FC is responsible for adding the MSS data (name, label, manufacturer, and so forth) to the IdML, when the book is opened. If the Open IdML FC has acquired a book that is already opened, it will not attempt to add the MSS source information, but instead log a message to the user and pass the execution to the next Component in the AssemblyLine.
An additional function of this FC is to guarantee that the opened IdML book will be closed (the ending tag, </idml:idml> is added). By default this is the function of the Close IdML Function Component, but can be performed by the Open IdML FC, in case of a normal Feed-Flow AssemblyLine.
In fact, there are two AssemblyLine types in which the IdML suite can participate:
The Open IdML FC supports two storage types for IdMLs:
Depending of the Open IdML FC's configuration, either one can be generated (in the case of file stored ones, the user needs to specify where they will be stored or the current Solution Directory will be used).
As advanced configuration options, the FC accepts a custom Book Name, so that the default book will not be used (as mentioned before). This is particularly handy if the default book is already in use. Then we can simply specify another name for the IdML Components you want to use, and the AssemblyLine will run successfully. Moreover, the name of the used book can be overridden in the Output Map of the FC, thus allowing new names to be provided at run time. This can be very useful, if the Flow section of the AssemblyLine (shown above) is responding to requests from different users (through some server Connector, for instance). In this case, if several users request an IdML using the default book, only the first will succeed, while the others will have to wait for him to finish or get an error message. To solve this, the user can be asked to provide a unique identifier in its request, which to be used as a book name by the IdML suite.
Another advanced option is to generate a Refresh IdML, as opposed to a regular Delta one. In the IdML terminology there are two types of IdMLs:
The FC can also use the DL Certification Tool to validate the generated IdML, either as a file or in-memory.
Another detail of this Function Component is how it handles the Common Data Model (CDM). Since it adds information for a MSS in the IdML, and MSSs are in essence ordinary Configuration Items, the user working with the Component needs to know the names of the needed MSS attributes (for example, cdm:MSSName, cdm:Hostname). This information is provided by the CDM. In fact, two separate sources of CDM meta-data are supported by this Component:
An important limitation is that all attributes are displayed, without signifying which are needed by any of the naming rules for the ManagementSoftwareSystem class (also known as identifying attributes). To ease usage, the Open IdML FC expects the attribute cdm:MSSName to be mapped, or if it is not present the combination of cdm:Hostname, cdm:Manufacturer and cdm:ProductName (these are the identifying attributes required by the two naming rules of the process.ManagementSoftwareSystem class). If neither of them is provided, an exception will be thrown.
When dealing with the CDM attributes needed by the Open IdML FC, there are several important points. The CDM Version parameter specifies the CDM meta-data used when generating the IdML. Thus, it is best to rely on the provided button for setting its value. The available CDM attributes for the process.ManagementSoftwareSystem will be listed when the Query Schema functionality is used. Most of them specify properties of the MSS, but several affect other aspects as well. For instance, the $id attribute will override the default MSS ID formed by concatenating the Application Code and Hostname attributes and will change the name of the generated IdML file.
Also, if the cdm:MSSName attribute is not explicitly provided, the cdm:Hstname, cdm:ManufacturerName and cdm:ProductName are used to set the unique name of the MSS. If any of them is not provided either, an Exception will be thrown by the Component.
Finally, the cdm:SourceToken attribute can be used to specify the source token of the created MSS. Through it the we can specify how the MSS can be contacted (for example it can contain an HTTP URL that can be used to connect to the MSS and query for additional information).
Output Schema
The Open IdML Function Component uses the following parameters:
The OPEN IdML Function component exposes a package for handling CDM meta-data - com.ibm.di.fc.idml.md. See the Javadocs for details.
This Component attempts to close an already opened IdML book. If the book has been already closed, it just logs a message to the user. If the closing procedure was executed successfully, the Close IdML Component can provide additional information of the book in the $idmlBook attribute, and the book is no longer statically shared. It will contain either the full path to the IdML document (in case it is stored as a file), or its actual contents (for the in-memory IdML snippets).
As all Components from the IdML suite, this FC accepts a Book Name parameter, and will look up a different book, depending on its value. The name of the used book can also be overridden at runtime by the $idmlBookName attribute.
The Close IdML Function Component cannot be used in normal Feed-Flow AssemblyLines, since it will attempt to close the IdML book on each iteration. For more information see section Open IdML Function Component.
Output Schema
Input Schema
The Close IdML Function Component uses the following parameter:
The Rolling IdML Function Component is used when there is a need to limit the size of the generated IdML files. Thus it is only applicable for IdML documents stored as files. For in-memory IdML books, its usage will lead to an Exception.
The FC will attempt to close the IdML document when any of its conditions is met and open a new one where the next data to be stored. It performs these operations in the context of one book, so to the user it appears that one book is repeatedly rolled, not that several books are opened and closed. Each of the resulting IdML files is a valid IdML document and contains the MSS data heading. Since the IdML file names are generated using the current time, this avoids the need to specify a name pattern for the rolled files.
When the FC has rolled the IdML book, it returns the name of the closed file as an attribute in its Input Map - $idmlFileName. Otherwise, the value of this attribute is null. If the IdML book was configured with the Validate option by the Open IdML FC, each rolled file will be validated upon its closure.
The Rolling IdML FC will attempt to perform its function based on two conditions:
the MSS itself is not considered a CI by this FC.
Since for the closing of the file several tags must be added, be aware that the size of the resulting file can exceed this value with several bytes.
The default value for the above conditions is 0, meaning unlimited artifact count and file size. If you add this Component to an AssemblyLine, but do not specify specific values for the rolling conditions, the FC will not attempt to split the generated IdML file.
As with all Components from the IdML suite, this FC accepts a Book Name parameter, and will look up a different book depending on its value. The name of the used book can also be overridden at runtime by the $idmlBookName attribute.
Output Schema
Input Schema
The Rolling IdML Function Component uses the following parameters:
This Connector adds an artifact - either a Configuration Item (CIs) or a Relationship, to the IdML book. Thus, in its configuration the user needs to specify the desired artifact type and the class of the added item. In order to ease this task the user can directly discover the class names supported by the used CDM.
As with the Open IdML FC, the CDM meta-data can be retrieved from a local JAR file or by connecting to a remote IT Registry system. We can choose between these options in the Advanced section of the Connector's configuration panel. Other usability features are the ability to check the version of the used CDM, and test the connection to the remote IT Registry system (if it is being used). For the IT Registry case, we can also provide information how to connect to it (for example, JDBC URL, Driver, username and password). If these fields are left blank the default values specified in the etc/it_registry.properties file will be used.
The selected CDM source also affects the attributes displayed when querying the schema of the Output Map of the Connector. For instance, if the IT Registry CDM is used, when listing a CI's attributes you will get not only the specific attributes of its class (as is when the JAR meta-data is used), but also those of its parent classes. This leads to a somewhat slower response than when the JAR definitions are used. This feature is most notable when listing the Relationship types in the Connector's configuration panel. If the IT Registry CDM is used, you are able to see additional information, specifying which classes of CIs can act as sources and which as targets of the chosen Relationship. This can be very useful if you are unaware of the restrictions of the needed Relationship, but is a fairly slow operation which requires much more time than if the JAR CDM is used (then only the Relationship types will be listed, without the class restrictions).
The naming rules limitation is also valid for this Connector. You are able to see all attributes of the chosen CI class but can not determine which ones are part of a naming rule (also known as identifying attributes) and what rule exactly. Thus, in order to satisfy the required attributes of a CI, must find information for them in the CDM.
As with all Components of the IdML suite, this Connector accepts a Book Name parameter, and will look up a different book, depending on its value. The name of the used book can also be overridden at runtime by the $idmlBookName attribute.
Another important parameter which can be provided in the Connector's Output Map is the $operation attribute. It determines what operation will be performed with the specified CI/Relationship, when the IdML file is imported into a CMDB. It can be either added to the CMDB (the CREATE operation), updated (MODIFY), or removed (DELETE). These values - CREATE, MODIFY and DELETE (case insensitive) can be set to the $operation attribute. Note that if the used IdML is opened as a Refresh one (see section Open IdML Function Component) only the CREATE operation is supported and passing another value will cause an Exception. If you specify no value for the $operation attribute, the CREATE value will be used by default.
The other option for setting the IdML operation is to pass a delta enabled work entry to the Connector. Since the IdML Ci and Relationship Connector is "delta aware", it will interpret the delta operation set to the entry and map its value to the IdML operations. The mapping is fairly straightforward:
Delta operation | IdML operation |
---|---|
ADD (Entry.OP_ADD) | CREATE |
MODIFY (Entry.OP_MOD) | MODIFY |
DELETE (Entry.OP_DEL) | DELETE |
Keep in mind that the provided delta operation will ALWAYS override the value of the $operation attribute.
When you query the schema of the Output Map of this Connector, the attributes of the chosen CI/Relationship class will be listed. For the CI case they will include the $id and cdm:SourceToken attributes. The $id permits you to override the default value of the CI's ID, and supply a custom one (any string can be used as an ID). The default ID is an integer identifier that is incremented each time an artifact is added to the IdML book. This ensures the ID's uniqueness, while if you provide a custom one, guarantee it will not overlap an existing one. If the IdML book contains several CIs with the same ID, this can cause problems when importing it to a CMDB. Since the ID is used to determine the CIs participating in a Relationship, having several of them with the same identifier will produce corrupt results. Such a problem can be detected earlier by enabling the book certification feature of the Open IdML FC.
The $id attribute will be returned by the Input Map of the Connector as well, so that we can map it directly to another IdML Ci and Relationship Connector, configured to add a Relationship.
The cdm:SourceToken attribute can be used to provide a unique identifier for the added CI. While the $id attribute is unique in the IdML book, the cdm:SourceToken must be unique in the whole realm of the MSS. It can later be used to uniquely identify and locate a CI by the IT Registry.
Querying the schema of a Relationship using the IdML Ci and Relationship Connector will return only two attributes:
Both of these attributes must be provided when defining a Relationship, otherwise an exception will be thrown.
There is one additional type of attributes supported by the IdML Ci and Relationship Connector: extension attributes. As opposed to regular ones, these are stored in a sub-element of the artifact's element in the IdML XML document and are used to provide additional information for each Configuration Item. Since they do not need to comply with the CDM schema of that CI, users can rely on them to map any specific data. Such attributes are distinguished by their names, which follow this template: cdm:extattr:AttributeName (for example, cdm:extattr:Testing).
The IdML Ci and Relationship Connector provides additional functionality when dealing with IdML snippets. This can be especially useful when an AssemblyLine is configured to send the generated IdML using an HTTP Server Connector. In the normal case, the whole IdML should be accumulated, before it can be mapped to the $idmlBook attribute of the Close IdML FC and returned to the caller. However, the IdML Ci and Relationship Connector provides a method: resetBook(), that can be used to retrieve the current content of the IdML book and send it using the HTTP Server Connector. This way, the response will be chunked and the caller will not have to wait for the whole IdML to be completed, but will get the data as soon as it is ready. If this method is called for an IdML book stored as a file, null is returned.
Output Schema
Input Schema
The IdML Ci and Relationship Connector uses the following parameters:
The purpose of the IdML Parser is to parse the contents of an IdML file. It can only be used for reading IdML documents, while the Open IdML FC, IdML Ci and Relationship Connector, Close IdMl FC and Rolling IdML FC should be used for creating them. It relies on the XML Parser for handling the IdML files and snippets.
Please see section Open IdML Function Component to see more information about the schema of the IdML XML.
The MSS section of an IdML XML is parsed only during the first iteration of the Component and the received MSS data is returned on every subsequent iteration. With each iteration the Parser also reads a single artifact (either a CI or a Relationship) from the operationSet section, parse its attributes and map them to the returned entry. Along with regular CDM attributes like cdm:Manufacturer, cdm:SourceToken, and so forth, the Parser reads the artifact's extension attributes and returns them as well. The only difference is that instead of cdm:AttributeName, they will be named cdm:extattr:ExtendedAttributeName.
The input file/snippet passed to the IdML Parser can be either a Delta or a Refresh IdML. For more information about IdML types refer to section Open IdML Function Component. Its type will determine the value of the $idmlType attribute in the Input Map of the Parser. The supported values are DELTA and REFRESH (case insensitive).
Depending on the type of artifact read by the Parser, the $artifactType attribute can be either CI (for Configuration Items) or Relationship. Both values are case insensitive.
Similarly, the class type will be mapped to the $classType attribute in the Input Map. It will have values like cdm:ComputerSystem, cdm:OperatingSystem, and so forth for CIs, and cdm:installedOn, cdm:runsOn and so forth for Relationships.
The value of the $operation attribute of the input map can be CREATE, MODIFY or DELETE (case insensitive). It is determined based on the operation element of the IdML document where the artifact was read from.
Output Schema
This parser does not have any Output Schema.
Input Schema
The Parser has the following parameters:
The Data Cleanser Function Component relies on the IT Registry to clean the value of a string provided as its $inputString attribute. However, before attempting this operation, we need to specify the type of CDM attribute that this value belongs to (for example, cdm:Manufacturer, cdm:ProductName, and so forth). By pressing the Select... button in the configuration panel of the FC, we can easily see a list of all the CDM attribute types supported by the IT Registry.
If no cleansing rule exists for the provided string, it will be returned unmodified in the Input Map of the FC.
Output Schema
Input Schema
The Data Cleanser Function Component's Configuration panel uses the following parameter:
The Init IT registry Function Component registers a Management Software System (MSS) to the IT registry database and returns the GUID of registered MSS as an Input Map attribute. However, the GUID is wrapped as a com.ibm.di.fc.itregistry.ConfigurationItemId object. The reason is that not work directly with GUIDs or view their contents. Instead, we should simply map the ConfigurationItemId-s to other Components from the IT registry suite, when this is needed (for example, when registering a Relationship).
This Component will bypass IdML files, commonly used for transferring data between CMDBs. Instead of creating such a file, which then must be bulk loaded in the IT Registry, this Component can directly import its data to the IT Registry database. As its equivalent in the IdML suite, the Init IT registry FC will rely on static sharing of a book (in this case meaning, sessions of operations performed on the IT Registry DB). By using it, the IT Registry Components will share information about the performed operations. It includes a flag denoting whether a Refresh operation should be performed by the Components using the book (meaning that only artifacts registered by them should be kept in the database and all older ones should be removed) and a timestamp marking the moment when the Init IT registry FC was executed (used for separating the old from the new artifacts in the database, during a Refresh). To determine which book should be used, we can either provide a value for the Book Name field in the configuration panel of the FC or through the $itRegistryBookName attribute in its Output Map.
Since this Component connects to the IT Registry database, it relies on the values specified in the etc/it_registry.properties file (JDBC URL, JDBC Driver, username and password). To allow easier modifications of these properties each Component has fields for overwriting their values in its Advanced configuration section. For more information on the IT Registry properties, see The it_registry.properties file.
The flag Use IT registry for CDM in the configuration panel of the FC is used to indicate whether to use the local JAR or an IT Registry system for retrieving CDM's meta-data definitions.
By pressing the Connect button in the Output Map of the FC, you can populate its Output Schema. This way, all the CDM attributes supported by the process.ManagementSoftwareSystem class will be displayed and we can directly map them (even without exactly knowing their names).
The naming rules limitation is valid for this Function Component. We will be able to see all attributes of the ManagementSoftwareSystem but can not determine which ones are part of a naming rule and what rule exactly. To ease usage, the Init IT registry FC will expect the attribute cdm:MSSName to be mapped, or if it is not present the combination of cdm:Hostname, cdm:Manufacturer and cdm:ProductName (these are the identifying attributes required by the two naming rules of the process.ManagementSoftwareSystem class). If neither of them is provided, an exception will be thrown.
Output Schema
Input Schema
The Init IT Registry Function Component uses the following parameters:
All JDBC-related parameters derive their default values from the etc/it_registry.properties file.
The IT registry Ci and Relationship Connector will add, update, delete, search or iterate CIs (Configuration Items) and the Relationships between them. For performing all listed operations, this Connector works directly with the IT Registry database. By using the Artifact Type parameter in the configuration panel of the Connector, we can specify whether its operation will be performed on Configuration Items or Relationships. Furthermore, we do not need to know the exact name of the artifact's class, and can directly discover the ones supported by the CDM (by pressing the Select... button in the configuration panel).
As with the IdML CI and Relationship Connector, the CDM meta-data can be retrieved from the local jar file or by connecting to a remote IT Registry system. We can choose between these options in the Advanced section of the Connector's configuration panel. Other usability features are the ability to test the connection to the remote IT Registry system (if it is being used). For the IT Registry case, the user can also provide information how to connect to IT Registry (for example, JDBC URL, Driver, username and password). If these fields are left blank the default values specified in the etc/it_registry.properties file will be used.
The selected CDM source also affects what attributes are displayed when querying the schema of the Output Map and Input Map of the Connector. For instance, if the IT Registry CDM is used, when listing a CI's attributes, we will get not only the specific attributes of its class (as is when the JAR meta-data is used), but also those of its parent classes. This of course leads to a bit slower response than when the JAR CDM is used. This is most notable when listing the Relationship types in the Connector's configuration panel. With the IT Registry CDM, you will be able to see additional information, specifying which classes of CIs can act as sources and which as targets of the chosen Relationship. This can be very useful if you are unaware of the restrictions for the needed Relationship, but is a fairly slow operation which requires much more time than if the JAR CDM is used (then only the relationship types will be listed, without the class restrictions).
The naming rules limitation is also valid for this Connector. You are able to see all attributes of the chosen artifact class but can not determine which ones are part of a naming rule (also known as identifying attributes) and what rule exactly. Thus, in order to satisfy the required attributes of a CI, we need to find information for them in the CDM.
This Connector accepts a Book Name parameter, and will look up a different book, depending on its value. The name of the used book can also be overridden at runtime by the $itRegistryBookName attribute.
All GUIDs handled by this Connector should be wrapped as com.ibm.di.fc.itregistry.ConfigurationItemId objects. This is valid for both GUIDs that are passed to the Connector and those that are returned by it. This way, we can still map them in AssemblyLines, while not seeing the specific GUID format.
When the IT registry Ci and Relationship Connector is in CallReply mode, it is capable of registering/modifying both Configuration Items and Relationships along with their attributes. However, when users register CIs they should provide only identifying attributes, as IT registry does not support non-identifying ones in its current release.
For working properly in CallReply mode, this Connector depends upon the Init IT registry FC for details about the operation it should perform. This data is accessed through a shared book and includes a flag determining whether the Connector should perform a Refresh operation and a timestamp, used for distinguishing which artifacts should be "refreshed" and which not. See section Open IdML Function Component for details on the Refresh operation.
For this mode, an important parameter which is provided in the Connector's Output Map is the $operation attribute. It determines what operation will be performed with the specified CI/Relationship, when it is registered to the IT Registry database. It can be either added to the IT Registry database (the CREATE operation), updated (MODIFY), or removed (DELETE). These values - CREATE, MODIFY and DELETE (case insensitive) can be set in the $operation attribute. Note that if the used book is opened as a Refresh one (see the Init IT Registry Function Component for information) only the CREATE operation is supported and passing any other value will cause an Exception. If no value is specified for the $operation attribute the CREATE value will be used by default.
The other option for setting the IT Registry operation is to pass a delta enabled work entry to the Connector. Since the IT Registry Ci and Relationship Connector is "delta aware", it will interpret the delta operation set to the entry and map its value to the IdML operations. The mapping is fairly straightforward:
Delta operation | IdML operation |
---|---|
ADD (Entry.OP_ADD) | CREATE |
MODIFY (Entry.OP_MOD) | MODIFY |
DELETE (Entry.OP_DEL) | DELETE |
Keep in mind that the provided delta operation will ALWAYS override the value of the $operation attribute.
Due to limitations of the current version of the IT Registry, the UPDATE operation is not supported by the IT Registry Ci and Relationship Connector. If you try to provide it, an exception will be thrown.
When you query the schema of the Output Map of this Connector, the attributes of the chosen CI/Relationship class will be listed.
Querying the schema of a Relationship using the IT Registry Ci and Relationship Connector will return only two attributes:
If any of these attributes is not provided, an Exception will be thrown.
To delete a Configuration Item, a set of identifying attributes must be passed to this Connector along with a DELETE value for the $operation attribute. If the Configuration Item is owned by several ManagementSoftwareSystem-s, it will not be deleted from the database. Instead, only the associations between the CI and the ManagementSoftwareSystem will be removed, thus releasing it from the MSS context. Similarly, to release/delete a Relationship for a specified MSS, the MSS's GUID and the GUIDs of the source and target CIs are required.
To facilitate reading of CIs, Iterator mode is provided. When the IT registry Ci and Relationship Connector is in this mode, it returns Configuration Items based on three separate criteria:
Either the Attribute Filter or Date Filter can be used by a single Connector. By default the Attribute Filter is enabled, but we can switch to Date Filter by checking the Enable Date Filter option.
This parameter can be used independently of the Attribute Filter and Date Filter.
Lookup mode can be used to return a Configuration Item with all of its identifying attributes as stored in the IT Registry database. The search is performed using the conditions provided in the Link criteria of the Connector, If they are not specific enough, multiple Configuration Items can be returned. In this case, we should enable the On Multiple Entries hook and add some custom logic for handling the situation.
When specifying the Link Criteria, bear in mind to use only AND logical operations between the conditions and to rely only on the EQUALS operator.(for example, cdm:Model=T61p AND cdm:Manufacturer=IBM is a valid criteria). As in Iterator mode, we can further limit the returned CIs by providing a value for the MSS Name filter.
Output Schema
Input Schema
The IT Registry Ci and Relationship Connector uses the following parameters:
The IdML Components, Open IdML Function Component and IdML Ci and Relationship Connector, need information how to connect to a IT Registry system, in order to retrieve the needed CDM meta-data. The same information is needed for the IT Registry Components, Init IT Registry Function Component and IT Registry Ci and Relationship Connector, which use it for registering information in the IT Registry database, as well as for retrieving CDM meta-data. Therefore, all of these Components have configuration fields in their panels, where we can specify the IT Registry information.
However, since in most cases you rely on only one IT Registry system most of the time, it is easier if there is a common place for this data. Therefore, TDI provides a properties file, TDI_solution_dir/etc/it_registry.properties in which we can place the JDBC URL, JDBC Driver, Username and Password properties and they will be used by all IdML and IT Registry Components as default values.
The format of the it_registry.properties file is:
it_registry.jdbcUrl= it_registry.jdbcDriver= it_registry.dbUsername= it_registry.dbPassword=
If different values are needed for any of these properties, you can either edit the it_registry.properties file (applying the change for all of the Components) or set the new values locally in the configuration panels of those Components needing them.