If you are writing Java code that will be executed in the TDI Server JVM (for example a new Connector, or a Java class that we will access through scripting) and we want to execute Server API calls, you'll need a local Server API session.
We can obtain a local Server API session by calling:
import com.ibm.di.api.APIEngine; import com.ibm.di.api.local.Session; ... Session session = APIEngine.getLocalSession();
getLocalSession() is a static method of the com.ibm.di.api.APIEngine class. It creates and returns a new com.ibm.di.api.local.Session object. This session returned has admin rights and can execute all Server API calls.
Users can get access to the Server API from a scripting context (for example from AssemblyLine hooks) by calling the session script object. TDI Server registers session objects by calling com.ibm.di.api.APIEngine.getLocalSession() method.
A client application that uses the remote Server API would first need to connect to the TDI Server and obtain a Server API Session.
Use the following Java code to lookup the RMI SessionFactory object and obtain a Server API Session:
import com.ibm.di.api.remote.Session;
import com.ibm.di.api.remote.SessionFactory;
...
SessionFactory sessionFactory = (SessionFactory) Naming.lookup("rmi://<TDI_Server_host>:
<TDI_Server_RMI_port>/SessionFactory");
Session session = sessionFactory.createSession();
We need to replace TDI_Server_host and TDI_Server_RMI_port with the host and the RMI port of the TDI Server; for example:
Naming.lookup("rmi://127.0.0.1:1099/SessionFactory")
The calls provided by the local and remote Session objects are identical. All examples below assume that we have already obtained a session and will operate in a remote context. In other words, the remote versions of the Server API interfaces will be used.
The Config Instance represents a configuration loaded on the TDI Server and the associated Server object. Each AssemblyLine is running in the context of a Config Instance. Through a Config Instance we can query the configuration of AssemblyLines, Connectors, Parsers, Functional Components, start AssemblyLines, get access to running AssemblyLines and query their log files.
We can obtain access to all Config Instances running on the TDI Server by executing the following piece of code:
ConfigInstance[] configInstances = session.getConfigInstances();
for (int i=0; i<configInstances.length; i++) {
// do something with configInstances[i]
}
The getConfigInstances() method will return an array with Config Instance Server API objects representing all Config Instances running on the Server.
In order to load a new configuration on the TDI Server we need to start a new Config Instance, pointing it to the XML configuration file:
ConfigInstance configInstance = session.startConfigInstance("testconfig.xml");
This loads the testconfig.xml configuration file and start a new Config Instance object associated with that configuration. Once you get that Config Instance object we can use it to change the configuration itself, start AssemblyLines or stop the Config Instance on the Server when you no longer need it.
If we need to start multiple configuration instances from a single configuration file (for example if we want to use a different set of properties for each instance), provide a unique Run Name for each instance:
ConfigInstance configInstance = session.startConfigInstance("testconfig.xml", true, null, "myrunname", "mystore=mynewstore.properties");
The above call will start a new configuration instance from the "testconfig.xml" file with a Run Name "myrunname". That Run Name will be used as the id of the configuration instance. Furthermore, the property store "mystore" of the instance will be redirected to load its contents from the "mynewstore.properties" file.
Assuming that we have a reference to the Config Instance Server API object, we can stop the Config Instance by calling:
configInstance.stop();
For a reference to the Config Instance object, we have the following options:
If a config instance has not fully loaded the configuration file when a server API call is made, it returns a null object. In TDI 7.1, a time out interval is configurable by means of a property - api.config.timeout . This is set to 2 mins by default. An exception will be thrown if the config has not been loaded within this time interval.
The Config Instance is representing a configuration loaded on the TDI Server and the associated Server object. Each AssemblyLine is running in the context of a Config Instance. Through a Config Instance we can query the configuration of AssemblyLines, Connectors, Parsers, Functional Components, start AssemblyLines, get access to running AssemblyLines and query their log files.
When a configuration file is loaded by the TDI Server, it becomes a running configuration instance. Each configuration instance has its own configuration id. No two configuration instances running at the same time are allowed to have the same configuration id (a configuration id uniquely identifies a running configuration instance within the TDI Server).
When a configuration instance is started off a configuration file, the TDI Server first checks if the configuration file has a defined Solution Name (a configuration field of the Solution Interface configuration screen). If the Solution Name is present and non-emtpy, the Server uses this name as the configuration instance id. If the Solution Name is missing or empty, the TDI Server automatically generates a configuration id.
For example if a configuration file with an absolute file name "C:/IBM/TDI/configs/rs.xml" is loaded into the TDI Server and the file has a Solution Name set to "mysoluname", then the id of the spawned configuration instance is "mysoluname". If the same configuration had no Solution Name defined, the configuration instance id would be something like "C__IBM_TDI_configs_rs.xml".
The clients of the TDI Server API must perceive the automatically generated configuration instance ids as transparent entities - they must not try to guess how these ids are generated, because the algorithm is subject to change in the future. The only guarantee is that if a configuration instance once existed under some automatically generated configuration id, then certain artifacts such as tombstones and system logs can be accessed later using the same configuration id. There is no guarantee, however, that if the same configuration file is run again, the newly spawned configuration instance will have the same automatically generated id as before.
Generally if the client specifies nothing more than the path to the configuration file while starting a configuration instance, the configuration id is based solely on the attributes of the configuration file (the Solution Name, if any, or the absolute file name). As a result, if no additional information is provided, a configuration file can be loaded as a configuration instance only once (otherwise there is a conflict of configuration instance ids).
If it is necessary to load multiple configuration instances from the same configuration file, the client needs to provide a unique Run Name for each of the instances. If a run name is supplied when starting a configuration instance, that run name is used as the configuration instance id of the instance. Consequently a Run Name must not coincide with any of the ids of already running configuration instances.
Each Solution Name and each Run Name must be a valid file name on the platform on which the TDI Server is currently running. The reason for this restriction is that the configuration instance id (which derives from the Solution Name or the Run Name) is used when storing certain configuration-instance-specific information, such as the System Logs. To avoid file system problems, TDI forbids the following symbols to appear inside a Run Name or a Solution Name: \ / : * " < > | ?
Notes:
Another consequence is that Solution Names and Run Names must appear in the User Registry instead of absolute file-system paths, for configuration instances which use such names.
Suppose that we have a configuration file with absolute file name "C:/IBM/TDI/configs/rs.xml". The table below describes how to refer to configuration instances launched from that file in the User Registry; the table takes into account whether the configuration file has a Solution Name and whether the configuration instance is started with a Run Name:
| Solution Name | Run Name | Section in the User Registry |
|---|---|---|
| - | - | [CONFIG]:C:/IBM/TDI/configs/rs.xml |
| - | myrunname | [CONFIG]:myrunname |
| mysoluname | - | [CONFIG]:mysoluname |
| mysoluname | myrunname | [CONFIG]:myrunname |
It is important to note that permissions in the User Registry are assigned per configuration instance and not per configuration file.
Only the configuration files placed in this folder can be edited using the Server API.
In TDI 6.1 and previous releases starting a config instance as well as the check-in/check-out functionality of the Server API required the URL (file path) of the config file to be provided. This is no longer necessary in TDI 7.1, because the same Server API interface methods can be passed the corresponding Solution Name instead. This is a user convenience as Server API clients like the AMC and CLI now accept user-friendly Solution Names instead of cryptic config file paths.
The config file path has a higher priority than the Solution Name. This means that if the method for starting a config instance (for example) is passed a string (either a config file path or the corresponding Solution Name) and it is a valid config file path then the method treats this value as referring to this config file. If there is a config file and a Solution Name which are identical as strings then the config file path takes precedence. This behavior ensures backward compatibility with previous versions of TDI when there were no Solution Names.
At TDI Server startup time, only Configs residing in the TDI configs folder (as specified by the global.properties or solution.properties file parameter api.config.folder) as well as those residing in the Solution Directory can be referred to by their Solution Name.
At startup the TDI Server scans the configs folder (specified by the api.config.folder property in global.properties or solution.properties) for the Solution Names of the config files located in the configs folder. The Server then builds an internal map which maps Solution Names to config file paths so that Solution Names can be used in place of config file paths.
The following rules are used when scanning the configs folder:
This would lead to the following situations depending on how the TDI server is started:
| TDI server in Secure mode | TDI server in Normal mode |
|---|---|
| PKI-encrypted config - solution name displayed (if existing) | PKI-encrypted config - file name displayed (if extension is .cfg or .xml) |
| unencrypted config - file name displayed (if extension is .cfg or .xml) | Unencrypted config -TDI solution name displayed (if existing) |
| password-encrypted config - file name displayed (if extension is .cfg or .xml) | Password-encrypted config - file name displayed (if extension is .cfg or .xml) |
| non-TDI config file (other, text or binary) - file name displayed (if extension is .cfg or .xml) | Non-TDI config file (other, text or binary) - file name displayed (if extension is .cfg or .xml) |
We do not recommend that you store files other than valid Config files (XML format or cfg file format) in the configs folder. During attempts to parse any non-config file, errors may be reported and the file is ignored - this does not affect the proper operation of the Server.
Assuming that you already have a reference to the Config Instance object, obtain the MetamergeConfig object representing the configuration data structure for the whole Config Instance and then get the available AssemblyLines:
import com.ibm.di.config.interfaces.MetamergeConfig;
import com.ibm.di.config.interfaces.MetamergeFolder;
import com.ibm.di.config.interfaces.AssemblyLineConfig;
...
MetamergeConfig configuration = configInstance.getConfiguration();
MetamergeFolder configFolder =
configuration.getDefaultFolder(MetamergeConfig.ASSEMBLYLINE_FOLDER);
String[] assemblyLineNames = configFolder.getNames();
if (assemblyLineNames != null) {
for (int i=0; i<assemblyLineNames.length; i++) {
System.out.println(assemblyLineNames[i]);
// get the AssemblyLine configuration object
AssemblyLineConfig alConfig =
configuration.getAssemblyLine(assemblyLineNames[i]);
// do something with alConfig ...
This block of code prints to the standard output the names of all AssemblyLines in the configuration and demonstrates how to get the AssemblyLine configuration objects. We can use the AssemblyLine configuration object to get more detailed information, such as which Connectors are configured in the AssemblyLine, their parameters, etc.
Note that the MetamergeConfig, MetamergeFolder and AssemblyLineConfig interfaces are not part of the Server API interfaces. They are part of the TDI configuration driver (see the import clauses in the example) and they are not remote objects. When configInstance.getConfiguration() is executed the MetamergeConfig object is serialized and transferred over the wire. Your code will then work with the local copy of that object.
We can get the active AssemblyLines either for a specific Config Instance or for all active AssemblyLines on the TDI Server for all running Config Instances.
AssemblyLine[] assemblyLines = configInstance.getAssemblyLines();
for (int i=0; i
for (int i=0; i<assemblyLines.length; i++) {
System.out.println(assemblyLines[i].getName());
// do someting with assemblyLines[i]
}
AssemblyLine[] assemblyLines = session.getAssemblyLines();
for (int i=0; i<assemblyLines.length; i++) {
System.out.println(assemblyLines[i].getName());
// do someting with assemblyLines[i]
// which Config Instance this AssemblyLine belongs to?
ConfigInstance alConfigInstance = assemblyLines[i].getConfigInstance();
}Note that this is executed at the session level and not for
a particular Config Instance. If we need to know which Config Instance
a running AssemblyLine belongs to, we can get a reference to the
parent Config Instance object through the AssemblyLine object.
You can use the AssemblyLine Server API object to get various AssemblyLine properties, the AssemblyLine configuration object, AssemblyLine log, AssemblyLine result Entry as well as stop the AssemblyLine.
We can start an AssemblyLine through the Config Instance object to which the AssemblyLine belongs. We need to know the name of the AssemblyLine we want to start:
AssemblyLine assemblyLine = configInstance.startAssemblyLine("MyAssemblyLine");
You also receive a reference to the newly started AssemblyLine instance.
The Server API provides a mechanism for manually running an AssemblyLine. In manual mode the AssemblyLine is not running in its own thread. Instead, when we start the AssemblyLine, it is only initialized. Iterations on the AssemblyLine are done in a synchronous manner when the executeCycle() method of the AssemblyLine object is called. This call blocks the current thread and when the AssemblyLine iteration is done it returns the result Entry object.
The following code will start the TestAL AssemblyLine in manual mode and execute three iterations on it. The result Entry from each iteration is printed to the standard output:
AssemblyLineHandler alHandler = configInstance.startAssemblyLineManual("TestAL", null);
Entry entry = null;
for (int i=0; i<3; i++) {
entry = alh.executeCycle();
System.out.println("TestAL entry: " + entry);
}
alHandler.close();
The startAssemblyLineManual(String aAssemblyLineName, Entry aInputData) method of the Config Instance object starts an AssemblyLine in manual mode and returns an object of type com.ibm.di.api.remote.AssemblyLineHandler. Through this object we can manually iterate through the AssemblyLine, we can pass an initial work Entry and various Task Call Block parameters, we can get a reference to the AssemblyLine Server API object and we can terminate the AssemblyLine when you are done with it.
We can imitate the AssemblyLine runtime behavior by calling executeCycle() until it returns NULL.
When starting an AssemblyLine through the Server API we can register a specific AssemblyLine listener that will receive notifications on each AssemblyLine iteration, delivering the result Entry, and also when the AssemblyLine terminates. Through this mechanism we can start an AssemblyLine from a remote application and easily receive all Entries produced by the AssemblyLine. The AssemblyLine listener will also deliver all messages logged during the execution of the AssemblyLine.
Your listener class must implement the com.ibm.di.api.remote.AssemblyLineListener interface (or com.ibm.di.api.local.AssemblyLineListener for local access).
The methods specify are:
A sample AssemblyLine listener class that only prints to the standard output all Entries received and all AssemblyLine log messages might look like this:
import com.ibm.di.api.DIException;
import com.ibm.di.api.remote.AssemblyLineListener;
import com.ibm.di.entry.Entry;
import java.rmi.RemoteException;
public class MyRemoteALListener implements AssemblyLineListener {
public void assemblyLineCycleDone(Entry aEntry)
throws DIException, RemoteException
{
System.out.println("AssemblyLine iteration: " + aEntry.toString());
System.out.println();
}
public void assemblyLineFinished()
throws DIException, RemoteException
{
System.out.println("AssemblyLine terminated.");
System.out.println();
}
public void messageLogged(String aMessage)
throws DIException, RemoteException
{
System.out.println("AssemblyLine log message: " + aMessage);
System.out.println();
}
}
Once we have implemented your AssemblyLine listener class, you need to instantiate a listener object and pass it when starting the AssemblyLine:
MyRemoteALListener alListener = new MyRemoteALListener();
configInstance.startAssemblyLine("TestAL", null,
AssemblyLineListenerBase.createInstance(alListener,true), true);
The startAssemblyLine(String aAssemblyLineName, Entry aInputData, AssemblyLineListener aListener, boolean aGetLogs) method specifies the name of the AssemblyLine, an initial work Entry, the listener object and whether we want to receive log messages - when aGetLogs is false, the messageLogged(String aMessage) listener method will not be called by the Server API.
When you are registering a listener in a remote context, we have to wrap your specific listener in an AssemblyLine Base Listener class - this is necessary to provide a bridge between your custom listener Java class that is not available on the Server side and the Server API notification mechanism. A base listener class is created by calling the static createInstance(AssemblyLineListener aListener, boolean aSSLon) method of the com.ibm.di.api.remote.impl.AssemblyLineListenerBase class. We need to provide the object representing your listener class and specify whether SSL is used for communication with the Server or not ( specify how the Server API is configured on the Server side - otherwise the communication for that listener will fail).
By setting the simulation flag of an AssemblyLine to true you specify that the components behavior in the AssemblyLine will be simulated. The simulation functionality is described in more detail in IBM TDI V7.1 Users Guide; here we will only show how to set the simulation flag:
usertcb.setProperty(com.ibm.di.server.AssemblyLine.TCB_SIMULATE_MODE, Boolean.TRUE);
Where "usertcb" is a TaskCallBlock object, and then start the AssemblyLine using this object.
We need a reference to the AssemblyLine object in order to stop it. We can keep the reference to the AssemblyLine object from when we started the AssemblyLine or we can iterate through all running AssemblyLines and find the one we need. Execute the following line of code to stop the AssemblyLine:
assemblyLine.stop();
A TDI Server property api.config.folder is available in the TDI Server configuration file global.properties - it specifies a folder on the local disk. The Server API will provide calls for browsing and loading configurations placed in this folder or its subfolders. For example:
api.config.folder=configs
This means that all configuration files placed in "<TDI_root>/configs" and its subfolders are eligible for browsing and loading through the Server API (locally and remotely).
The Server API provides new calls for browsing the files and folders in the folder specified by the api.config.folder property.
In TDI 6.0 configurations can be edited only after the corresponding Config Instance has been started on the TDI Server. Then there are API calls for getting the Config object, setting the Config object back (probably modified) and saving the configuration on the disk.
TDI 7.1 will not allow modification of the Config object of an active Config Instance. Server API users will still be able to get the Config object for an active Config Instance, but the following calls for setting the Config object and saving it on the disk will throw an exception when executed on a normal running Config Instance:
When a configuration is loaded for editing with a temporary Config Instance it will be able to execute the setConfiguration(...) method in order to test the changes applied to the configuration. The saveConfiguration(...) methods will however still throw exceptions. TDI 7.1 will present new Server API calls for loading configurations for editing and for saving the edited configurations on the disk.
The Server API internally tracks all configurations loaded for editing. When another Server API user requests a configuration already loaded for editing, the method call will fail with exception. A new Server API call has been added for checking whether a configuration is currently loaded for editing (locked).
The lock on a configuration will be released when the user that loaded the configuration for editing saves it back or cancels the update. The Server API provides an option to specify a timeout value for keeping a configuration locked. When that timeout is reached for a configuration the lock is released and the user that locked the configuration will not be able to save it before loading it again.
A new property "api.config.lock.timeout" has been added in the TDI Server configuration file global.properties. It specifies the timeout value in minutes. When the property is left empty or is set a value of 0, this means that there is no timeout. The default value for this property is 0. The timeout logic is implemented by a new thread in the TDI Server. This thread is activated only when "api.config.lock.timeout" is set to a value greater than 0 and will check for and release expired locks each 30 seconds.
A special call for a forced releasing of the lock on a configuration loaded for editing has been added to the Server API. Only Server API users with the admin role will be able to execute it.
All configurations are identified through the relative file path of the configuration file according the TDI Server configurations folder.
All paths specified as method parameters are relative to the TDI Server configurations folder.
The following new calls will be added to the local and remote Server API Session objects and in the JMX interfaces:
Administratively releases the lock of the specified configuration. This call can be only executed by users with the admin role.
Releases the lock on the specified configuration, aborting all changes being done. This call can only be executed from a user that has previously checked out the configuration and if the configuration lock has not timed out.
Returns a list of the file names of all configurations in the specified folder. The configurations file paths returned are relative to the TDI Server configurations folder.
Returns a list of the child folders of the specified folder
Returns a list of the file names of all configurations in the directory subtree of the TDI Server configurations folder. The configurations file paths returned are relative to the TDI Server configurations folder.
Checks out the specified configuration. Returns the MetamergeConfig object representing the configuration and locks that configuration on the Server.
Checks out the specified password protected configuration. Returns the MetamergeConfig object representing the configuration and locks that configuration on the Server.
Saves the specified configuration and releases the lock. If a temporary Config Instance has been started on check out, it will be stopped as well.
Encrypts and saves the specified configuration and releases the lock. If a temporary Config Instance has been started on check out, it will be stopped as well.
Checks in the specified configuration and leaves it checked out. The timeout for the lock on the configuration is reset.
Creates a new empty configuration and immediately checks it out. If a configuration with the specified path already exists and the aOverwrite parameter is set to false the operation will fail and an Exception will be thrown.
Checks out the specified configuration and starts a temporary Config Instance on the Server. This Config Instance will be stopped when the configuration is checked in or when the lock on the configuration expires. The method returns the ConfigInstance object. The MetamergeConfig object can be retrieved through the ConfigInstance object.
Checks out the specified password protected configuration and starts a temporary Config Instance on the Server. This Config Instance will be stopped when the configuration is checked in or when the lock on the configuration expires. The method returns the ConfigInstance object. The MetamergeConfig object can be retrieved through the ConfigInstance object.
Creates a new empty configuration, immediately checks it out and loads a temporary Config Instance on the Server. If a configuration with the specified path already exists and the aOverwrite parameter is set to false the operation will fail and an Exception will be thrown. The temporary Config Instance will be stopped when the configuration is checked in or when the lock on the configuration expires. The method returns the ConfigInstance object. The MetamergeConfig object can be retrieved through the ConfigInstance object.
Checks if the specified configuration is checked out on the Server.
This is a special version of the load for edit mechanism - the difference is that when the configuration is loaded for editing a temporary Config Instance will be started as well. This will allow testing the configuration and its AssemblyLines while they are being developed and will be particularly useful for development tools like the TDI Config Editor.
The Config Instance will be automatically stopped when the configuration is released or when the lock on the configuration expires.
The temporary Config Instances are independent of the normal long running Config Instances on the Server. A normal Config Instance from configuration rs.xml might be running on the Server and at the same time the rs.xml configuration can be loaded for editing with a temporary Config Instance. This will result in starting a new temporary Config Instance from the rs.xml file in addition to the normal long running rs.xml Config Instance.
The same locking mechanism applies for configurations loaded for editing with a temporary Config Instance. This means that a configuration can be loaded for editing only once regardless of whether it has been loaded for editing with a temporary Config Instance or without.
Traditionally, starting a config instance as well as the check-in/check-out functionality of the Server API required the URL (file path) of the config file to be provided. This is no longer necessary in the current version of TDI, because the same Server API interface methods can be passed the corresponding Solution Name instead. This is a user convenience as Server API clients like the AMC and CLI can take user-friendly Solution Names from the user instead of cryptic config file paths.
Config file path precedence over Solution Names
The config file path has a higher priority than the Solution Name. This means that if the method for starting a config instance is passed a string (either a config file path or the corresponding Solution Name) and it is a valid config file path then the method will treat this value as referring to this config file. This means that if there is a config file and a Solution Name which are identical as strings then the config file path takes precedence. This behavior ensures backward compatibility with previous versions of TDI when there were no Solution Names.
Configs Folder
Only config files residing in the TDI configs folder at TDI Server startup time can be referred to by their Solution Name.
See also "Optional Config instance ID in a Config file" in IBM TDI V7.1 Installation and Administrator Guide for more information about Solution Names, Run Names and how to configure these.
A Server API event di.ci.file.updated will be fired whenever a configuration that has been locked is saved on the TDI Server.
This notification will allow Server API clients to get notified for changes in configurations they are using and for example reload them to get the latest version.
The System Queue is a TDI server module which TDI internal objects as well as TDI components can use as a general purpose queue. The purpose of the System Queue is to connect to a JMS Provider and provide functionality for getting from JMS message queues and putting into JMS message queues general messages as well as TDI Entry objects. The System Queue can connect to different JMS Providers using different TDI JMS Drivers. For more information on the System Queue please see the "System Queue" chapter of the IBM TDI V7.1 Installation and Administrator Guide.
The System Queue functionality is exposed through both the local and remote interfaces of the Server API as well as through the JMX layer of the Server API. TDI components and subsystems which run in the Java Virtual Machine of the local TDI server are expected to use the local Server API interfaces to interact with the System Queue. Remote Server API client applications as well as TDI components and sub-systems which run in the Java Virtual Machine of a remote TDI server are expected to use the remote Server API interfaces.
The TDI 7.1 Server API JMX layer contains a SystemQueue MBean. This MBean provides JMX access to the SystemQueue. A JMX client can access the SystemQueue JMX MBean and thus work with the System Queue through JMX.
The System Queue must be properly configured before it can be accessed through the Server API. A simple way to configure the System Queue is like the following procedure:
TDI provides the MQ Everyplace JMS Provider out of the box. We can setup a MQe Queue Manager via the mqeconfig command line utility (the mqeconfig utility is located in the 'jars/plugins' subfolder of our TDI installation).
Modify the mqeconfig.props configuration file.
serverRootFolder=C:\\TDI\\MQePWStore
serverIP=127.0.0.1
mqeconfig mqeconfig.props create server
mqeconfig mqeconfig.props create queue myqueue
systemqueue.on=true
systemqueue.jmsdriver.name=com.ibm.di.systemqueue.driver.IBMMQe
systemqueue.jmsdriver.param.mqe.file.ini=C:\TDI\MQePWStore\pwstore_server.ini
For a stand-alone Java™ program to operate successfully with the System Queue through the Server API, a JMS implementation must be included in the CLASSPATH of the program. We can use the JMS implementation distributed with TDI: jars/3rdparty/IBM/ibmjms.jar
Once a Server API session is initiated, the System Queue can be accessed like this:
import com.ibm.di.api.remote.SystemQueue; ... SystemQueue systemQueue = session.getSystemQueue();
The following code puts a text message into a queue named "myqueue" (the call will not create the specified queue automatically - the queue must be created manually first):
systemQueue.putTextMessage("myqueue", "mytextmessage");
The following code retrieves a text message from a queue named "myqueue" (the queue must exist). The method call waits a maximum of 10 seconds for a message to become available:
String textMessage = systemQueue.getTextMessage("myqueue", 10);
Previous versions of TDI do not keep track of configurations or AssemblyLines that have terminated. Therefore, administrators have no way of knowing when their AssemblyLines last ran, without going into the log of each one. Bundlers that initiate AssemblyLines have no way of querying their status after they've terminated.
The solution is a Tombstone Manager that creates records ("tombstones") for each AssemblyLine and configuration as they terminate, that contain exit status and other information that later can be requested through the Server API.
Globally Unique Identifiers (GUID) are created by the Server API to uniquely identify Config Instance and AssemblyLine instances. The GUID is a string value that is unique for each instance of a Config Instance, an AssemblyLine or an EventHandler (from older versions of TDI) ever created by a particular TDI Server.
GUIDs are defined as the string representation of the Config Instance/AssemblyLine object hashcode concatenated with the string representation of the Config Instance/AssemblyLine start time in milliseconds.
A method is available in the Config Instance and AssemblyLine Server API interfaces: String getGlobalUniqueID ();
A field GlobalUniqueID is available in the AssemblyLine and Config Instance stop Server API events.
The Server API provides a new class com.ibm.di.api.Tombstone whose instances represent tombstone objects. The public interface of the Tombstone class follows:
public class Tombstone implements Serializable {
public int getComponentTypeID ()
public int getEventTypeID ()
public java.utilDate getStartTime ()
public java.utilDate getTombstoneCreateTime ()
public String getComponentName ()
public String getConfigID ()
public int getExitCode ()
public String getErrorDescription ()
public String getGUID ()
public Entry getStat ()
public String getUserMessage ()
}
Tombstones are retrieved through the Tombstone Manager. We can access the Tombstone Manager via the Server API like this:
import com.ibm.di.api.remote.TombstoneManager; ... TombstoneManager tombstoneManager = session.getTombstoneManager();
With the Tombstone Manager at hand, we can search for specific Tombstones. The following code iterates through all tombstones created last week:
Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.DATE, -7);
Tombstone[] tombstones = tombstoneManager.getTombstones(calendar.getTime(), new Date());
for (int i = 0; i < tombstones.length; ++i) {
System.out.println("Tombstone found for : "+tombstones[i].getComponentName());
System.out.println("\t GUID : "+tombstones[i].getGUID());
System.out.println("\t statistics : "+tombstones[i].getStatistics());
}
All tombstones for a particular AssemblyLine can be retrieved this way (the example AssemblyLine is named "myline" and the ID of the configuration is "C__TDI_myconfig.xml"):
Tombstone[] alTombstones = tombstoneManager.getAssemblyLineTombstones("AssemblyLines/myline", "C__TDI_myconfig.xml");
The following new Server API calls are provided for querying the Tombstone Manager - these are methods of the com.ibm.di.api.local.TombstoneManager interface:
Returns a single tombstone object uniquely identified by the specified GUID.
Returns all available tombstones for the specified AssemblyLine.
Returns all available tombstones for the specified AssemblyLine with timestamps in the interval specified by aStartTime and aEndTime.
Returns all available tombstones for the specified Config Instance.
Returns all available tombstones for the specified Config Instance.
Returns all available tombstones with timestamps in the interval specified by aStartTime and aEndTime.
When tombstones are no longer needed they should be deleted.
The following code deletes all tombstones from the last week:
tombstoneManager.deleteTombstones(7);
The following Server API calls are provided for deleting old tombstone records:
Deletes all tombstones that are older than the specified number of days. Returns the number of deleted tombstone records.
After this method is executed only the aMostRecentToKeep most recent tombstone records are kept and all other are deleted. Returns the number of deleted tombstone records.
Deletes all tombstones for specified AssemblyLine. Returns the number of deleted tombstone records.
Deletes all tombstones for the specified AssemblyLine that are older than the specified number of days. Returns the number of deleted tombstone records.
After this method is executed only the aMostRecentToKeep most recent tombstone records for the specified AssemblyLine are kept and all other are deleted. Returns the number of deleted tombstone records.
Deletes all tombstones for specified Config Instance. Returns the number of deleted tombstone records.
Deletes all tombstones for the specified Config Instance that are older than the specified number of days. Returns the number of deleted tombstone records.
After this method is executed only the aMostRecentToKeep most recent tombstone records for the specified Config Instance are kept and all other are deleted. Returns the number of deleted tombstone records.
Deletes the tombstone with the specified GUID. Returns true only when the tombstone object with the specified GUID is found and deleted.
The task script object represents the AssemblyLine object in an AssemblyLine context so that we can use this object when scripting.
The interface of the task object is extended to provide a method for setting a custom message that will be saved in the UserMessage field of the tombstone for this AssemblyLine. The signature of the new method, accessible through the task script object is as follows:
task.setTombstoneUserMessage(String aUserMessage);
This method can be used from AssemblyLine scripts to provide additional information in the AssemblyLine tombstone.
The user message of a tombstone can be retrieved like this:
String userMessage = tombstone.getUserMessage();
No user defined messages can be set for ConfigInstance tombstones.
For a remote client to query/get/set properties (or stores), it needs to be provided a remote reference of the TDIProperties object in the server. A remote client can obtain the com.ibm.di.api.remote.TDIProperties interface remote reference via the following method in com.ibm.di.api.remote.ConfigInstance:
public TDIProperties getTDIProperties() throws DIException,RemoteException;
A similar interface and implementation is available in the local Server API interfaces.
For a description of the interface methods please see the TDI JavaDocs.
The following example lists all available Property Stores for a given configuration instance:
TDIProperties tdiProperties = configInstance.getTDIProperties();
List stores = tdiProperties.getPropertyStoreNames();
Iterator it = stores.iterator();
System.out.println("Available property stores :");
while (it.hasNext()) {
String storeName = (String) it.next();
System.out.println("\t"+storeName);
}
Individual properties can be acquired by their name. The following code prints all properties available in the Global Property Store (global.properties) :
String storeName = "Global-Properties";
System.out.println(storeName+" store contents :");
String[] storeKeys = tdiProperties.getPropertyStoreKeys(storeName);
for (int i = 0; i < storeKeys.length; ++i) {
System.out.println("\t"+storeKeys[i]+" : "+ tdiProperties.getProperty(storeName, storeKeys[i]));
}
Property values can be changed and new properties can be created like this:
tdiProperties.setProperty(storeName, "mykey", "myvalue");
The following code removes a property from a Property Store:
tdiProperties.removeProperty(storeName, "mykey");
Before any changes to a Property Store (adding a new property, changing the value of a property or removing a property) take effect, the changes must be committed:
tdiProperties.commit();
A TDIPropertiesMBean interface is available in the com.ibm.di.api.jmx.mbeans package. The methods exposed in TDIPropertiesMBean interface are similar to the ones exposed in the com.ibm.di.api.remote.TDIProperties interface.
A method getTDIProperties( ) is available in the com.ibm.di.api.jmx.mbeans.ConfigInstanceMBean class via which a JMX client can obtain a reference to a javax.management.ObjectName interface.
The Server API provides an event notification mechanism for Server events like starting and stopping of Config Instances and AssemblyLines. This allows a local or remote client application to register for event notifications and react to various events.
Applications that need to register and receive notifications should implement a listener class that implements the DIEventListener interface (com.ibm.di.api.remote.DIEventListener for remote applications and com.ibm.di.api.local.DIEventListener for local access). This class is responsible for processing the Server events. The handleEvent(DIEvent aEvent) method from the DIEventListener interface is where we need to put your code that processes Server events. Of course you may implement as many listener classes as we need, with different implementations of the handleEvent(DIEvent aEvent) method and register all of them as event listeners. A sample listener that just logs the event object might look like this:
import java.rmi.RemoteException;
import com.ibm.di.api.DIEvent;
import com.ibm.di.api.DIException;
import com.ibm.di.api.remote.DIEventListener;
public class MyListener implements DIEventListener
{
public void handleEvent (DIEvent aEvent) throws DIException, RemoteException
{
System.out.println("TDI Server event: " + aEvent);
System.out.println();
}
}
Once we have implemented your listener we will need to register it with the Server API. If however you are implementing a remote application there is one extra step we need to perform before actually registering the listener object with the Server API - we need to instantiate and use a base listener object that will wrap the listener you implemented. The base listener class allows us to use our own listener classes without having the same Java classes available on the Server:
DIEventListener myListener = new MyListener(); DIEventListener myBaseListener = DIEventListenerBase.createInstance(myListener, true);
The base listener object implements the same DIEventListener interface - its class however is already present on the Server and it can act as a bridge between your local client side listener class and the Server. A base listener object is created by calling the static method createInstance(DIEventListener aListener, boolean aSSLon) of the com.ibm.di.api.remote.impl.DIEventListenerBase class. The first parameter aListener represents the actual listener object and the second one specifies whether SSL is used or not by the Server API (note that this is not an option for you to select whether to use SSL or not with this listener object; here we have to specify how the Server API is configured on the Server side - otherwise the communication for that listener will fail).
When we have the listener object ready (or a base listener for remote access), we can register for event notifications through the session object:
session.addEventListener(myBaseListnener, "di.*", "*");
The addEventListener(DIEventListener aListener, String aTypeFilter, String aIdFilter) method of the session object will register your listener. The first parameter aListener is the listener object (or the base listener object for remote access), aTypeFilter and aIdFilter let you specify what types of events we want to receive:
If at some point we want to stop receiving event notifications from a listener already registered with the Server API, we need to unregister the listener. This is done through the same session object it was registered with by calling:
session.removeEventListener(myListener);
A new Server API event notification has been added to signal Server shutdown events. This event is available to Server API clients and JMX clients, both in local and remote context. The event type is "di.server.stop" for both the Server API and JMX notification layers. As an additional user data the event object conveys the Server boot time.
New Server API functionality has been added for sending custom, user defined event notifications. The following new call has been added to the local and remote Server API Session objects and also to the DIServer MBean so that it can be accessed from the JMX context as well:
public void sendCustomNotification (String aType, String aId, Object aData)
The invocation of this method will result in broadcasting a new user defined event notification. The parameters that must be passed to this method have the same meaning as the respective parameters of standard Server API notifications. The aType parameter specifies the type of the event. The value given by the user will be prefixed with the user. prefix. For example if the type passed by the user is process.X.completed the type of the event broadcast will be user.process.X.completed. A client application can register for all custom events specifying a type filter of user.*. The aId parameter can be used to identify the object this event originated from. The standard Server API events use this value to specify a Config Instance or AssemblyLine. The aData parameter is where the user can pass on any additional data related to this event; if the event is expected to be sent and received in a remote context, this object has to be serializable.
Starting an AssemblyLine with a listener describes how listeners can be used to get AssemblyLine log messages in real time as they are produced.
The Server API provides another mechanism for direct access to log files produced by AssemblyLines. This mechanism only provides access to the log files generated by the AssemblyLine SystemLog logger.
You don't need a reference to an AssemblyLine Server API object to get to the log file. Also we can access old logs of AssemblyLines that have terminated.
First we need to get hold of the SystemLog object:
SystemLog systemLog = session.getSystemLog();
We can then ask for all the log files generated by an AssemblyLine:
String[] alLogFileNames = systemLog.getALLogFileNames("C__Dev_TDI_rs.xml", "TestAL");
if (alLogFileNames != null) {
System.out.println("Availalbe AssemblyLine log files:");
for (int i=0; i<allLogFileNames.length; i++) {
System.out.println(alLogFileNames[i]);
}
}
The getALLogFileNames(String aConfigId, String aALName) method is passed the Config ID (see Stopping a Config Instance for more details on the Config ID) and the name of the AssemblyLine. This will return an array with the names of all log files generated by runs of the specified AssemblyLine.
If you are interested in the last run of the AssemblyLine only, there is a Server API call that will give you the name of that log file only:
String lastALLogFileName = systemLog.getALLastLogFileName("C__Dev_TDI_rs.xml", "TestAL");
System.out.println("AssemblyLine last log file name: " + lastALLogFileName);
When we have got the name of a log file we can retrieve the actual content of the log file:
String alLog = systemLog.getALLog("C__Dev_TDI_rs.xml", "TestAL", lastALLogFileName);
System.out.println("TestAL AssemblyLine log: ");
System.out.println(alLog);
In cases where the log file can be huge, you might want to retrieve only the last chunk of the log. The sample code below specifies that only the last 10 kilobytes from the log file should be retrieved:
String alLog = systemLog.getALLogLastChunk("C__Dev_TDI_rs.xml", "TestAL", lastALLogFileName, 10);
System.out.println("Last 10K of the TestAL AssemblyLine log: ");
System.out.println(alLog);
The Server API also provides methods for cleaning up (deleting) old log files.
We can delete all log files (for all configurations and all AssemblyLines) older than a specific date. The sample code below will delete all log files older than a week:
Calendar calendar = Calendar.getInstance(); calendar.add(Calendar.DATE, -7); systemLog.cleanAllOldLogs(calendar.getTime());
Another criterion we can use for log files clean up is the number of log files for each AssemblyLine. We can specify that we want to delete all log files except the 5 most recent logs for all AssemblyLines:
systemLog.cleanAllOldLogs(5);
We can also delete the log files for AssemblyLines only or for a specific AssemblyLine. The same two criteria are available: date and number of log files but in addition we can specify the name of an AssemblyLine or use calls that operate on all AssemblyLines. Consult the JavaDoc of the com.ibm.di.api.remote.SystemLog or com.ibm.di.api.local.SystemLog interfaces for the signatures and the descriptions of all log clean up methods.
Through the Server API we can get various types of information about the TDI Server itself like the Server version, IP address, operating system, boot time and information about what Connectors, Parsers and Function Components are installed and available on the Server.
It is the ServerInfo object that provides access to this information. We can get the ServerInfo object through the session object:
ServerInfo serverInfo = session.getServerInfo();
We can then get and print out details of the Server environment:
System.out.println("Server IP address: " + serverInfo.getIPAddress());
System.out.println("Server host name: " + serverInfo.getHostName());
System.out.println("Server boot time: " + serverInfo.getServerBootTime());
System.out.println("Server version: " + serverInfo.getServerVersion());
System.out.println("Server operating system: " + serverInfo.getOperatingSystem());
We can also output a list of all Connectors installed and available on the Server:
String[] connectorNames = serverInfo.getInstalledConnectorsNames();
System.out.println("Connectors available on the Server: ");
for (int i=0; i<connectorNames.length; i++) {
System.out.println(connectorNames[i]);
}
We can output more details for each installed Connector including its description and version:
String[] connectorNames = serverInfo.getInstalledConnectorsNames();
for (int i=0; i<connectorNames.length; i++) {
System.out.println("Installed connector: ");
System.out.println(" name: " + connectorNames[i]);
System.out.println(" description: " + serverInfo.getConnectorDescription(connectorNames[i]));
System.out.println(" version: " + serverInfo.getConnectorVersionInfo(connectorNames[i]));
System.out.println();
}
In information for other components can be retrieved in a similar manner - Parsers and Functional Components.
The Security Registry is a special Server API object that lets you query what rights a user is granted and whether he/she is authorized to execute a specific action. This is useful if an application is building an authentication and authorization logic of its own - for example the application is using internally a single admin user for communication with the TDI Server and it manages its own set of users and rights.
The Security Registry object is only available to users with the admin role. It is obtained through the session object:
SecurityRegistry securityRegistry = session.getSecurityRegistry();
We can then check various user rights. For example, securityRegistry.userIsAdmin("Stan") will return true if Stan is granted the admin role; securityRegistry.userCanExecuteAL ("User1", "rs.xml", "TestAL") will return true only if Stan is allowed to execute AssemblyLine "TestAL" from configuration "rs.xml".
Check the JavaDoc of com.ibm.di.api.remote.SecurityRegistry for all available methods.
You sometimes need to implement our own functionality and be able to access it from the Server API, both locally and remotely. This was supported by the Server API in TDI 6.0, but it needed to be simplified so that we can drop a JAR file of our own in the TDI classpath and then access it from the Remote Server API without having to deal with RMI.
Two methods are now available in the following interfaces:
The two methods are:
public Object invokeCustom(String aCustomClassName, String aMethodName, Object[] aParams) throws DIException;
and
public Object invokeCustom(String aCustomClassName, String aMethodName, Object[] aParamsValue, String[] aParamsClass) throws DIException;
Both methods invoke a custom method described by its class name, method name and method parameters.
These methods can invoke only static methods of the custom class. This is not a limitation, because the static method of the custom class can instantiate an object of the custom class and then call instance methods of the custom class.
The main difference between the two methods is that the invokeCustom(String, String, Object[], String[]) method requires the type of the parameters to be explicitly set (in the paramsClass String array) when invoking the method. This helps when we want to invoke a custom method from a custom class, but also want to invoke this method with a null parameter value. Since the parameter's value is null its type can not be determined and so the desired method to be called cannot be determined.
If the we need to invoke a custom method with a null value you must use the invokeCustom(String, String, Object[], String[]) method, where the desired method is determined by the elements of the String array which represents the types and the exact order of the method parameters. If the user uses invokeCustom(String, String, Object[]) and in the object array put a value which is null than an Exception will be thrown.
api.custom.method.invoke.on=true api.custom.method.invoke.allowed.classes=com.ibm.MyClass,com.ibm.MyOtherClass
The first line of this example specifies that custom invocation is turned on and thus the two invokeCustom() methods are allowed to be used. The second line specifies which classes can be invoked. In this case only com.ibm.MyClass and com.ibm.MyOtherClass classes are allowed to be invoked. If one of the two invokeCustom() methods is used to invoke a different class then an exception is thrown.
Suppose the following class is packaged in a jar file, which is then placed in the 'jars' folder of TDI:
public class MyClass {
public static Integer multiply(Integer a, Integer b) {
return new Integer(a.intValue() * b.intValue());
}
}
Suppose the global.properties TDI configuration file contains the following lines:
api.custom.method.invoke.on=true api.custom.method.invoke.allowed.classes=MyClass
Then in a client application the 'multiply' method of 'MyClass' can be invoked in a Server API session like this:
Integer result = (Integer) session.invokeCustom(
"MyClass",
"multiply",
new Object[] {new Integer(3), new Integer (5)});