AssemblyLine availability

Improving AL availability means two things: 1) doing what we can to ensure that the AssemblyLines do not stop, and 2) restarting ALs that have stopped as quickly as possible. For scenarios like long-running migrations and synchronizations, restarted AssemblyLines may also need to continue where they left off at the point of failure.

An AssemblyLine will stop if an unhandled exception occurs. You can safeguard against this eventuality by handling all errors, which in Security Directory Integrator terms amounts to at least enabling the 'Default On Error' Hook of each Connector and Function component. By enabling Error Hooks you are instructing the Server to continue in spite of an exception. As you saw during the tutorial exercises, if an AssemblyLine stops due to an error then you get a stack trace which is preceded with info on where the error occurred and why. If you prevent the AL from stopping by enabling Error Hooks then it is your responsibility to report error status by using special objects available to your scripts.

task.logmsg("ERROR", "[" + thisConnector.getName() + "] - " +
             error);  

The above example script makes use of two such objects: thisConnector which always references the component to which the executing script is tied, and the pre-defined variable called error. The error object is an Entry, just like work and conn, and it holds Attributes like 'status', 'connectorname'¹ and 'message', plus other relevant details about any recent error situation. Since it’s an Entry object, we can use task.dumpEntry(error) to display its contents, as well as direct references to Attribute names – for example: work.message – or just simply appending error to a string message like in the above example since all Entry objects can convert themselves to string representations as required. To handle exceptions occurring in scripts, like in Attribute Map assignment, Hooks and Script components, wrap your code in try-catch blocks. This allows you to catch exceptions and deal with them yourself:

try {
	res = myLib.callToSomeFunctionThatMightFail();
} catch (excptn) {
	task.logmsg("Call failed with error: " + excptn);
}

In addition to handling errors, you will want to use the Auto Reconnect feature described in the previous section. This will prevent your AssemblyLine from failing due to transient connectivity problems, like being timed out by a data source or firewall.

If for some reason the AssemblyLine still stops prematurely, your next step is to get it restarted. One approach is to use the Web Admin tool to define failure/response behaviors.

Another, even complimentary approach is to make a 'Launcher' AL and leverage the AssemblyLine Function Component that you used in the tutorial exercises of the last chapter. By placing this AL FC in a ConditionalLoop which never stops – in other words, with a Condition that is always true – and then configuring the AL FC to call the desired service AL and wait for it to complete, you ensure that whenever control returns to the 'LauncherAL' then the never-ending Loop will simply restart your data flow again.

If you apply the restart-loop technique outlined above for a synchronization AL based on one of the Change Detection Connectors (or the Delta Engine, both described elsewhere) then the AssemblyLine will automatically continue from the point where it failed. If not, then the burden of state handling rests on you. A common technique is to use the System Store to persist state information, like timestamps or other key values for sorted result sets. Then, whenever the AssemblyLine initializes, this state information is applied to the Iterator Connector in order to resume processing immediately after the previously handled entry.

If the iteration data source for the AL does not support sorted returns, then it might be necessary to start iteration from the very beginning again. In this case, note that the Connector Update mode offers a Compute Changes feature which compares Output Mapped Attributes with those currently found in the target system, skipping the modify operation if no differences are detected.

We can avoid a single point of failure by introducing a secure transport between multiple SDI Servers, like IBM MQ. In this way, any number of Servers (and AssemblyLines) can be used to initiate processing by placing data and even processing instructions into the queue. At the receiving end, multiple Servers/ALs pick these up on a first-come-first-serve basis and carry out the requested work. This not only results in a more robust solution, it allows for easy scaling through adding sender and receiver AssemblyLines.

This has been a very brief discussion of a much larger subject. However, the goal is more for inspiration than the prescription of a particular approach. It is advised that you look to community websites and discussion groups for more specific recommendations and examples.

Parent topic:

Hardening your Integration Solutions

¹ You may have noticed that the 'connector' word is sometime synonymous with 'component', such that variables like thisConnector can also refer to FC’s, SC’s and AttributeMap components. The same applies to the 'connectorname' Attribute of the error object, which can also hold the name of any type of component.