Verification Point Comparator
The Verification Point Comparator is used to compare verification point data after you play back a script with a verification point and to update the baseline file. If the verification point failed, the Comparator shows both the expected and actual values, so you can analyze the differences. You can then load the baseline file and edit it or update it with the values from the actual file.
You open the Comparator by double-clicking on the verification point in the Functional Tester log in TestManager, or by clicking the View Results link in the Functional Tester HTML log. (Note that if you experience an error regarding the Java plug-in when trying to launch the Comparator from the View Results link in the HTML log, you need to make sure your plug-in is configured properly. See the related topic about enabling the Java plug-in of a browser.
If you have one failed verification point, and you are using a log, select the log in the Functional Tester Projects view. Right-click on the log, and click Failed Verification Points. The Verification Point Comparator opens.
If you have more than one failed verification point, and you are using a log, the Results for Verification Points wizard opens. Click a failed verification point in the list and click View Results or Finish. For information, see Viewing Results in the Log topic. The Comparator banner will display the name of your verification point.
You can specify color settings for several elements in the Verification Point Comparator.
For information on how to compare verification points after playback, see the related topic about comparing and updating verification point data using the Comparator. To edit verification point data, load the baseline by clicking the Load Baseline to Edit toolbar button .
The following sections explain the parts of the Verification Point Comparator window, and the toolbars.
Metadata
The metadata is displayed in the left pane of the window. It displays a set of properties that define how specific data is managed. This grid can be edited. For example, you could edit the 'ignore case' or 'white space rule' in a text verification point in this metadata grid. To edit, double-click the value in the Value column.
Main toolbar
The toolbar at the top of the Verification Point Comparator has six buttons.
File: Save -- Saves any edits you have made.
File: Revert -- Reverts to the state of the data at the last save you made. If you have not saved edits since opening the Comparator, it will revert to the state it was in when opened. If you have done editing and made saves, it will revert to the state at your last Save.
Load Baseline to edit -- Loads the baseline file so you can edit it. The baseline values will be displayed, instead of the expected values. These values can be edited individually or replaced with the actual values.
Replace Baseline with actual value -- Replaces the baseline values with all the values in the actual file. Then those values will become the baseline for future playbacks. If you want to replace only some of the values, edit them individually. This command replaces the entire file.
Hide/Show TestObject Info -- Toggles the display of the Test Objects and Recognition Data panes of the Comparator window. When this information is hidden, the entire Comparator window is used for the main data area. This is a sticky setting--the next time you open the Comparator it will appear as you last set it. However, note that if your Test Objects tree has multiple nodes, the Verification Point Comparator will show these panes again the next time you open it, regardless of this setting.
Help -- Brings up the Help for the Verification Point Comparator. You can bring up the Functional Tester Help any time from the Help menu in Functional Tester.
Menu bar
The menu bar contains the same commands that are represented with the toolbar buttons described in this topic.
File -- These are the same Save, Revert, Baseline, and Replace commands as the buttons listed above in the Main Toolbar section.
Edit -- These are the same commands as the buttons listed below in the Properties Verification Point section: Check All, Uncheck All, and Hide. This menu is grayed out until you load the baseline for editing (using the Load Baseline to edit toolbar button ).
Difference -- These are the same commands as the buttons listed below in the Navigation Toolbar Buttons section: First, Previous, Next, and Last.
Test Object > Highlight -- If your test application is open, you can select an object in the Test Objects tree and then click this command to see the object highlighted in the application. Use this feature if you need to verify an object in the application.
Preferences -- Toolbars controls the display of the toolbars. Test Object Appearance on the Tree displays the Edit Test Object Description dialog box, which enables you to customize the text displayed for each object in the Test Object Hierarchy. Hide TestObject Info is the same as the toolbar button .
Help - Brings up the Help for the Verification Point Comparator. You can bring up the Functional Tester Help any time from the Help menu in Functional Tester.
Main data area
The right pane of the Verification Point Comparator is where the verification point data is displayed. For example, in the case of a Properties verification point, the Property and Value columns are displayed here. This is where you compare the verification point data. If the verification point failed when you played back the script, the expected and actual values will both be displayed, regardless of which type of data display is being used. In some cases, the expected values are shown on the left and the actual values are shown on the right. In other cases, they are shown contiguously (such as nodes in a tree view), and the expected and actual values are shown in different colors if they are different. (Expected value is in red, actual value is in green.) The actual values are what were recorded when you played back the script.
There are seven types of displays you can get from recording verification points, as described in the following sections, after the next section, Navigation Toolbar Buttons.
Navigation toolbar buttons
These four navigation buttons jump to the differences between the expected and actual files or the baseline and actual files. Differences are shown in red. The currently selected difference will be highlighted.
Jump to First Difference -- Goes to the first difference in the expected/baseline and actual files.
Backward to Previous Difference -- Goes backward to the previous difference in the expected/baseline and actual files.
Forward to Next Difference -- Goes forward to the next difference in the expected/baseline and actual files.
Jump to Last Difference -- Goes to the last difference in the expected/baseline and actual files.
Properties verification point -- grid display
When you create a Properties verification point, the object properties are displayed in a grid format. The properties that are shown in the grid belong to the object that is highlighted in the Test Objects tree. The properties appear in the left column and their values appear in the right column. You can edit which properties get tested in the Property column, and can edit the property values themselves in the Value column.
Properties with no checkmark will not be tested. Choose which properties you want to test by checking each of them. Checked properties will be tested each time you play back a script with this verification point. You can check all properties in the list by clicking the Check All toolbar button above the grid. Use the Uncheck All button to clear all properties. Depending on how many properties you want to test, it is often easiest to either select or clear all of them using one of those buttons, and then individually select or clear exceptions.
The grid uses a nested tree hierarchy. If a folder shows up on the list, you can expand it by double-clicking on it or selecting the expand icon. If you check or clear the folder icon itself, all the properties underneath it will be tested or not tested.
To edit a value, double-click the grid cell. That cell will then be editable. Click outside the cell to make the edit take effect. In most cases double-clicking a value makes the cell an editable field, and you can just change the value. In some special cases, another dialog box comes up containing the information. For example, if the property is color, when you double-click the color value, the standard Color dialog box opens. Make your edit there and close the Color box. In other cases, a drop-down list may appear in the Value column when you double-click a value. For example, values that are either true or false will appear in a drop-down list.
The grid has the following toolbar buttons for the Properties verification point display. In the Comparator, these buttons appear only when you are editing the baseline.
Check All -- Puts a checkmark in front of every property in the list. Checked properties will be tested each time you play back the script with this verification point. And only checked properties will be compared in the Comparator.
Uncheck All -- Clears the checkmark in front of every property in the list. Unchecked properties will not be tested when you play back the script with this verification point.
Hide the Unchecked Properties/Show All Properties -- Click Hide the Unchecked Properties to hide the unchecked properties. Then you will only see the properties that will be tested. Click Show All Properties to display all properties, including any unchecked ones.
The grid has the following pop-up menu commands for the Properties verification point display. To access them, right-click on a value in the Value column.
Open -- If the value is a string or a complex value type, this will display the value in a separate window, which enables you to see long lines of text and makes it easier to edit.
Case Sensitive Regular Expression -- Toggles case-sensitive regular expression comparison on and off.
Evaluate Regular Expression -- Displays the Regular Expression Evaluator, which enables you to test the regular expression before you try it in a verification point.
Convert Value to Regular Expression -- Converts the property value to a regular expression. See the related topic about replacing an exact-match property with a pattern for more information.
Undo/Redo Regular Expression -- Cancels or redoes the regular expression conversion.
Convert Value to Numeric Range -- Converts the property value to a numeric range. See the related topic about replacing an exact-match property with a pattern for more information.
Undo Numeric Range -- Cancels the numeric range.
Convert Value to Datapool Reference -- Uses a datapool reference to use a datapool instead of a literal value in a verification point.
Undo Datapool Reference -- Cancels the datapool reference in the verification point.
Replace Baseline On Current Selection -- Replaces the baseline value with the actual value for just the selected property. This is a per-property version of the Replace Baseline With Actual Value toolbar button .
To compare object properties:
To compare object properties, look at the expected/baseline and actual values columns. The expected or baseline values are shown on the left and the actual values are shown on the right. The actual values are what were captured when you played back the script. Use the navigation buttons to navigate to all the differences, which are shown in red. For information on editing the baseline values or replacing the baseline with the actual file, see the related topic about comparing and updating verification point data using the Comparator.
Data verification point -- menu hierarchy display
When you create a Data verification point and choose the Menu Hierarchy or Menu Hierarchy with Properties test, the menus are displayed in a tree format in the main data area (right pane). Menu Hierarchy and Menu Hierarchy with Properties are two examples. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than these two may be shown.
The tree will display the entire menu hierarchy of your application, or one top-level menu and its sub-items, depending on how you recorded the verification point. If you chose the whole menu bar, each top-level menu will be shown from top to bottom in the tree in the order they appear from left to right in the menu bar. Each individual menu item is shown under its top-level menu. Use the plus and minus signs to open and close the list for each top-level menu.
To edit a menu, double-click it in the tree. (If you haven't done so, load the baseline first.) This opens its properties displayed in a grid, which you can then edit. You can edit the actual values by double-clicking a value in the Value column. You can also edit the list of which properties get tested during playback by using the check box beside each property. Checked items get tested. The toolbar buttons above the grid are the same ones that are found in the object properties grid described above in the Properties Verification Point--Grid Display section, except for Hide/Show. The buttons work the same as described there, except they apply to the selected menu property or value.
To compare menu hierarchy data:
To compare menu hierarchy data, look at any differences shown in red and green. The expected values are shown in red, and the actual values are shown underneath them in green. The actual values are what were captured when you played back the script. If the descriptions for the expected and baseline values are the same, but there are some differences in their properties, the node is shown in blue. Use the navigation buttons to navigate to all the differences. For information on editing the baseline values or replacing the baseline with the actual file, see the related topic about comparing and updating verification point data using the Comparator.
Data verification point -- text display
When you create a Data verification point and choose the Visible Text test, the text is displayed in a text box format in the main data area (right pane). Visible Text is one example. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than this one may be shown.
The text is displayed in a text box area. You cannot edit directly in this area. To edit the verification point data, click the Edit Text button above the data display. (If you haven't done so, load the baseline first.) A small text editor containing the text will open. You can edit the text in this editor, and when you close it, the edited text will appear in the baseline column of the Comparator.
To compare text data:
To compare text data, look at the expected and actual values columns. The expected values are shown on the left and the actual values are shown on the right. The actual values are what were captured when you played back the script. Use the navigation buttons to navigate to all the differences, which are shown in red. For information on editing the baseline values or replacing the baseline with the actual file,see the related topic about comparing and updating verification point data using the Comparator.
Data verification point -- table display
When you create a Data verification point and choose the Table Contents or Selected Table Cells test, the table data is displayed in a table in the main data area (right pane). Table Contents and Selected Table Cells are two examples. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than these may be shown.
The table displays the same information as the table in your application. To edit the verification point data, double-click any cell in the table to edit that cell. (If you haven't done so, load the baseline first.)
You can also edit which cells in the table get tested. Table cells that are within the comparison regions are shown with a grey background. If you are testing the entire table, all cells will be grey. You can use the drop-down list in the toolbar above the data region as a selection mechanism. (This doesn't show up until you load the baseline.) Choose Column, Row, or Cell Selection in the list, then make your selections in the table. For example, if you select Row Selection, when you click on a cell in the second row, the whole second row will be selected. If you had chosen Cell Selection, only that cell would have been selected. After you select the data you want to compare, click the Update Comparison Region button to have your changes take effect.
The Cut, Copy, Paste, and Delete toolbar buttons above the table area apply to the selected row(s), and are only applicable within the Verification Point Comparator. (It does not use the system clipboard.)
You can right-click on a table item to access a pop-up menu. The commands are the same as those listed above in the Properties Verification Point--Grid Display section.
There are features in the Metadata tab that you can also use to edit the table data. For example, you can edit the table's column headers or row headers by accessing them in the MetaData tab. To edit column headers, double-click the Value column of the columnHeaders property. A small editor opens that lets you edit the headers. The row headers work the same way if your table has them. Double-click the rowHeaders Value to edit them. In order for the column headers to be compared, change the compareColumnHeaders property to true in the MetaData tab. The compareRowHeaders value works the same way to indicate whether row headers will be compared.
If you double-click the Value of the compareRegions property in the Metadata tab, an editor will open showing the selected regions of your table. For selected sells, it shows the row index or key value pairs and the column header or index of each selected cell. For selected rows, it shows the row index or key value pairs. For selected columns, it shows the column header or index. Using this compare regions editor is another way you can select which regions get compared. If you click the Compare All Cells button in this editor, all of the table cells will be tested.
If your table supports row keys or column keys, you can edit those and insert keys by double-clicking on the columnKeys and rowKeys values in the Metadata tab.
To compare table data:
To compare table data, look at the expected and actual values columns. The expected values are shown on the left and the actual values are shown on the right. The actual values are what were captured when you played back the script. Use the navigation buttons to navigate to all the differences, which are shown in red. For information on editing the baseline values or replacing the baseline with the actual file, see the related topic about comparing and updating verification point data using the Comparator.
Data verification point -- tree hierarchy display
When you create a Data verification point and choose the Tree Hierarchy test, the data is displayed in a tree format in the main data area (right pane). Tree Hierarchy is one example. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than this one may be shown.
The tree will display the entire tree hierarchy in your application, or the part of the tree selected when you create the verification point. Each item in the tree will be shown from top to bottom in the tree in the order it appears in your application. Each individual item is shown under its top-level item. Use the plus and minus signs to open and close the list for each top-level item.
To edit an item in the hierarchy, double-click on it in the tree. This opens a small text box, which you can use to edit the item. (If you haven't done so, load the baseline first.)
To compare tree hierarchy data:
To compare tree hierarchy data, look at any differences shown in red and green. The expected values are shown in red, and the actual values are shown underneath them in green. The actual values are what were captured when you played back the script. Use the navigation buttons to navigate to all the differences. For information on editing the baseline values or replacing the baseline with the actual file, see the related topic about comparing and updating verification point data using the Comparator.
Data verification point -- list display
When you create a Data verification point and choose the List Elements test, the data is displayed in a list format in the main data area (right pane). List Elements is one example. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than this one may be shown.
The list displays the same information as the list in your application, and in the same order top to bottom. To edit a list item, double-click on it in the list display. (If you haven't done so, load the baseline first.) The field then becomes editable. You can also edit the list of which items get tested during playback by using the check box beside each item. Checked items get tested.
The toolbar buttons above the list are the same ones that are found in the object properties grid described above in the Properties Verification Point--Grid Display section. The buttons work the same as described there, except they apply to the selected list item(s).
You can right-click on a table item to access a pop-up menu. The commands are the same as those listed above in the Properties Verification Point--Grid Display section.
To compare list data:
To compare list data, look at the expected and actual values columns. The expected values are shown on the left and the actual values are shown on the right. The actual values are what were captured when you played back the script. Use the navigation buttons to navigate to all the differences, which are shown in red. For information on editing the baseline values or replacing the baseline with the actual file, see the related topic about comparing and updating verification point data using the Comparator.
Data verification point -- state display
When you create a Data verification point and choose the CheckBox Button State or Toggle Button State test, the data is displayed in a list format in the main data area (right pane). CheckBox Button State or Toggle Button State are two examples. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than this one may be shown.
To compare state data:
To compare state data, look at the expected and actual values columns. The expected values are shown on the left and the actual values are shown on the right. The actual values are what were captured when you played back the script. For information on editing the baseline values or replacing the baseline with the actual file, see the related topic about comparing and updating verification point data using the Comparator.
Test object data in the Verification point comparator window
While inserting the verification points, if you have not checked the Record Test Object relative Verification Points option available in the General Recorder page of the Windows > Preferences window, you can view the following test object data in the Verification Point comparator:
- Test objects
- Recognition and Administrative data
Test objects
This is the upper left pane of the Verification Point Comparator window. It's a partial version of the script's object map. This hierarchical display includes only the objects in your verification point. You cannot edit the Test Objects tree. You can choose an object within it and edit its properties or data in the right pane.
You can double-click folders in the tree to expand and collapse the objects beneath them. Click on an individual object in the tree to see its properties or data in the right pane.
The check boxes to the left of each node indicate whether that node will be tested or not. Checked items get tested. After you load the baseline to edit you can check or uncheck items.
If your test application is open, you can select an object in the Test Objects tree and then click Test Object > Highlight from the Verification Point Comparator menu to see the object highlighted in the application. Use this feature if you need to verify an object in the application.
Recognition and Administrative data
This is the lower left pane of the Comparator window. The Recognition tab displays recognition data used by Functional Tester and is not editable. Some of these properties are the recognition properties that were listed in the Select an Object tab of the Verification Point and Action Wizard when you created the verification point. The Administrative tab displays internal administrative data of the object and is not editable. These properties are used to manage and describe the test object. Recognition and administrative data are the properties from the script's object map used to locate and manage this test object in the context of the associated script. You can use this information to determine what test object this is in the associated application under test.
The MetaData tab displays a set of properties that define how specific data is managed. This grid can be edited if you load the baseline. For example, you could edit the 'ignore case' or 'white space rule' in a text verification point in this metadata grid. To edit, double-click the value in the Value column.
The Recognition and Administrative properties are a snapshot of the object map properties for the test object at the time the verification point was created. They become historical information as the application evolves.
Note for ClearCase users
If you use the Functional Tester ClearCase® integration, you can check out your verification point files from the Comparator.
If the verification point baseline is not editable and is checked in, if you replace your baseline file (by clicking File > Replace or the Replace Baseline with Actual Value toolbar button ), Functional Tester will do an unreserved check-out of the scripts associated with the verification point.
If the verification point baseline is not editable and is checked in, if you load the baseline file (by clicking File > Baseline or the Load Baseline to Edit toolbar button ), Functional Tester will open the ClearCase check-out dialog box to allow you to check out the files, reserved or unreserved, if you want to. If you check out the files, when you click Finish the scripts will be checked out, and the baseline will be loaded and become editable. If you click Cancel, the baseline is loaded but is not editable.
If the verification point baseline is not editable and is not checked in, you cannot replace the baseline (the File > Replace menu and the Replace Baseline with Actual Value toolbar button are disabled).
If the verification point baseline is not editable and is not checked in, if you load the baseline file (by clicking File > Baseline or the Load Baseline to Edit toolbar button ), Functional Tester will not open the ClearCase check-out dialog box. The baseline is loaded but is not editable.
Related tasks
Comparing and updating verification point data using the Comparator
Enable the Java plug-in of a browser
Related reference