Scripting Perforce: Daemons and Triggers

Daemons and Triggers

Overview
User-written scripts can enhance Perforce's functionality. There are three primary methods of scripting Perforce:

Wrappers are scripts that call Perforce commands. Wrappers can be written in any scripting language, are usually tailored to your own site's needs, and are not discussed here.

Daemons run at predetermined times, looking for changes to the Perforce metadata. When a daemon determines that the state of the depot has changed in some useful way, it runs other commands. For example, a daemon might look for newly submitted changelists and send email to users who have previously updated the files that were submitted in those changelists. Perforce provides a number of tools that make daemon-writing easier.

Pre-submit triggers are scripts that Perforce runs whenever users attempt to submit files. A trigger script returns a value to Perforce that determines whether or not the submit should succeed. For example, you might write a script that watches for a particular executable file to be submitted; when this file is submitted, the trigger script might tell the submit to fail if the release notes file has not been updated within the same changelist.

This chapter assumes that you know how to write scripts.

Triggers

Although Perforce commands that only read data from the depot can be called in a trigger script, running Perforce commands that write data to the depot is dangerous and should be avoided. In particular, do not run the p4 submit command from within a trigger script.
A pre-submit trigger is a script called by Perforce when files that you've specified have been submitted. If the script exits with status 0, the submit continues. If it exits with a nonzero status, the submit fails. In the event of failure, the script's standard output is displayed as part of the error message returned to the user by the failed submit.
Triggers can be useful in many situations. Consider the following common uses:

To validate submits above and beyond the Perforce protections mechanism. For example, it might ensure that Ed isn't allowed to submit file foo until Courtney has submitted file bar.

To tell the submit to fail if file foo is not submitted in the same changelist as file bar.

To ensure that every submit to a particular codeline fixes at least one job.

Triggers are created and edited with p4 triggers. Only the Perforce superuser can run this command. The p4 triggers form looks like this:
    Triggers:
            relnotes_check  //depot/bld/...   "perl relcheck.pl %user%"
            verify_jobs     //depot/...       "python /usr/bin/job.py %change%"
Each line in the trigger table has three fields:

Field Meaning
Trigger Name The name of the trigger. The name can be any arbitrary string.
File Specification A file specification in depot syntax. If a changelist contains any files that match this specification, the script will be run.
Multiple file specifications can be linked to the same trigger by listing the trigger multiple times in the trigger table.

Script The command that Perforce should run when a matching file is submitted.
The submit will continue if the trigger script exits with 0, and fail otherwise.
The script must be specified in a way that allows the Perforce server account to find the file. You can place the directory of the script in the PATH of the environment in which p4d is running, or specify the full path name of the script within the trigger table.
This command should be quoted, and can take any or all of a number of variables as parameters. The most useful of these variables are:

%user%, which provides the Perforce name of the user who submitted the changelist;
%changelist%, (also abbreviated as %change%), the number of the changelist that's being submitted; and
%client%, the name of the client workspace from which the submit was run.

Example: Creating a trigger
The development group wants to make sure that whenever a .exe file is submitted to the depot, the release notes for the program are submitted at the same time.
You write a trigger script that takes a changelist number as its only argument, does a p4 opened on the changelist, parses the results to find the files included in the changelist, and ensures that for every executable file that's been submitted, a RELNOTES file in the same directory has been submitted. If the changelist includes a RELNOTES file, the script terminates with an exit status of 0; otherwise the exit status is set to 1.
The script written, you add it to the trigger table by editing the form displayed by p4 triggers:
    Triggers:
            rnotes //depot/....exe   "/usr/bin/rnotetest.pl %changelist%"
Whenever an .exe file is submitted, this trigger is run. If the script fails, it returns a nonzero exit status, and Perforce aborts the submit.

Using triggers
Triggers are run in the order entered in the triggers table.
If you have multiple triggers associated with a file pattern, each will be run in the order in which it appears in the triggers table. If one of these triggers fails, no further triggers are executed.

Example: Multiple triggers on the same file:
In the next few examples, we're using %change% as an abbreviation for %changelist%. Either form is acceptable, as they are interchangeable.
All *.c files must pass through the scripts check1.sh, check2.sh, and check3.sh:
    Triggers:
            check1     //depot/src/*.c  "/usr/bin/check1.sh %change%"
            check2     //depot/src/*.c  "/usr/bin/check2.sh %change%"
            check3     //depot/src/*.c  "/usr/bin/check3.sh %change%"
If any trigger fails (say, check1.sh), the submit fails immediately and none of the subsequent triggers (that is, check2.sh and check3.sh) are called. Each time a trigger succeeds, the next matching trigger is run.
If you have multiple filepatterns triggering the same script, you should create multiple triggers with separate names pointing to the same script. This is due to a limitation of Perforce.

Example: A limitation: activating the same trigger for multiple filespecs:
    Triggers:
            bugcheck     //depot/*.c     "/usr/bin/checkit.pl %change%"
            bugcheck     //depot/*.h     "/usr/bin/checkit.pl %change%"
            bugcheck     //depot/*.cpp   "/usr/bin/checkit.pl %change%"
In this case, the detection of foo.c in a changelist causes subsequent triggers with the same name (including the one intended for bar.h) to be ignored. The checkit.pl script only runs for *.c files. The *.h files and *.cpp files will not have the trigger applied.
The workaround is to specify separately named triggers for each filespec: *.c, *.h, and *.cpp
    Triggers:
            bugcheck1     //depot/*.c    "/usr/bin/checkit.pl %change%"
            bugcheck2     //depot/*.h    "/usr/bin/checkit.pl %change%"
            bugcheck3     //depot/*.cpp  "/usr/bin/checkit.pl %change%"
In this case, the bugcheck1 trigger runs on the *.c files, the bugcheck2 trigger runs for the *.h files, and bugcheck3 runs on the *.cpp files.
Some trigger scripts need to know the files that are included in the changelist. Since p4d can only pass 1K of data to a trigger script, the file list can't be passed via the trigger. Use p4 opened -ac changelist# in your trigger scripts to get the list of files for the changelist number provided as an argument. The actual contents of the files are not accessible from within the trigger script, since the files are not stored in the depot until the submit completes.
Before p4 submit runs the trigger script, it creates the changelist, assigns it a number, and locks the files in the changelist. After a trigger script completes, p4 submit may run subsequent trigger scripts that cause the submit to fail. For this reason, trigger scripts should not take any actions that assume the submit will succeed. Trigger scripts are meant primarily for changelist validation; if you need to take particular actions based on the success of a submit, use a daemon.

In order to use triggers, the server (p4d) must be able to "fork", or spawn off processes to run the triggers. This is the default configuration of Perforce.
If you invoke p4d with the -f (run in foreground without forking) option, however, you will not be able to use triggers until you restart the server without the -f option.

Triggers and security

Because triggers are spawned by the p4d process, p4d should never be run as root on UNIX systems.

Triggers and Windows
By default, the Perforce service runs under the Windows local System account.
Because Windows requires a real account name and password to access files on a network drive, if the trigger script resides on a network drive, configure the service to use a real userid and password to access the script.
For details, see "Installing the Perforce service on a network drive" on page 106.

Daemons
Daemons are processes that are called periodically or run continuously in the background. Daemons that use Perforce usually work by examining the server metadata as often as needed and taking action as often as necessary. Typical daemon applications include:

A change review daemon that wakes up every ten minutes to see if any changelists have been submitted to the production depot. If any changelists have been submitted, the daemon sends email to those users who have "subscribed" to any of the files included in those changelists. The message informs them that the files they're interested in have changed.

A jobs daemon that generates a report at the end of each day to create a report on open jobs. It shows the number of jobs in each category, the severity each job, and more. The report is mailed to all interested users.

A Web daemon that looks for changes to files in a particular depot subdirectory. If new file revisions are found there, they are synced to a client workspace that contains the live web pages.

Daemons can be used for almost any task that needs to occur when Perforce metadata has changed. Unlike triggers, which are used primarily for submission validation, daemons can also be used to write information (that is, submit files) to a depot.

Perforce's change review daemon
The change review daemon described above can be downloaded from http://www.perforce.com/perforce/loadsupp.html. It runs under Python, which can be retrieved from http://www.python.org/. Before running the script, please be sure to read and follow the configuration instructions included in the script itself.
The change review daemon looks at the files included in each newly submitted changelist and emails those users who have "subscribed" to any of the files included in the changelist, letting those users know that the file(s) they're interested in have changed. Users subscribe to files by calling p4 user and entering filepatterns in the Reviews: field of the resulting form:
    User:     sarahm
    Email:    sarahm@elmco.com
    Update:   1997/04/29 11:52:08
    Access:   1997/04/29 11:52:08
    FullName: Sarah MacLonnogan
    Reviews:
            //depot/doc/...
            //depot.../README
Users should enter their email addresses in the Email: field, and enter any number of filepatterns corresponding to the files in which they're interested into the Reviews: field. The daemon reports changes to these files to the users.
The change review daemon implements the following scheme:

p4 counter is used to read and change a variable, called a counter, in the Perforce metadata. The counter used by this daemon, review, stores the number of the latest changelist that's been reviewed.

The Perforce depot is polled for submitted, unreviewed changelists with the p4 review -t review command.

p4 reviews generates a list of reviewers for each of these changelists.

The Python mail module mails the p4 describe changelist description to each reviewer.

The first three steps are repeated every three minutes, or at some other interval configured the time of installation.

The command used in the fourth step (p4 describe) is a straightforward reporting command. The other commands (p4 review, p4 reviews, and p4 counter) are used almost exclusively by review daemons.

Creating other daemons
You can use p4review.py as a starting point to create your own daemons, changing it as needed. As an example, another daemon might upload Perforce job information into an external bug tracking system after changelist submission. It would use the p4 review command with a new review counter to list new changelists, and use p4 fixes to get the list of jobs fixed by the newly submitted changelists. This information might then be fed to the external system, notifying it that certain jobs have been completed.
If you write a daemon of your own and would like to share it with other users, you can submit it into the Perforce Public Depot. For more information, go to http://www.perforce.com and follow the "Perforce Public Depot" link.

Commands used by daemons
Certain Perforce commands are used almost exclusively by review daemons.
These commands are:

Command Usage
p4 counter name [value] When a value argument is not included, p4 counter returns the value of the variable name.
When a value argument appears, p4 counter sets the value of the variable name to value.
Requires at least review access to run.
WARNING: The review counters named journal, job, and change are used internally by Perforce; use of any of these three names as review numbers could corrupt the Perforce database.
For Release 99.2 and above, Perforce will not let you change the values of journal, job, and change.

p4 counters List all counters and their values.
p4 review -c changelist# For all changelists between changelist# and the latest submitted changelist, this command lists the changelists' numbers, creators, and creators' email addresses.
Requires at least review access to run.

p4 reviews -c changelist# filespec Lists all users who have subscribed to review the named files or any files in the specified changelist.
It is hard to imagine any use for this command outside of our own change review daemon.

p4 changes -m 1 -s submitted Output a single line showing the changelist number of the last submitted changelist, as opposed to the highest changelist number known to the Perforce server.

Daemons and counters
If you're writing a change review daemon or other daemon that deals with submitted changelists, you may also wish to keep track of the changelist number of the last submitted changelist, which is the second field in the output of a p4 changes -m 1 -s submitted command.
This is not the same as the output of p4 counter change. The last changelist number known to the Perforce server (the output of p4 counter change) includes pending changelists created by users, but not yet submitted to the depot.

Scripting and buffering
Depending on your platform, the output of individual p4 commands may be fully-buffered (output flushed only after a given number of bytes generated), line-buffered (as on a tty, one line sent per linefeed), or unbuffered.
In general, stdout to a file or pipe is fully-buffered, and stdout to a tty is line-buffered. If your trigger or daemon requires line-buffering (or no buffering), you can disable buffering by supplying the -v0 debug flag to the p4 command in question.
If you're using pipes to transfer standard output from a Perforce command (with or without the -v0 flag), you may also experience buffering issues introduced by the kernel, as the -v0 flag can only unbuffer the output of the command itself.

Field	Meaning
Trigger Name	The name of the trigger. The name can be any arbitrary string.
File Specification	A file specification in depot syntax. If a changelist contains any files that match this specification, the script will be run. Multiple file specifications can be linked to the same trigger by listing the trigger multiple times in the trigger table.
Script	The command that Perforce should run when a matching file is submitted. The submit will continue if the trigger script exits with 0, and fail otherwise. The script must be specified in a way that allows the Perforce server account to find the file. You can place the directory of the script in the PATH of the environment in which p4d is running, or specify the full path name of the script within the trigger table. This command should be quoted, and can take any or all of a number of variables as parameters. The most useful of these variables are: %user%, which provides the Perforce name of the user who submitted the changelist; %changelist%, (also abbreviated as %change%), the number of the changelist that's being submitted; and %client%, the name of the client workspace from which the submit was run.

Command	Usage
p4 counter name [value]	When a value argument is not included, p4 counter returns the value of the variable name. When a value argument appears, p4 counter sets the value of the variable name to value. Requires at least review access to run. WARNING: The review counters named journal, job, and change are used internally by Perforce; use of any of these three names as review numbers could corrupt the Perforce database. For Release 99.2 and above, Perforce will not let you change the values of journal, job, and change.
p4 counters	List all counters and their values.
p4 review -c changelist#	For all changelists between changelist# and the latest submitted changelist, this command lists the changelists' numbers, creators, and creators' email addresses. Requires at least review access to run.
p4 reviews -c changelist# filespec	Lists all users who have subscribed to review the named files or any files in the specified changelist. It is hard to imagine any use for this command outside of our own change review daemon.
p4 changes -m 1 -s submitted	Output a single line showing the changelist number of the last submitted changelist, as opposed to the highest changelist number known to the Perforce server.