Administration Guide

Incoming E-mail Processing Rules

On this page

When the Mail Connector processes a message, several kinds of rules decide what happens to it:

  • The MessageID tag and the system parsing rule match a reply to the record it answers.
  • Parsing rules cut values out of the message for workflow to use.
  • Preprocessing rules trim the body of a reply.
  • The job's processing operations decide which record is created or updated.

In the Admin Center, the E-mail Processing section has the pages Mail Connector, MessageID Tag, Preprocessing, Parsing, and Logs. In the Settings App, E-mail Processing has the nodes Mail Connector, MessageID Tag, Preprocessing, Parsing, and Logs. Where both applications show the same setting, they edit the same stored value. See Admin Center and Settings App.

Before you begin

  • In the Admin Center, the E-mail Processing pages are available with the same permission as the E-mail Notifications page; see Permissions Reference.

Rules in the Admin Center

Set the MessageID tag

  1. Go to E-mail Processing > MessageID Tag. The page explains that the subject line of incoming e-mail is searched for the MessageID tag to check whether the message belongs to an existing thread.
  2. In the MessageID Tag section, enter the Prefix and the Suffix. Both are required. The read-only Example shows the prefix, T000001, and the suffix as you type.
  3. Under Position, choose how to Insert the MessageID value automatically: At the beginning of email subject line, At the end of email subject line, or No automatic insertion, use only the "MessageID Tag" system macro.
  4. Click Save. Save and Cancel appear once you change something.

A blank prefix or suffix is refused with a message that names the field and says it is mandatory. Saving also updates the system parsing rule, ObjectRef, to search for the new prefix and suffix; its other criteria stay as they are. For the system macro, see Macros Reference.

Create or edit a parsing rule

The Parsing page lists the rules with their Name and Description. Its toolbar has New, Edit, Delete, a refresh button, and a search box. Edit is available when exactly one row is selected.

  1. Go to E-mail Processing > Parsing.
  2. Click New, or open a rule: select it and click Edit, double-click it, or click its name.
  3. In the Field section, enter the Name, choose the Type, and enter a Description.
  4. In the Parsing Rule section, choose where to Extract value from: Sender, To, CC, Subject, Content, Subject & Content, or Custom header field.
  5. Set Starting after to the following text or the following position, and set continues until to end of line, the following text, any of the following characters, or number of characters read is equal to. A position or a number of characters is a whole number from 1 to 99,999,999.
  6. Select or clear the case-sensitivity check box and Remove leading and trailing spaces.
  7. Click Ok. Cancel closes the drawer without saving.

A new rule starts with the name New Field, the type String, Subject & Content, spaces removed, matching not case-sensitive, and empty start and end values. The Name is required: 'The "Name" field is mandatory, it can't be left blank.'

The Type list offers Integer, Logical, Currency, String, Memo, Date, Date/Time, Float, Reference to object, HTML, and Script Object. The drawer has no control for the test value of a Logical rule, and saving a rule in the drawer replaces its whole criteria, so a Logical rule's test value is removed.

For a system rule, such as ObjectRef, the Name, Type, and Description, the custom header text, Starting after and continues until with their values, and both check boxes are unavailable. Extract value from stays available and then offers only Subject and Subject & Content.

Delete first asks "Are you sure you want to delete <n> selected items?". For each system rule in the selection, it then shows the error '"<name>" is a system record and it cannot be deleted.' with OK and Help; the remaining rules are deleted. The Parsing page has no copy command.

Create or edit a preprocessing rule

The Preprocessing page lists the rules with their Name and Expression, sorted by name. Its toolbar has Copy Item, which the Parsing page does not have.

  1. Go to E-mail Processing > Preprocessing.
  2. Open a rule or start a new one, or select a rule and click Copy Item.
  3. Enter the Name and the Regular Expression.
  4. Under Processing options, set Remove text to after the expression or before the expression, and select or clear Stop processing remaining rules and Also remove matched text.
  5. To test the expression, enter a sample in the Sample Message editor and click Test Regular Expression.
  6. Save the rule.

A new rule starts with the name New Rule, after the expression, Also remove matched text selected, and Stop processing remaining rules cleared. A copy starts with the name of the original; when that name is already in the list, a number in parentheses is added and increased until the name matches no rule in the list. The System check box is read-only.

Test Regular Expression runs in your browser, not with the Mail Connector's own expression engine, so its result can differ from what the Mail Connector does with the same expression. The Settings App's test uses the Mail Connector's own processing. Line breaks in the expression are normalized, and . always matches line breaks. An expression that does not compile shows its error on the expression field; a match opens the Test Results dialog with the trimmed sample text.

Delete asks for confirmation and deletes every selected rule; it has no check for system rules.

Preprocessing rules of a job

A Mail Connector job has the tabs Settings, Processing, Preprocessing rules, and Sessions; Sessions is shown once the job exists. A rule opened from the job's Preprocessing rules tab is shown read-only. To change a rule itself, use the E-mail Processing > Preprocessing page.

Processing operations of a job

On a Mail Connector job's Processing tab, the Operations tree is display-only, with the note "The current logic is displayed for informational purposes only and can be configured using a dedicated desktop Settings application." To change the operations, use the Settings App; see Build processing operations. The Unmatched sender workflow, Email attachment, Incoming email handling, and End of session options of the tab stay editable; see Set Up Mail Connector Jobs.

Rules in the Settings App

Set the MessageID tag

  1. Go to E-mail Processing > MessageID Tag. The page explains that the subject line of incoming e-mail is searched for the MessageID tag to check whether the message belongs to an existing thread.
  2. In the MessageID Tag group, enter the Prefix and the Suffix. The disabled Example shows the prefix, T000001, and the suffix as you type.
  3. In the Position group, under Insert the MessageID value automatically, choose At the beginning of email subject line, At the end of email subject line, or No automatic insertion, use only the "MessageID Tag" system macro.

Both the prefix and the suffix are mandatory: a blank one refuses the save and names the field. Saving the page also rebuilds the system parsing rule, ObjectRef: it searches the subject for the new prefix and suffix, not case sensitive and with spaces removed, and any other change made to that rule's criteria is discarded.

Parsing rules in the Edit Field dialog

The Settings App edits a parsing rule in the Edit Field dialog. For a Logical rule, the dialog replaces Starting after and continues until with a single Check For value, which the extracted text is tested against. A rule's name must be unique among the parsing rules and cannot be the name of a built-in message field; either conflict blocks saving with an error message.

Create or edit a preprocessing rule

  1. Go to E-mail Processing > Preprocessing, the Preprocessing Rules list.
  2. Click New, or select a rule and click Open or Copy. The rule opens in the Reply Preprocessing Rule dialog.
  3. Enter the rule name and the regular expression.
  4. Choose one of Remove text after the Expression, before the Expression, or Ignore message (SPAM).
  5. Select or clear Also remove matched text and Stop processing remaining rules.
  6. To test the rule, type a sample HTML body in the Sample Message panel and click Test Regular Expression. The Test Result panel shows the result, produced by the same processing the Mail Connector applies at run time.
  7. Click OK to save the rule.

The dialog's check box that marks a rule as a system rule is read-only. A rule that is enabled on any job cannot be deleted, and neither can a system rule.

Apply preprocessing rules to a job

  1. Go to E-mail Processing > Mail Connector and open the job. It opens in a window titled Mail Connector, followed by the job name in brackets.
  2. Open the Preprocessing Rules tab. It lists every preprocessing rule: the rules linked to the job first, in their order, then the others. Each row has a check box and the rule's name, and the position of a row is its order.
  3. Select or clear a rule's check box, by clicking it or pressing Space.
  4. Use Move Up and Move Down to change the order. Open shows a rule read-only.
  5. Click OK. Nothing is saved until you do.

When the job is saved, every listed rule is stored with the job, selected or not, with its order.

Build processing operations

  1. Open the job from E-mail Processing > Mail Connector.
  2. On the Processing tab, in the Operations group, click New and choose Create Object, Update Object, "IF" Group, "ELSE" Group, or "ELSE IF" Group.
  3. Arrange the tree with Move Up and Move Down. The toolbar also has Expand All and Collapse All.
  4. In Create Persons:, choose the action to run for message senders that cannot be matched to a person. Press Delete or Backspace to clear it.
  5. Select or clear Attach original e-mail message to object/activity.
  6. Click OK.

Create Object and Update Object open the Operation dialog, which edits both kinds:

  • A create operation (Create New:) names a licensed object class and the create action to run for it.
  • An update operation also identifies the record to update, by an object ID value or by an expression built from the job's message fields and parsing rules. Once you choose the class, the row is labeled with it, for example Incident: or Work Order:. Its expression button, F(x), opens the Build Expression window, where the message fields and the parsing rules are listed under E-mail Fields. An entry whose name begins with $ is left out, and an entry that holds an object ID is listed only when the ID column of the table it refers to can be resolved. Choosing an entry writes it into the expression as %[EML <name>]%, for example %[EML ObjectRef]%.
  • Enable operation turns one operation off without deleting it, independently of whether the job itself is enabled.

An operation whose object class is not licensed cannot be opened or enabled again, and reports "Your license does not support this feature."

When you create a job, its tree is filled in for you. With the ITSM module licensed, it has update branches for Incident, Change Request, Problem, Work Order, Service Request, Approval Request, and Task, plus Purchase Order when ITAM is also licensed, and creates an Incident otherwise. With only ITAM licensed, it has branches for Purchase Order and Approval Request and no create fallback. Each branch uses the first enabled action, by rank, of the right kind for its class, and a class with no such action gets no branch. The default tree tests ObjectRef: when it holds a record, the branch for that record's class runs its update operation on that record.

In the field mappings of a Mail Connector action, the Settings App offers the message fields From, To, CC, BCC, Subject, Content (the message body), Sender, Sender Name, Sender E-mail, Priority, Auto-Generated, and Auto-Replied, plus every parsing rule, and both ObjectRef and InactiveObjectRef. Sender is offered only when the action's class is not the Person class. The mapping dialog shows the replacement mode only for update actions; on a create action it hides the control, but a mode already stored with a mapping still applies.

How the rules work

Order of processing

The Mail Connector takes each message through the same steps, in this order:

  1. It matches the sender to a person, or creates one.
  2. It sets the auto-generated and auto-replied flags, and then evaluates every parsing rule, including the system parsing rule that finds the record a reply answers.
  3. It normalizes the HTML body.
  4. It applies the preprocessing rules, only when ObjectRef holds a record: the message was matched, by its tag or by its headers, to an existing record whose status is not a closed one.
  5. It loads the message fields: the sender, the addresses, the subject, the body, the priority, and the auto flags.
  6. It runs the job's processing operations. At least one create or update operation must run, or the message is treated as unprocessed.

Because preprocessing comes after the parsing rules and before the message fields are handed to the operations, a preprocessing rule can only shorten the body that workflow finally sees.

Matching replies

Notifications carry a MessageID tag in the subject: a prefix, the object's OID, and a suffix. By default the prefix is MessageID:, the suffix is #, and the tag is added at the end of the subject. The tag is added only when the object has an OID, and not when the subject already contains it. The prefix and the suffix are both the text that is inserted and the text that incoming mail is searched for.

A reply is matched to a record in two ways, tried in order:

  1. The system parsing rule, ObjectRef, extracts the OID from the tag in the subject and looks the object up.
  2. If that finds nothing, the Mail Connector compares the message's References header, and then its In-Reply-To header, with the message IDs stored for objects, and takes the most recently created matching object.

ObjectRef takes the object when its status is not a closed one; a second field, InactiveObjectRef, takes it when the status is closed. With the header method, ObjectRef takes only an active object, and InactiveObjectRef takes the match only when the newest match is inactive. Either way, the tag is removed from the subject that workflow receives. A match on an object whose class is not marked as visible is dropped.

When the tag is not inserted automatically, a reply without a tag can be matched only through its headers.

The Mail Connector stores the message IDs that the header method uses. When a create operation makes a record, the message's ID is stored with the new record, provided the message's Message-ID header matches the message ID that the mail parser reported. When an update operation runs, the message's ID is added to the IDs stored for the record in ObjectRef; nothing is added when ObjectRef is empty or that record has no stored IDs yet. This is what lets a reply in a conversation that Alloy Navigator started be matched without a tag.

Parsing rules

A parsing rule cuts a value out of one part of the message: the sender, To, Cc, the subject, the body, the subject and then the body, the priority, or named additional headers. The value starts and ends at a text, a set of characters, a position, or the end of the line, with options for letter case and spaces.

The value is then checked against the rule's type:

  • Integer, Currency, Date, Date/Time, and Float values are checked. A value that does not convert is cleared and logged as "Unable to calculate E-Mail parser field '<name>'. '' is not a valid <type> value.", where the type is integer, currency, date, date and time, or float. The value is cleared before the log entry is written, so the quoted value in it is always empty. The failed conversion does not fail the e-mail message. Dates are read with the default date settings of the Mail Connector process.
  • String, Memo, HTML, and Script Object values are kept as extracted, without a check.
  • A Logical rule compares the extracted text with a test value, the Check For value of the Settings App's dialog, and yields true or false.
  • A Reference to object rule looks its value up as an object ID (OID), as the system parsing rule does.

Every header of the message is read, so a parsing rule can search named headers. From the headers, the Mail Connector also sets two flags that workflow receives as message fields: auto-replied and auto-generated. A message that is auto-replied also counts as auto-generated. The flags are set from the headers Auto-Submitted, X-Auto-Response-Suppress, X-Autorespond, X-Autoreply, Precedence, X-Precedence, X-Autogenerated, List-Unsubscribe, List-Id, and Feedback-ID, each with its own accepted values. The headers that Alloy Navigator puts on its own automatic notifications are among them, so a notification of this installation that comes back is recognized as automatic.

Preprocessing rules

A preprocessing rule is a regular expression plus what to do with the text it matches. The rules form one shared list; each job links the rules it uses, each with its own order and an enabled flag, so the same rule can serve several jobs at different positions. A run applies only the enabled rules of its own job, in their order.

Preprocessing rules run only on replies: only when ObjectRef holds a record, that is, when the message was matched to an existing record whose status is not a closed one. A message that creates a new record keeps its full body, including quoted history and signature, and so does a reply to a record whose status is closed.

A preprocessing rule is the only step that shortens the body of a reply before workflow receives it. When no rule matches, workflow gets the whole body, including any earlier correspondence that the sender's mail client quoted in the reply; the Mail Connector itself adds no history to the body.

Processing option Effect With "also remove matched text" cleared
Remove text after the expression Keeps everything up to the match The match itself is kept
Remove text before the expression Keeps everything after the match On an HTML body, the cut moves back to the opening tag that encloses the match, so the match is kept; on a plain-text body, the match is removed anyway
Ignore message (SPAM) Highlights the match in yellow; the body is not shortened Same

Rules apply in their order, each to the body as the previous rules left it. When no rule matches, the body stays as it was. A matching rule with "stop processing remaining rules" ends the pass; a rule that does not match never ends it. When a matching rule that stops processing is not the last enabled rule of the job, the pass ends without keeping its changes: the message keeps its original body.

A new 2026.1 database created with the Default Workflow (see Default Workflow Configuration) has three rules in the list, none of them a system rule:

Rule Regular expression
Replies in Mozilla Thunderbird messages (?is)On [0-9/]+.*wrote:
Replies in Microsoft Outlook messages (v.2) -----Original Message-----.*Subject:.+?\r\n
Replies in Microsoft Outlook messages From:.*Subject:.+?\r\n

The expressions read as follows:

  • Replies in Mozilla Thunderbird messages: in (?is), i makes the match case-insensitive, and s repeats what the engine does anyway. On [0-9/]+ is On, a space, and one or more digits and slashes. .*wrote: is any text, across line breaks, followed by wrote:.
  • Replies in Microsoft Outlook messages (v.2): the text -----Original Message-----, then any text up to Subject:, then .+?\r\n: at least one character, taking as few as possible, up to the first carriage return and line feed.
  • Replies in Microsoft Outlook messages: From:, then any text up to Subject:, then .+?\r\n as in the previous rule. This expression and the previous one are case-sensitive.

Each rule removes the text after its expression together with the matched text, so the body is cut where the first match begins; none of the rules stops processing. None of them is linked to a job, and the configuration creates no Mail Connector job, so no job applies them until you link them to it. The Mail Connector update actions supplied in the same configuration write the body of a reply into the details of the activity they add to the record; while no rule trims the reply, that activity carries whatever earlier correspondence the sender's mail client quoted. An upgraded database need not have these rules, and an administrator may have changed them.

Regular expressions

The Mail Connector uses a Perl-style regular expression engine and acts on the first match in the body. When the body is HTML, the expression is matched against the HTML source.

  • By default, matching is case-sensitive, . also matches line breaks, and repetition is greedy.
  • Supported escapes are \d, \w, \s, \b, \t, \n, and \r, along with back-references and the lazy forms *?, +?, and ??.
  • A (? group can be a comment, (?#...), or a set of inline modifiers from i, r, s, g, m, x, and -, for example (?is). Anything else, such as the lookahead (?=, does not compile and fails with an error that begins with "TRegExpr(comp):" and reports an unrecognized modifier.
  • POSIX character classes such as [:digit:] are not supported.

CC and BCC

The Cc header of a message is loaded as a message field next to To, From, and Bcc, and a parsing rule can extract a value from it. The Mail Connector does not turn a CC'd address into a recipient, a watcher, or a related person by itself: that is decided by the field mappings of the job's service actions; see Service Actions and Field Mappings.

The Mail Connector removes excluded addresses from the To and Cc values of a message before any parsing rule is evaluated. The addresses are listed in the [Exclude] section of the MailConnector.cfg file, next to the Mail Connector program. No other header is filtered. The list applies to POP3, IMAP4, Gmail, EWS, Exchange Online, and Graph jobs; it does not apply to the To, CC, and BCC values of MAPI jobs. A typical use is to keep the address of the support mailbox itself out of a CC list that workflow copies onto a ticket.

Processing operations

A job's operations form a tree of two kinds: one creates an object, the other runs an action on an existing object. The operations sit in groups of the form if, else if, and else. The usual shape is: if the message resolved to an object, branch on that object's class and run the matching update action; otherwise, create a new object.

A message counts as processed when at least one create or update operation actually runs. A processed message is always marked as read, except with POP3, which has no read flag; with POP3, a processed message is always deleted. Otherwise, whether a processed message is also deleted, and whether an unprocessed message is marked as read or deleted, is set in the job's incoming e-mail handling options; see Set Up Mail Connector Jobs.

Before the operations run, the sender is matched to a person by name and e-mail address. When no person matches, the job's person create action (Create Persons in the Settings App) creates one. The run depends on that action:

  • When the job has no person create action, the run fails right after it connects to the database.
  • When the action is not one of the enabled Mail Connector create and update actions, the run fails before any message is processed, with "Person Create action not specified.". When the action is a disabled Mail Connector create action of the Person class, the run fails with a message that the action is disabled instead. An enabled Mail Connector action of another kind or of another class passes this check.
  • With POP3, IMAP4, Gmail, and Graph, a run that finds the mailbox empty ends before the action is looked up among the enabled Mail Connector actions.

When a create operation runs, the action's template is applied first and the action's field mappings are written over it:

  1. Every row of the template is written into the new record. A calculated value, such as the current date and time, is resolved at that moment; other values are taken as stored. A read-only field is skipped, and a lookup value that does not exist in its list is stored empty.
  2. Each field mapping then writes its field according to its replacement mode: Always, unconditionally; When Target is empty, if the field is still empty; or When Source has value, if the message field is not empty.

A mapping set to Always therefore replaces a template value for the same field, even with an empty value: a message with an empty subject writes an empty value over the template value of a field mapped from the subject. For actions and their mappings, see Service Actions and Field Mappings.