Access Keys:
Skip to content (Access Key - 0)

Automatically extract ticket custom field values from email

Context

  • Request Tracker (RT) on help.mit.edu
  • You should be an RT queue administrator
  • You should be familiar with creating simple Templates and Scrips for your queue
  • You should already have custom fields for your queue

If you have not already created your desired custom fields, please refer to Creating and managing Request Tracker Custom Fields to learn about requesting, managing, and using custom fields with your queue.

Solution

Our central RT instance on help.mit.edu implements the RT extension ExtractCustomFieldValues from Best Practical. This article will give you a basic overview and two simple examples of how to set up your queue and format incoming email to allow you to automatically extract custom field values from an email message when a ticket is created, and automatically populate the respective custom fields with those values.

For more detail on how the extension works or to look at the source code please see ExtractCustomFieldValues on CPAN or at the corresponding wiki article ExtractCustomFieldValues on the Request Tracker wiki.

0. Summary

This solution has four basic steps. You will want to carefully read through them, and use the examples as guidance.

  1. Collect the information you need about your CUSTOM FIELDS
  2. Create a processing TEMPLATE for your queue
  3. Create a processing SCRIP for your queue that uses the template
  4. Create and test your WEB FORM that will send email to your queue

1. Collect the information you need about your Custom Fields

You will need to reference your Custom Field names in the template and in your web form, and the field names will need to match the field names exactly or the match will fail. The same is true for the field values So it's a good idea to collect the field names and values ahead of time and save them in a text file so you can cut and paste them into your template, rather than re-typing them by hand.

  1. Navigate to Tools > Configuration > Queues > Select and select your queue from the list of queues
  2. From the Queue Basics screen navigate to the Ticket Custom Fields tab

In the section titled Selected Custom Fields you will see a list of all the custom fields attached to your queue. You can record field names and field types from this screen. You can click on a field name to find a list of possible field values for fields of type Select One Value or Select Multiple Values.

Once you've recorded all the information you need about your custom fields, it's time to create your template.

2. Create a processing Template for your queue

  1. Navigate to Tools > Configuration > Queues > Select and select your queue from the list of queues
  2. From the Queue Basics screen navigate to the Templates heading and choose Create
    (If you already have a template you want to modify, choose Select instead.)
  3. Fill out the Create a new Template form as follows:
    • Name: Extract Custom Field Values (or another name that makes sense to you)
    • Description: Extract custom field values from incoming email (or a similar description)
    • Type: Simple
    • Content: the content of your template, using the Content subsection below as a guide
  4. Click the Create button to create and save your Template

Template Content

The general format for your template will be one line per Custom Field you want to match and extract. Each line will follow this format:

CF-Name | Headername or "Body" | MatchString(re) | Postcmd | Options |
CF-Name

This is the name of your Custom Field, exactly as it appears in RT, including spaces and any special characters. For example, if your custom field is called User-visible Impact, you would put User-visible Impact into that first slot.

Headername or "Body"

This will either be the name of the email header you want to map to this custom field, or the literal word Body if you want RT to look for the value in the body of the email message instead of the header. The name of the email header should be similar to the name of your custom field, but since you can't have spaces in email header names it can't be identical. It is also customary to begin non-standard email header names with X- and identify what the header contains.

Convention for our templates, and what's used in the example below, will be a header name that begins with X-QCF- followed by the name of the custom field with spaces replaced by dashes. For example, if your custom field is called User-visible Impact, you would put X-QCF-User-visible-Impact into this second slot.

MatchString(re)

This is a regular expression that tells RT what to match after the email header field, or what to match in the message body, to find the value of the custom field. If you are using email headers, this will almost always be .* (a period followed by an asterisk), which will tell RT to match the entire header value. If you are looking for custom field values in the message body, this regular expression will be more complex. See the simple example below for an expression you can use.

Postcmd

An optional command to run after the value has been matched. This will usually be empty.

Options

You can leave this blank or put the option q here, which will tell RT to quietly populate the custom field and not record a history transaction for each custom field it sets (the way it does if you set or change a custom field in the web interface). You will usually want this set to q.

This may not make total sense just yet, but hopefully the examples at the end of this article will help to clear things up.

3. Create a processing Scrip for your queue that uses the Template

Now that you have your Template, it's time to create the Scrip to process and apply your template whenever a new ticket is created by email.

  1. Navigate to Tools > Configuration > Queues > Select and select your queue from the list of queues
  2. From the Queue Basics screen navigate to the Scrips heading and choose Create
    (If you already have a scrip you want to modify, choose Select instead.)
  3. Fill out the Create a new Scrip form as follows:
    • Description: Extract custom field values from email header (or something similar that makes sense to you)
    • Condition: On Email Create
    • Action: Extract Custom Field Values
    • Template: Extract Custom Field Values (or whatever you called your template when you created it)
    • Stage: TransactionCreate (the default value)
  4. Click the Create button to create and save your Scrip

All the RT work is done now. You will still need to set up your web form or whatever other mechanism you plan to use to send formatted email to RT.

4. Create and test your Web Form that will send email to your queue

There are many different mechanisms for creating web forms and sending emails based on forms that are submitted. Any of them will work as long as you can format the mail that is sent so that the correct email headers and form values are inserted, or the form values are appropriately formatted in the message body. See the examples below for the kinds of email messages you will need to send.

Two examples

Here are two example templates and corresponding email message examples that may help to clarify how this all works.

The queue used in our examples has four Custom Fields attached to it, which are:

  • Service Area: a "select one value" field with the possible values of "Human Resources", "Concessions", "Web Services", and "Pet Care")
  • User-visible Impact: a "select one value" field with possible values "Yes" or "No")
  • Must be done by: a "date" field, and
  • Interaction Count: an "enter one value" field that expects a number

Example 1: Custom Email Headers

In our first example, the template is set up for email that contains special header lines with the field values. This would commonly be generated by a web form, and is the best way to go if you want to keep the values from cluttering up the message body (and possibly confusing the client).

Example 1: The Template
Name: Extract Custom Field Values
Description: Extract custom field values from header of incoming email
Type: Simple
Content:
Service Area|X-QCF-Service-Area|.*||q|
User-visible Impact|X-QCF-User-visible-Impact|.*||q|
Must be done by|X-QCF-Must-be-done-by|.*||q|
Interaction Count|X-QCF-Interaction-Count|.*||q|
Example 1: The Email Message

Here is an example of a correctly formatted email message that will work with the template above to extract values from the email header and place them into custom fields in the new ticket.

To: tooltime-test@mit.edu
From: othomas@mit.edu
Date: Fri, 14 Feb 2014 16:08:26 -0500
X-QCF-Service-Area: Pet Care
X-QCF-User-visible-Impact: Yes
X-QCF-Must-be-done-by: 12/24/2014
X-QCF-Interaction-Count: 13
Subject: A question from Oliver Thomas (via web form)

Hello, I've filled out your web form requesting someone to take care of
Fluffy on Christmas Eve 2014. It took me quite a few tries to get the
form right. Eagerly awaiting your response!

-Oliver

Example 2: Embedded in Email Body

In our second example the template is set up for email that contains name-value pairs embedded in the body of the message. This can be a little more confusing for the client, on the other hand they will easily see exactly what values were submitted by them.

Example 2: The Template

As mentioned above, the template for extracting values from the body of an email is a little more complicated. In the case of the email header, the mail system takes care of parsing the email header into name-value pairs for you. (The name is the header name, the colon is the separator, and everything after the colon becomes the value.) When extracting values from the body, however, you need to tell the system what to look for in the body to locate the beginning of the value, and how much to pull out of the message body after the beginning to use as the value.

We do this with the regular expression in the template. The regular expression will look for a string in the body. A pair of parenthesis inside the regular expression will indicate which portion of the found string to use as the field value.

Name: Extract Custom Field Values
Description: Extract custom field values from body of incoming email
Type: Simple
Content:
Service Area|Body|Service Area-> (.+)$||q|
User-visible Impact|Body|User-visible Impact-> (.+)$||q|
Must be done by|Body|Must be done by-> (.+)$||q|
Interaction Count|Body|Interaction Count-> (.+)$||q|

To dissect one of the above lines, let's take a look at the first one. The beginning, Service Area, tells the template the name of the field this line is extracting a value for. The second slot, Body, tells the template to look in the body of the email message. The third slot, Service Area-> (.+)$, tells the template to look for a string that begins with Service Area-> (including the space) and to pull out everything after that up to the end of the line, matched by the dollar sign. We're using the -> here only because many other separators, like :, have special meanings inside regular expressions.

Example 2: The Email Message

Here is an example of a correctly formatted email message that will work with the template above to extract values from the email body and place them into custom fields in the new ticket.

To: tooltime-test@mit.edu
From: othomas@mit.edu
Date: Fri, 14 Feb 2014 16:08:26 -0500
Subject: A question from Oliver Thomas (via web form)

Hello, I've filled out your web form requesting someone to take care of
Fluffy on Christmas Eve 2014. It took me quite a few tries to get the
form right. Eagerly awaiting your response!

-Oliver

Submitted form values below
===========================
Service Area-> Pet Care
User-visible Impact-> Yes
Must be done by-> 12/24/2014
Interaction Count-> 13

See also

IS&T Contributions

Documentation and information provided by IS&T staff members


Last Modified:

July 01, 2020

Get Help

Request help
from the Help Desk
Report a security incident
to the Security Team
Labels:
rt rt Delete
rt4 rt4 Delete
request request Delete
tracker tracker Delete
ticket ticket Delete
fields fields Delete
email email Delete
e-mail e-mail Delete
queue queue Delete
c-rt-custom c-rt-custom Delete
c-rt-queue-admin c-rt-queue-admin Delete
c-rt-queue-admin-shortcut c-rt-queue-admin-shortcut Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
Feedback
This product/service is:
Easy to use
Average
Difficult to use

This article is:
Helpful
Inaccurate
Obsolete
Adaptavist Theme Builder (4.2.3) Powered by Atlassian Confluence 3.5.13, the Enterprise Wiki