WO2004086250A1 - E-mail management system and method - Google Patents

Abstract

An e-mail management method for managing electronic mail messages includes classifying incoming e-mail messages using ripple down rules stored in a rule base, wherein each rule is defined by at least one or more rule conditions and a rule conclusion. According to the method, if a particular e-mail message is not appropriately classified according to any existing rule in the rule base a new rule is built by selecting rule conditions based upon characterising features of the particular e-mail message, specifying a rule conclusion to be assigned to the particular email message and to each subsequent message that is classified according to the new rule, and adding the rule defined by said rule conditions and rule conclusion to the rule base. A system and software product implementing the method are also provided.

Description

E-MAIL MANAGEMENT SYSTEM AND METHOD

FIELD OF THE INVENTION

The present invention relates to an electronic mail management system and method which addresses the process of electronic mail management.

BACKGROUND OF THE INVENTION

As electronic mail (hereinafter referred to as e-mail) becomes a more prevalent form of communication, dealing with e-mail is becoming more and more costly and time consuming. There is a real need for methods and systems that can improve the management of personal and organisational e-mail. Preferably, such e-mail management "assistants" should be provided as an add-on software solution that relies on existing architectures and can be incorporated into or added as a module to existing e-mail programs. Such e-mail "assistants" should be able to 'manage' incoming e-mails by initially sorting e-mail messages into selectable virtual folders, prioritising the messages (based on recipient and not sender criteria), "reading" message contents and replying (automatically or serni- auiomatically) where necessary and/or appropriate, and archiving or deleting mail items based on user preferences. Despite much research into the development of e-mail assistants, current e-mail clients and research prototypes provide only limited help in addressing problems of communication overload.

Generally, the process of e-mail management would typically include, for a single e-mail message, the message first arriving in a user's "Inbox" and perhaps being sorted and/or prioritised (using set parameters) for display to the user before being read. Then, the message may be left in the Inbox for some time before the user reads and eventually replies to it. It is common for messages to be subsequently archived and/or eventually deleted, where archived messages may be later culled (deleted). However, for large volumes of e-mail messages such an e-mail management process becomes problematic.

In Maes, "Agents that Reduce Work and Information Overload", Communications of the ACM, 37(7), pp. 31-40, 1994, competence and trust are identified as the two main criteria for the effectiveness of personal assistants. In this context, "competence" refers to the acquisition of knowledge by the assistant and the use of this knowledge to aid the user, and "trust" refers to the user's willingness to delegate tasks to the assistant. Of course, this assumes that the agent is "competent" to the extent that the user model it develops is correct and able to be employed to help the user. In this more everyday sense, competence refers to the correctness of the agent's behaviour. To these, we can add "usability", which is the ease of interaction between the user and the assistant, including the ease with which knowledge is acquired by the agent.

In the case of e-mail, user group studies (see Singh et al., "A Design for the Effective Use of Corporate E-mail", Research Report 27, Centre for International Research on Communication and Information Technologies, RMIT University, Melbourne, Australia, 2000) have endorsed Maes' view by showing that users' lack of trust in their systems is a major barrier to the use of e-mail in Organisations, and this would apply even more to the use of e-mail assistants. Accordingly, the main reason that existing prototype e-mail assistants are not in widespread use appears to be that such systems (typically based on machine learning techniques, such as Bayesian classification) do not provide competence (in the sense of correctness) and hence do not engender the trust of their users. Another reason why existing e-mail assistant prototypes are of limited use appears to be that they typically address only one of the above aspects of the e- rnail management process. An effective e-mail assistant should ideally address most but preferably all aspects of the process.

Another difficulty in developing a general-purpose email assistant is that users vary widely in the type and quantity of e-mail they receive and their behaviour in managing their e-mail. Some users receive more than a hundred e- mails per day, others only very few. There have been surprisingly few studies of exactly how users manage e-mail, but it appears that most people use folders for filing their e-mails, though some keep all of their mail in the Inbox and rely on efficient search functions to re-locate messages. Of those who use folders, most may organise folders based on the content or sender of a message, some may organise folders based on the action to be taken in response to the e-mail (reply, delete, etc.), while others may organise folders based on the urgency of the required response. Thus, for an e-mail assistant to be widely usable, it must cater for the wide variety of users and the different strategies they have adopted to cope with managing their e-mail.

Some research has been conducted in an attempt to address these aspects of e-mail management. For example, Segal et al., "Incremental Learning in Swift-File", In Proceedings of the Seventeenth International Conference on Machine Learning, pp. 863-870, 2000, discloses a prototype assistant in an extension of the Lotus-Notes® environment, which presents a user with three options for suggesting the folder(s) where a message should be archived, with the suggested folders being calculated using the tf-ύff method. Three options are presented because this provides an acceptable degree of accuracy (73% to 90% in trials). However, the use of three options means that this assistant can only be used relatively "late" in the e-mail management process, certainly after the user has opened and scanned the message. On the other hand, sorting incoming messages using the first recommendation of this assistant does not give an acceptable degree of accuracy (ranging from 52% to 76% in trials). . . Cohen, in "Learning Pules That Classify E-Mail", In Machine Learning in Information Access: Papers from the 1996 Spring Symposium, pp. 18-25, 1996, compares two techniques for e-mail sorting into folders, the above mentioned tt-irff approach and a technique linov n sis RIPPER, a rule induction algorithm. In this latter technique, once a training set of messages, classified into folders, is provided, keyword spotting is used to repeatedly add classification rules into rule sets for each folder until all positive examples are covered while excluding all negative examples. The initial results of this technique appear impressive, with accuracy between 87% and 94% on varying sized data sets when 80% of the data is used for training and 20% for testing, but it appears this assumes all messages are classified in exactly one of the given folders. Thus, these statistics address accuracy, but not coverage of the data, and the accuracy is attained using a "batch" method on a sufficiently large training set.

Having regard to the above mentioned shortcomings, it would be desirable to create an e-mail management system and method for providing an e-mail assistant which is competent and trustworthy, usable and applicable to a wide variety of users, while addressing the above mentioned aspects of the e-mail management process.

SUMMARY OF THE INVENTION In one aspect, the present invention provides an e-mail management method for managing electronic mail messages, the method including the steps of: classifying incoming or previously archived e-mail messages using ripple down rules stored in a rule base, wherein each rule is defined by at least one or more rule conditions and a rule conclusion; and if a particular e-mail message is not appropriately classified according to any existing rule in the rule base, building a new rule using the steps of: selecting rule conditions based upon characterising features of the particular e-mail message; specifying a rule conclusion to be assigned to the particular email message a d to each subsequent message that is classified according to the new rule; and adding the rule defined by said rule conditions and rule conclusion to the rule base. It is accordingly a notable advantage of the method that it is incremental in nature. Specifically, starting with an empty rule base, rules may be added whil processing particular examples. Thus, in contrast to known machine learning algorithms, no training set is required before the method of the present invention can be used. Furthermore, the method enables new rules to be created in the context of a specific example e-mail message by identifying characteristic features of the message that distinguish it from messages that should be classified differently, and without the need to examine the existing rules in the rule base.

Preferably the step of specifying a rule conclusion includes specifying one or more of a virtual folder in which the message is to be stored, a priority of the message, and/or an action to be taken in relation to the message. Other types of rule conclusions can be defined and implemented in accordance with the needs of a user. As noted, the method may be implemented to deal with incoming e-mail messages immediately upon receipt or after having been saved for subsequent processing in a dedicated folder.

Building a new rule may further include the step of using a machine learning algorithm to formulate a suggestion, wherein the suggestion includes one or more rule conditions and/or a rule conclusion. The step of selecting rule conditions may include accepting or rejecting rule conditions in the suggestion, and the step of specifying a rule conclusion may include accepting or rejecting a rule conclusion in the suggestion. In a particularly preferred embodiment, the machine learning algorithm includes a naive Bayes classification method adapted to suggest a rule conclusion including a virtual folder in which to store the message selected from a set of existing virtual folders and optionally to further suggest rule conditions including a set of keywords selected from the message that maximise the conditional probability that the message is correctly allocated to the suggested virtual folder.

The actions that may be undertaken in the rule conclusion can include automatic and semi-automatic deletion of unwanted emails which are identified by user specified characteristics, eg if an email has a specified sender, certain key words, error messages or virus alert notifications, without actual storage into a virtual (trash) folder; automatic and semiautomatic reply notification^.; diary update entries; etc. As noted above, as the rule base is built up incrementally to suit the needs and preferences of the email user, the actions that are to be undertaken in relation to any given message will vary, and actually may include multiple actions in relation to the same message, eg save into a folder and send out automatic receipt notification, which may not apply to other messages.

Accordingly, the method would preferentially use classifying steps that include using multiple ripple down rules, such that a message may be classified according to multiple rules and assigned multiple conclusions. The characterising features of e-mail messages upon which the rule conditions are selected may include user specified key words or key phrases in the body, header or attachment title of an email. The key words may include subject matter identifiers or information associated with the sender of an email, as would be used by users employing a fully manual, user email archiving system.

In another aspect, the present invention provides an e-mail management system for managing electronic mail messages including: a rule base containing zero or more ripple down rules, wherein each rule is defined by at least one or more rule conditions and a rule conclusion; a classifier for applying the ripple down rules stored in the rule base for the classification of e-mail messages; and a rule building interface for defining and adding a new rule to the rule base by selecting rule conditions based upon characterising features of a particular e- mail message, and for specifying a rule conclusion to be assigned to each email message classified according to the new rule by the classifier; wherein the rule base is initially preferably empty and is built up incrementally by use of the rule building interface. It is preferred that the characterising features used in selecting rule conditions include one or more l eywords and/or phrases appearing in the headers, body and/or attachments body of the particular message.

In preferred embodiments, the system further includes a computer- implemented user interface for displaying said particular email message to a user, and for enabling the user to select and view the rule conditions and specify the corresponding conclusion.

It is advantageous for the rule building interface also to store a copy of the particular e-mail message used to define a new rule. When a further new rule is subsequently defined, the rule building interface may then identify prior messages used to define existing rules that also satisfy the rule conditions of the new rule, and the user interface may display said prior messages to the user and enable the user either to add further rule conditions to distinguish the present message from said prior messages, or alternatively to accept the new rule with the consequence that the new rule conclusion will also apply to said prior messages. It is particularly preferred that the rule base is logically structured as a hierarchy of rules, such as a tree, wherein a message that satisfies the rule conditions of a higher level rule will be classified by the classifier according to the higher level rule and assigned the higher level rule condition, unless the message also satisfies the rule conditions of an associated rule at the adjacent lower level, in which case the message is classified by the classifier according to a lower level rule, such that rules at lower levels of the hierarchy define exceptions to rules at higher levels. A new rule that defines an exception to an existing rule may be added to the rule base by the rule building interface at a level of the hierarchy immediately below the rule to which it is an exception, whereas a new rule that serves to classify a previously unclassified message or which defines an additional conclusion for a previously classified message may be added to the rule base by the rule building interface at the first level of the hierarchy. Preferably, the user interface enables the user to specify that the operation of a rule is to be stopped, and the classifier subsequently does not apply stopped rules. However, any exceptions to stopped rules, in the rule base, may still be applied if required. In preferred embodiments, rules are not deleted from the rule base, and stopped rules may be reinstated by the user using the user interface. In yet another aspect, the present invention provides a software product for use in a computer with which there is associated a rule base containing zero or more ripple down rules, wherein each rule is defined by at least one or more rule conditions and a rule conclusion, the software product being adapted to effect provision of an e-mail management service for use by a client e-mail application for managing electronic mail messages, the software product including: computer instruction code to instruct said computer to provide a classifier for applying the ripple down rules stored in the rule base for the classification of e- mail messages; and computer instruction code to instruct said computer to provide a rule building interface for defining a new rule including rule conditions based upon characterising features of a particular e-mail message and a rule conclusion to be assigned to each email message classified according to the new rule by the classifier.

The client e-mail application may be integrated with the software product providing the e-mail management service. Alternatively, the client e-mail application may be a separate software product. The client e-mail application may execute on the same computer as the e-mail management service, or on a different computer, whereby the client application is enabled to access the service over a data communications network to which both computers are connected. Communication between the client application and the email management service may be via remote method invocation (RMI).

The foregoing and further features and advantages of the present invention will be apparent to those skilled in the art from the following description of preferred embodiments with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 shows an exemplary Ripple Down Rule (RDR)-rule base tree for use in an email management assistant in accordance with the present invention; Figure 2 shows the RDR rule-base tree of fig. 1 after refinement (ie adding of additional rules) by a user in applying the email management method according to the present invention;

Figure 3 shows an exemplary main user interface window of the e-mail management assistant according to the present invention;

Figure 4 shows an exemplar/ rule building user interface of the e-rnail management assistant according to the present invention for implementing the building of management rules;

Figure 5 shows an exemplary window of the e-mail management assistant according to the present invention for viewing a cornerstone case indicating the example used by the user to create a rule;

Figure 6 shows an exemplary architecture of the e-mail management system (assistant) of the present invention;

Figure 7 is a class diagram used in the design of the e-mail management system and method of the present invention; and

Figure 8 shows a rule building flow chart employed with the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS The e-mail management system and method of the present invention attains a high degree of accuracy on e-mail classification by using a rule-based approach called Ripple Down Rules (RDR), as the basis of rule construction. Systems which typically use known forms of RDR are in the field of medical diag- nosis (see Compton et al., "A Philosophical Basis for Knowledge Acquisition", Knowledge Acquisition, 2(3), pp. 241-257, 1990). However, the present invention employs a specifically developed RDR system to implement an e-mail management system and method, and preferably further incorporates a Bayesian classification algorithm to improve the usability by suggesting keywords/phrases for filing e-mail messages into given folders.

In contrast to traditional rule-based systems; RDR systems provide extensive help to the user in defining rules and maintaining the consistency of a rule-base. As such, by employing RDR in the e-mail management system and method of the present invention, the ease of use is improved and the multiple aspects of the e-mail management process, from sorting messages into virtual folders to archiving messages and creating semiautomatic replies, are addressed.

Basically, an RDR system is one of 'if-then' rules organised into a hierarchy of rules and exceptions (exceptions to rules may themselves have exceptions, etc.). An RDR base is organised as a tree structure in which each nod a contains a rule and the children of a node represent exceptions to the rule. Such an RDR system can be viewed as a classification system when the conclusion of a rule defines a class to be assigned to an example. The use of this type of RDR supports an incremental approach to knowledge acquisition, since when starting with an empty rule-base, the user adds a rule to correctly classify new examples as they are encountered, and when an example is classified incorrectly, the user may define exceptions to the rules that resulted in the misclassification.

There are various types of RDR system. The preferred implementation of an email management assistant in accordance with the invention uses Multiple Classification RDR (see Kang et al., "Multiple Classification Ripple Down Rules: Evaluation and Possibilities", In Proceedings of the 9^th AAAI-Sponsored Banff Knowledge Acquisition for Knowledge-Based Systems Workshop, pp. 17.1-17.20, 1995) implemented specifically for e-mail management. With this type of rule- base, an example has a classification resulting from multiple conclusions in the rule-base. These conclusions are found by following every path in the RDR rule- tree to the most specific node applicable to the example. The classification is then the set of conclusions from all such nodes. Figure 1 shows an exemplary RDR rule-tree in accordance with the present invention. The root node is a dummy node giving no conclusion. Rules 1 to 5 are the rules in the tree that are not defined as exceptions to other rules, while Rules 6 and 7 are exceptions to Rule 1, etc. An example with features {a, b, c, e, f, x}, which might correspond to the sender of an e-mail message or keywords in the body of a message is considered. At the top level of the tree, Rules 1, 3 and 5 are applicable to the example. However, Rule 7, which is an exception to Rule 1, also applies to the example. So the most specific rules applying to the example are Rules 7, 3 and 5, giving the classification {CRC, Friends}. Note how in this example, there are two rules applying to the message that give the conclusion "Friends", but this makes no difference to the final classification.

An RDR system provides extensive support to the user in defining rules, and this is where RDR systems gain their power compared to traditional rule- based expert systems. The user does not ever need to examine the rule-base in order to define new rules, rather the user only needs to be able to define a new rule that correctly classifies a given example, and the system can determine where those rules should be placed in the hierarchy.

Typically, in the RDR system, rules are only refined or added, and not modified or removed. Refinement is the creation of an exception rule to correct a misclassification, while addition refers to adding a new rule at the top level of the tree. Rules are always defined in the context of a specific example and in relation to other "relevant" examples, and are not meant to be absolute. To define a new rule "in context", the user has only to identify features of the example that distinguish it from examples that should be classified differently.

I The RDR system can determine which examples need to be considered by checking the rule-base for prior examples that are also covered by the proposed rule. To do this, the system maintains for each rule the example that was used when the user created the rule (called the "cornerstone case"). The cornerstone cases of potentially conflicting rules can be presented to the user as the new rule is being defined, prompting the user to either accept the new conclusion as also applying to the existing cornerstone case, or to identify additional features to further distinguish the current example. The process continues iteratively until the user accepts the new rule and its consequences. This makes the task of defining rules much simpler than if the user is required to define a rule-base independently of the context of a specific example (as is required in traditional rule-based expert systems, and in the rule- bases of current e-mail filtering systems). In this way, an RDR system can be used by "end users" rather than only by knowledge engineers.

Consider again the exemplary RDR rule-tree shown in Figure 1. A message with features {a, c, x} is classified as {Friends}. Suppose the user wishes to change this classification to {ARC}, so needs to determine additional features of the example that will lead to the new conclusion. By using the RDR system, the user is certain that the selected features y and z do not result in incorrect conclusions being generated using other rules. For example, as the feature y of the message is selected, the user is shown any cornerstone cases that satisfy the conditions of the new rule, including y (there may be such cases even though y does not occur in the conditions of any existing rule).

The new conclusion {ARC} applies to such cases, so the user is prompted to determine whether or not this is the intended behaviour. If it is not, the user must identify additional features (here ∑) that further discriminate the current message from these cornerstone cases. Finally, two instances of the rule 'if y and z then ARC are added as exceptions to Rules 3 and 5 (so that both conclusions are overridden by more specific rules), as shown in Figure 2. The example will now be classified correctly.

In summary, the main advantages of RDR as a knowledge acquisition technique are the high degree of accuracy, the validation of conclusions that the system can provide and the ease of constructing the rule-base. The high degree of accuracy is attained because the user refines the rule-base to maintain consistency whenever necessary, so the system always classifies seen examples according to the user's (current) intentions (as the rule-base is being constructed, the user can change the classification of previously classified examples by creating exception rules). Total accuracy is not guaranteed, however, because the rules defined by the user are typically over-general, in that rules also apply to unseen examples in perhaps unforeseen ways. However, in practice, since rules apply only in specific contexts, misclassification applies to a relatively small proportion of unseen examples. More usual in practice is that the user's rule-base as a whole covers too precisely the set of existing examples, and the RDR system tends to be "conservative" and fails to generalise to unseen examples. Thus, in comparison to machine learning systems, RDR systems gain high accuracy (precision) at the expense of coverage (recall) of a set of examples, while typically machine learning algorithms have higher coverage but lower accuracy.

As mentioned above, typically rules are not deleted from an RDR rule- base. To achieve the effect of deleting a rule, a rule may be "stopped", which means that any conclusions that would normally be generated from the rule and any of its exceptions (and their exceptions, etc.) are ignored. The rules themselves, however, are retained so that if the rule is reinstated, the rule and all its exceptions can be recovered. One potential disadvantage of this approach is that an RDR rule-base can grow to contain a large number of redundant rules, especially in dynamic domains where the user may often want to stop the application of existing rules. However, with the hierarchical structure of an RDR rule-base, removing redundant rules is a nontrivial operation (see Wada ei al., "Extension of the RDR Method That Can Adapt to Environmental Changes and Acquire l .nowled.ge from Both Experts and Data", In PRICAI 2002: Trends in Artificial Intelligence, pp. 218-227, Springer- Verlag, Berlin, 2002), and in practice, has not proven to be necessary.

The make-up and operation of an e-mail management system and method according to the present invention and which incorporates or is based on a RDR rule base developed in an e-mail specific context, is described herein below in more detail. This provides an email management assistant which addresses as many aspects of e-mail management as possible for as wide a variety of users as possible. Competence of the present invention's e-mail management assistant is assured by the use of RDR systems for classification, because of the high degree of accuracy of the constructed rule-bases which leads to a high degree of trust in the system by the users. The present invention's e-mail management assistant further includes a user interface designed to make it easy for the user to view e- mail, and especially, to define rules. One reason that RDR is particularly suitable for e-mail classification is the incremental nature of the approach. There is no need, as there is with typical machine learning algorithms, for large training sets. The user starts with an empty rule-base, and gradually builds up the rule-base to classify incoming messages, leading to extensive personalisation of the rule-base, which enables the e-mail management assistant of the present invention to be used by a wide variety of users.

The rules used by the present invention are 'if-then' rules applied in e-mail specific contexts. The condition part of a rule may be keywords or keyphrases drawn from any of the e-mail message headers (e.g., Subject, From, etc.), or from the body of the message, or even possibly from attachments of the message. For the rule's conclusion, the user chooses a combination of a virtual display folder (for sorting), a priority (high, medium or low) and an action (read/reply then delete/archive). Using rules with such combined conclusions, rather than having separate rules for the virtual folder, priority and action, leads to fewer rules overall and a more consistent user interface for rule building (see Figure 4).

The virtual folder is used for sorting and displaying messages to the user, and is based on an IEMS system (see Crawford et al., "IEMS - The Intelligent Email Sorter", In Proceedings of the Nineteenth International Conference on Machine Learning, pp. 83-90, 2002). An exemplary main user interface window of the present invention is shown in Figure 3. Messages in the Inbox are grouped into a number of virtual folders and displayed together in the virtual folder(s). If there is no virtual folder associated with a message, as is the case when no rule applies to the message, the message is left in a virtual folder called 'Inbox'. Virtual folders may correspond to folders for archiving, but this is not essential to the present invention (e.g., a fine-grained set of folders can be used for archiving, while a more coarse-grained set of virtual folders can be used for display). Messages may be sorted by date, subject, sender, priority or action, but are still grouped together within virtual folders. The email management assistant uses multiple classification RDR. Thus, more than one rule may apply to a message, and the classification is a set of rule conclusions, i.e., in the embodiment illustrated, a set of combinations of virtual folder, priority and action. If there is more than one virtual folder for a message, the message headers are displayed under both virtual folders in the Inbox, but there is only ever one copy of the message in the Inbox file (so deleting one such message removes both sets of headers from the Inbox display). Within each virtual folder, the priority (and suggested action) of a message is that of the highest priority assigned to the message by a rule for that virtual folder, where multiple priorities and actions are selectable by the user. The user has the option to display the conditions leading to the given conclusions (keywords and keyphrases are highlighted in the message display), and to see the rule(s) that resulted in the current conclusion. As described above, one main advantage of RDR is the ease with which rules can be created. The rules are always defined to classify a given e-mail message. When the user of the e-mail management assistant wishes to define a rule, the message is displayed in a separate window and the user simply selects various words and phrases in the message headers and body. As shown in Figure 4, during the operation of rule building, a rule building user interface window is displayed and the rule being constructed is displayed in the upper portion of the window. In the case of a rniεclasεiiϊcatiori, the rule is preferably displayed along with any conditions of the rule inherited from its parent rule(s) in the rule-base, and an exception rule is added in order to refine the rule-base. As the user selects features for the condition part of the rule from amongst the message header and body, the rule being defined is updated in the upper window, but more importantly, an indication of the number of rules that conflict with the new rule is displayed. Each conflicting rule represents a possibly unintended side effect of the rule that the present rule will also apply to other previous examples. This indicates that the rule's conditions may be too general and need further specialisation.

The user can examine examples of the conflicting rules (the cornerstone cases, which are the messages that were used to create the rules in this case) and either accepts the new classification of the previous cornerstone case, or must further specialise the conditions of the newly constructed rule. Figure 5 shows an exemplary window for viewing cornerstone cases. Using machine learning techniques (see below), the assistant is able to suggest some typical keywords for the intended folder. For example, in Figure 4, the user has selected one such keyword "SPA" that has been incorporated into the condition of the new rule. Any occurrences of the keyword are highlighted in the body of the message. Finally, the rule is automatically created and added to the rule-base based on the user's chosen options. As such, the user does not have to examine the rule-base to determine where the rule(s) should be added. If the new rule is for a message that has been misclassified, the default action is to modify the conclusion and add exception rules for each rule generating the incorrect conclusion. For a message which has. not been classified or where the user wants to define an additional conclusion for a previously classified message, a new rule is added at the top level of the rule-tree.

The coverage of the above-described RDR classification algorithms used by in the assistant may be improved by incorporating machine learning algorithms, namely by applying one or more machine learning techniques when the rule-base fails to classify an example. However, such machine learning tech- niques generally rely on e: .tensive training sets which are not necessarily available for all users or whose characteristics can van/ widely between users.

Thus, rather than use machine learning algorithms to attempt to improve classification accuracy, the usability of the inventive assistant is improved by using a machine learning technique, namely, Bayesian classification, in at least one of the following three ways: ( i i to suggest a folder for filing a gh'en e-mail; (2) to suggest which keywords in a message are useful indicators of a given folder; and (3) to build "interest profiles" that characterise the content of a user's folders. This is the list of suggestions shown to the user in the rule building user interface window shown in Figure 4. Of these suggestions, (2), that is suggesting words from particular messages to be used in rule construction, is the most useful because rule construction is the most challenging part of using the RDR system and this also is the most accurate application of Bayesian classification.

The basic details of the Naive Bayes classification methods are known, see Mitchell, "Machine Learning", McGraw Hill, New York, NY, 1997. However, for the specific application in an email management assistant according to the invention, data is taken from the user's existing folders and for a given folder f the quantity P(f\ w) is calculated for all keywords w that occur in any message (in Mitchell's method, all such probabilities are nonzero). The folder / with the highest value for P(f\W), where I/Vis the set of words occurring in a message, is the one where the message is suggested to be archived, where using an independence assumption, P(f\W) is the product over the words w in l/Vof P(f\%v). Given a particular message which the user has selected to file in folder the keywords suggested to the user for defining rules are those words w in the message that maximise P(f\w). Finally, the "interest profile" for a folder is simply a list of words w (after removing stop-words) ranked according to P(f\w).

By using the Naive Bayes classification algorithm, it is assumed that the appearance of each word in a message in a folder is conditionally independent from that of any other word, which is an extremely simplifying assumption that has been shown to work in practice in many applications. Also, only single words are used in the algorithm of the present invention, rather than phrases, for which there is typically insufficient statistical information in a user's e-mail boxes.

The e-mail management assistant of the present invention preferably uses a multi-tiered basic architecture based on Java™ (JSD ) and Javaϊvlail™ technologies, although others are also employable. Howvere, the use of Java ensures that the e-rnail management assistant is able to be deployed across many different platforms on different devices, and moreover, enables more advanced features such as Java 2 Micro Edition (J2ME), PersonalJava, Java Messaging Sen/ice (JMS) and Jini i .etworl. Technology to be added as required. The client/server paradigm i chosen in order to support the evaluation environment (it enables multiple users to have their own RDR rule-base but share use of the RDR software) and to facilitate deployment of the system and method (possibly in larger-scale commercial environments). Communication between client and server is through remote method invocation (RMI).

As illustrated in Fig. 6, the e-mail management system employs two main clusters, a transaction processing or server cluster 1 and a presentation or client cluster 10. The client cluster 10 includes different graphical user interfaces 11 for users to interact with the e-mail management system. For example, the client cluster 10 can include interfaces that can be run on a desktop or laptop computer, using any platform that supports Java Virtual Machine Version 1.4, including Windows and Unix. On the other hand, the server cluster 1 is where the core objects for the e- mail management system are defined, including mail objects and the RDR rule- base. The servers of the server cluster 1 are designed to be remote servers, hence when a server starts, it broadcasts its remote services and waits for a client in the client cluster 10 to make a connection. When receiving a request from a client, the server authenticates the client's privileges and the connection is established if the client details are authorised. The main functions involved in the server cluster 1 are as follows:

. an e-mail parser that parses and marshals e-mail messages; . a message handler 2 that receives e-mail messages and maps them into the RDR framework for classification; . a classifier 3 (classification means) that processes "cases" corresponding to e-mail messages and classifies them using the RDR rule-base; and . a learner 4 (means for implementing machine learning techniques) that implements the (e.g., Bayesian) classification algorithm for providing suggestions to the user.

The client cluster 1C» interacts with the server cluster 1 through a coordinator 5, which acts as a gateway to the classifier 3 and the message handler 2, and also invokes the classifier 3 and the learner 4 as required. The classifier 2 uses an RDR classification engine $ to construct rule and classify messages. Rules are stored in a rule-base, one rule-base for each user. All the RDR rule-bases are stored at one central location on the server cluster 1.

Using the e-mail parser, the e-mail management system and method of the present invention is able to handle the three different e-mail and folder formats listed in Table 1. The format is also used when saving the contents of e-mail folders. Messages in a folder are saved into a file whose name corresponds to the folder, using the format chosen by the user (this choice need only be made once, i.e., when starting the e-mail assistant for the first time). Folders in the e- mail management assistant correspond to files in the e-mail repository. Subfolders are stored as files in a sub-directory of the directory corresponding to the parent folder. The creation of sub-directories also depends on the user's chosen e-mail format, as shown in Table 1. Table 1 : e-mail formats

The main concepts involved in the e-mail management system and method of the present invention and how the software components model these concepts is described in the following Specifically, how the generic RDR framework is applied in the e-mail management system and method and the RDR classification engine and process of building the rule-base is discussed in more detail. Firstly, the RDR framework is described. The core of the e-mail management system is the RDR classification engine 6. The classification engine @ is built-up using RDR software (e.g., RippleDown™ software supplied by Pacific Knowledge Systems Pfy. Ltd.). This software provides a generic framework for implementing RDR rule-bases, which include generic functions for classifying cases and for managing multiple rule-bases. The software provides links to an e.ctemal database for storing the rule-bases of multiple users. The clas diagram used in the design of the e-mail management assistant of the present invention is shown in Figure 7.

The main object that the e-mail management assistant handles is the e- mail message. In order to have messages classified through the RDR classification engine β, messages are converted into objects of type ECase. Each ECase is a collection of attribute-value pairs. The attributes are the message header names, and each value is a sequence of numeric values, string values, date values or lists of values. If two header objects have the same name, the values corresponding to those headers will be added into the sequence of values for the corresponding attribute. There are two types of conclusion (EConclusion) in the rules of the present invention:

. "intermediate conclusions", which are conclusions that are used as another attribute value in the representation of the case. This type of conclusion is used to define a priority for a message; and . "text conclusions", which are conclusions that form the classification of the case, such as for example, the name of the virtual display folder for sorting the message, the priority of the message defined through an intermediate conclusion, and the action and possibly the template corresponding to a reply action.

By defining priorities using the intermediate conclusions, it is possible to allow more complex rules to be included in the e-mail management system and method of the present invention. Further, rules whose conditions refer to the priority, e.g., if the priority is high, notify the user immediately. The RDR classification and rule building performed requires the traversing of a rule-tree. With intermediate conclusions, a two-pass strategy is used to determine the final classification (set of conclusions) of a case. The first pass computes all intermediate conclusions and augments the case representation accordingly, then the second pass generates the final classification. More precisely, classification of a case is done as follows: o the case classification is set to empty; _ values associated with intermediate conclusion are removed from the case representation; . the first pass over the rule-tree establishes those rules that add or modify intermediate conclusions applying to the case, and the case representation is augmented to include any intermediate conclusions; and * the second pass over the rule-tree using the augmented case representation establishes those rules that add or remove text conclusions applying to the case. The final classification is the set of such text conclusions.

In the e-mail management assistant, the RDR classification engine 6 is located on the server side 1 (see Figure 6). E-mail messages are sent to the classification engine 6 and returned with the classified result to the client (proxy) 10 for generating the display to the user through the user interface 11. As also described above, an RDR system enables users to correct the rule-base through the following three types of action:

. add a new conclusion, by adding a new rule to the top level of the rule-tree with the new conclusion; . modify an existing conclusion, by adding a new rule as a child of the node in the rule-tree containing the rule that gave the incorrect conclusion; and . stop an existing conclusion, by adding a new rule (a stopping rule) as a child of the node in the rule-tree containing the rule that gave the undesired conclusion.

Figure 8 shows how the e-mail management assistant supports the user in the process of building-up a rule. The user always defines a rule in the context of a particular example, and needs to select conditions and the conclusion for the rule. As the user selects conditions, the e-mail management system and method of the present invention computes the conflicting rules from the RDR rule-base and is able on request to present the cornerstone cases for those rules to the user, until either the user accepts the new rule as applying to all conflicting cases, or else defines a new set of conditions for the rule.

Next, an evaluation carried out on the e-mail management system and method according to the present invention is described. RDR systems cannot be evaluated in the same way as machine learning algorithms, partly because of the incremental nature of the approach. There is no clear division between "training set" and "test set", and there is little point in artificially creating such a division. In any case, such experiments typically show only general statistics concerning the performance of a machine learning algorithm, and do not address the question of the usability of a system.

One major question concerning the usefulness of RDR systems in a given application is the degree to which the user has to continually add rules to cover misclassifications or non-classifications. It is expected that in dynamic domains, users will have to keep adding rules to cover examples not seen previously, and e-mail classification can to a certain extent be considered dynamic in that the type of messages a user receives changes with time. The main question is whether the burden of creating new rules outweighs the usefulness of the classification system. This is a question that can only be answered by the users. Thus, in order to evaluate the specific RDR system implemented in accordance with the present invention, a two week trial was conducted with eight technical users who had been using the e-mail management assistant of the present invention for a period of two months prior to the trial. The aim of the trial was to evaluate the competence, trustworthiness and usability of the e-mail management assistant.

More particularly, the following aspects of the system and method of the present invention were evaluated. • The usefulness of virtual folders for displaying messages in the Inbox.

• The accuracy of the classification.

• The ease of the rule building process.

• The degree of use made of different types of conclusion (virtual folders, priorities and actions). • The usefulness and accuracy of the suggestions provided by the Bayesian algorithm.

It was expected that much of the users' evaluations would be influenced by their previous experience with e-mail clients, where the two clients most commonly used by the evaluators were the widely available clients of Netscape Messenger and Unix mail (i.e., Pine, Elm or Mutt). Since some of the evaluators had also been using the filtering systems of other e-mail clients, the e-rnail management assistant of the present invention was also compared with their existing e-mail system.

Table 2 shows the characteristics of the rule-bases and e-mail processed by the eight user in the two wee!: trial. In Table 2, the second column i the number of rules in the user's rule-base, where because the present invention automatically determines where to add exception rules, the users may not have had to define that many rules (in fact, most users had no idea how many rules they had defined). Further, the third column shows how many messages were received in the two week period, which could be a very high number, as was the case for user 6 who received 1766 messages in the evaluation period (most of which were so- called "junk" mail). The fourth column shows how many messages were classified according to the rule-base as it was constructed during the period and the fifth column shows how many messages were misclassified (see below for the definition of misclassification).

Finally, the sixth column shows the proportion of classified messages which were correctly classified, indicating the accuracy, and the seventh column shows the proportion of all messages received that were classified either correctly or incorrectly, indicating the coverage.

As can be seen from Table 2, due to the use of the e-mail specific RDR rule-base of the present invention, very high levels of accuracy in classifying messages were achieved, namely between about 95% and about 99%.

Table 2: evaluation trial results

With respect to rriisclassification, since there is no direct feedback from the user concerning misclassifications, the accuracy is determined as follows. The e- mail management assistant records the user's action when viewing a message. If the message i handled semi-automatically according to the classification result (i.e., the user accepts the classification via a toolbar displayed in the user interface window), or if the message is handled manually but in accordance with the classification, the classification is counted as correct. On the other hand, if the message is handled differently from the classification, e.g., a message from one virtual folder is archived into a different folder, the classification is counted as incorrect. Since most users had virtual folders that corresponded to their folders for archiving, they were able to define rules that resulted in high accuracy.

The coverage attained during the trial period was also high, ranging from about 70% to about 96%. This result was surprising due of the wide variety of e- mail received which makes it difficult to define rules to cover this variety and because much of the e-mail is simply read once and then deleted (where rules are not defined for this type of message). It is possible that because the trial was over a short period, the e-mail received by users during that period did not vary much. In any event, the combination of accuracy and coverage obtained by the e-mail management system and method of the present invention was better than those for the prior art techniques previously described. Further, all of the trial users stated that the rule building user interface of the present invention was easy to use and that virtual folders provided a useful interface.

As those skilled ih the art can appreciate from the foregoing description, the e-mail management assistant of the present invention may readily be implemented within a practical e-mail application that can be used as an alternative to existing clients. That is, the e-mail management assistant of the present invention is able to interwork with all of the basic functions of an e-mail client, such as composing, sending, replying, forwarding and archiving messages, plus managing sub-folders and attachments in messages. To facilitate switching between e-mail clients, the e-mail management assistant has been adapted to be compatible with popular e-mail clients, such a i ietscape Messenger and Unix mail (pine, elm, mutt).

In the above description of the embodiments of the present invention, whilst rules can only be refined or added in the RDR system used, those skilled in the art would appreciate that it i further possible within the scope of the present invention to enable the deletion of conditions from a rule so as to generalise its application, for example, by defining a new rule, thereby preventing the user from being confronted with cornerstone cases for old rules (which persist in the rule- base if not deleted) which do not need to be examined. Further, rules could be defined with "negative" conditions (e.g., the message does not contain "CRC"), especially in exception rules.

Any discgssion of documents, devices, acts or knowledge in this specification is included to explain the context of the invention. It should not be taken as an admission that any of the material formed part of the prior art base or the common general knowledge in the relevant art on or before the priority date.

Claims

CLAIMS:

1. An e-mail management method for managing electronic mail messages, the method including the steps of: classifying e-mail messages using ripple down rules stored in a rule base, wherein each rule is defined by at least one or more rule conditions and a rule conclusion; and if a particular e-mail message is not appropriately classified according to any existing rule in the rule base, building a new rule using the steps of: selecting rule conditions based upon characterising features of the particular e-mail message; specifying a rule conclusion to be assigned to the particular email message and to each subsequent message that is classified according to the new rule; and adding the rule defined by said rule conditions and rule conclusion to the rule base.

2. The method of claim 1 , wherein the step of specifying a rule conclusion include specifying one or more of a virtual folder in which the message is to be stored, a priority of the message, and/or an action to be taken in relation to the message.

3. The method of claim 1 or claim 2 including using multiple ripple down rules, wherein a message is classified according to multiple rules and assigned one or multiple corresponding conclusions.

4. The method of any one of the preceding claims, wherein building a new rule further includes the step of using a machine learning algorithm to formulate a suggestion, and wherein the suggestion includes one or more rule conditions and/or a rule conclusion.

5. The method of claim 4 wherein the step of selecting rule conditions includes accepting or rejecting rule conditions in the suggestion, and the step of specifying a rule conclusion includes accepting or rejecting a rule conclusion in the suggestion.

6. The method of claim 4 or claim 5, wherein the machine learning algorithm includes a naϊ^'ve Bayes classification method adapted to suggest a rule conclusion including a virtual folder in which to store the message selected from a set of existing virtual folders and optionally to further suggest rule conditions including a set of keywords selected from the message that maximise the conditional probability that the message is correctly allocated to the suggested virtual folder.

7. An e-mail management system for managing electronic mail messages including: a rule base containing initially zero or more ripple down rules, wherein each rule is defined by at least one or more rule conditions and a rule conclusion; a classifier for applying the ripple down rules iior d in the. rule base for the classification of e-mail messages; and a rule building interface for defining and adding a new rule to the rule base by selecting rule conditions based upon characterising features of a particular e- mail message, and for specifying a rule conclusion to be assigned to each email message classified according to the new rule by the classifier, wherein the rule base is built up incrementally by use of the rule building interface.

8. The system of claim 7, wherein the characterising features used in selecting rule conditions include one or more keywords and/or phrases appearing in the headers, body and/or attachments of a particular e-mail message.

9. The system of claim 7 or claim 8, further including a computer- implemented user interface for displaying said particular email message to a user, and for enabling the user to select and view the rule conditions and specify the corresponding conclusion.

10. The system. of claim 9, wherein the rule building interface enables storing of a copy of the particular e-mail message used to define a new rule, and wherein upon the subsequent definition of a further new rule, the rule building interface identifies prior messages used to define existing rules that also satisfy the rule conditions of the new rule, and the user interface displays said prior messages to the user and enables the user either to add further rule conditions to distinguish the present message from said prior messages, or alternatively to accept the new rule with the consequence that the new rule conclusion will also apply to said prior messages.

11. The system of any one of claims 7 to 10, wherein the rule base is logically structured as a hierarchy, such as a tree, of rules and wherein: a message that satisfie the rule conditions of a higher level rule will be classified by the classifier according to the higher level rule and assigned the higher level rule condition, unless the message also satisfies the rule conditions of an associated rule at an adjacent lower level, in which case the message is classified by the classifier according to a lower level rule,

^• such that rules at lower levels of the hierarchy define exception to rules at higher levels.

12. The system of claim 11 , wherein a new rule that defines an exception to an existing rule is added to the rule base by the rule building interface at a level of the hierarchy immediately below the rule to which it is an exception, whereas a new rule that serves to classify a previously unclassified message or which defines an additional conclusion for a previously classified message is added to the rule base by the rule building interface at the first level of the hierarchy.

13. The system of either one of claims 11 or 12, when dependent upon claim 9, wherein the user interface is arranged to enable the user to specify that the operation of a rule is to be stopped, and the classifier subsequently does not apply stopped rules.

14. A software product for use in a computer with which there is associated a rule base containing initially zero or more ripple down rules, wherein each rule is defined by at least one or more rule conditions and a rule conclusion, the software product being adapted to effect provision of an e-mail management service for use by a client e-mail application for managing electronic mail messages, the software product including: computer instruction code to instruct said computer to provide a classifier for applying the ripple down rules stored in the rule base for the classification of e- mail messages; and computer instruction code to instruct said computer to provide a rule building interface for defining a new rule including rule conditions based upon characterising features of a particular e-mail message and a rule conclusion to be assigned to each email message classified according to the new rule by the classifier.

15. The software product of claim 14, wherein said client e-mail application is integrated with said e-mail management service.

16. The software product of claim 14, further including computer instruction code to instruct said computer to communicate with said client e-mail application so as to provide said e-mail management service to the client e-mail application.

17. The software product of claim 16, further including computer instructi on code to instruct said computer to communicate with a client e-mail applicati on executing on a second computer over a data communications network to wh ch both computers are connected.

18. The software product of claim 17, wherein said communication between the client application and the email management service is implemented using remote method invocation.