WO1996018260A1

WO1996018260A1 - Computer apparatus with dialogue-based input system

Info

Publication number: WO1996018260A1
Application number: PCT/GB1995/002887
Authority: WO
Inventors: Kenneth Brownsey; Mary Zajicek
Original assignee: Oxford Brookes University
Priority date: 1994-12-09
Filing date: 1995-12-08
Publication date: 1996-06-13
Also published as: AU4182596A; GB9424887D0

Abstract

An automatic telephone answering system uses fuzzy set combinations to generate a command signal which causes switching of a caller to a telephone receiver associated with a person most likely to give the information required by the caller or to generate automatically an audio message signal containing information useful to the caller. The system produces, as part of a telephone dialogue with the caller, linguistic outputs which are dynamically variable, each output being assembled according to real-time processing of dialogue history data based on a plurality of the previous caller responses in the dialogue. The system progressively generates a user interest structure representing a model of the caller's goals. The use of fuzzy logic operations results in a system which is capable of connecting the caller to a suitable information source more quickly and with a greater possbility of avoiding human intervention than with prior automated answering systems.

Description

COMPUTER APPARATUS WITH DIALOGUE-BASED TNPI IT SYSTEM

This invention relates to computer apparatus having a dialogue-based input system, and in particular to a computerised telephone answering system.

The provision of human-computer interaction over the telephone, or computer-aided telephony, is an expanding industry. Organisations are replacing the human telephone operator with a computer in many routine situations. The introduction of computer-aided telephony cuts costs and provides a twenty-four hour service which might not otherwise be available. However, the resulting human-computer dialogue can be unsatisfactory and can contribute to failure of the caller to reach their goal. User acceptance of this form of interaction is a significant problem, and the users' correct anticipation of what response is required when engaging in dialogue with a computer is difficult to manipulate.

The linguistic inputs and outputs of known telephone answering systems tend, on the whole, to take the form of "goal seeking" dialogues. Traditionally, such a system produces linguistic outputs forcing the caller to step through a hierarchical series of menu lists of options. Each menu list is generated as an audio output to which the caller is required to respond by selecting the most appropriate option until they reach the end of the dialogue whereupon the system either outputs a pre-recorded audio message giving a particular piece of information, a message requesting the caller to leave a message for a particular person or office, or alternatively the system automatically switches the caller through to a human operator if this is appropriate.

Accordingly, currently marketed systems are capable of handling only a small proportion of incoming calls to organisations, and rely on the caller having a clear goal in mind and a good understanding of the structure of the information contained in an answering system.

There is a need for a computerised interface which is more user-oriented and more efficient in controlling call or enquiry routing. According to a first aspect of this invention, there is provided computer apparatus having a dialogue-based input system for generating a command signal which depends on user responses to a plurality of linguistic outputs provided to the user by the computer apparatus, wherein the computer apparatus comprises means for storing a plurality of words and/or phrases, a user response detector, means for generating at least some of the linguistic outputs dynamically from the stored words and/or phrases as a function of user responses which are detected by the detector and are in response to earlier linguistic outputs provided to the user, wherein the generating means is operable, in generating each of a plurality of the dynamically generated linguistic outputs, to process an electrical representation of the user responses to a plurality of the respective earlier linguistic outputs in the dialogue in order to determine the content of the linguistic output.

The apparatus preferably includes an audio output device connectable to the linguistic output generating means for providing the linguistic outputs to the user as audio signals, and speech recognition means for detecting spoken user responses.

In the case of the apparatus being in the form of a telephone answering system, it may include a call switching circuit coupled to the command signal generating means and arranged to route a telephone call to call receiving means selected in response to the command signal and according to the said dialogue.

Alternatively or in addition, the apparatus may include a message generator coupled to the command signal generating means for providing the user with an information message which is determined according to the dialogue between the user and the apparatus.

In its preferred form, the apparatus includes means for storing representations of objects as hereinafter defined from a domain of interest, together with their degree of membership of predetermined classes in the domain of interest, the stored words and/or phrases being related to the classes and objects. The membership relationship between classes and objects may be seen as the relationship between fuzzy sets and their members. Means may also be provided for storing a response history representing user responses, the means J for generating linguistic outputs including means for dynamically developing a set of the objects as being of interest to the user according to the stored response history, each object having its own degree of interest to the user. This is referred to hereinafter as the user or caller interest structure. Selection means may be provided for allowing different subsets of the user interest structure to be selected according to detected user responses, and then offered to the user in the form of a further linguistic output for further consideration. In this preferred form, there are also provided means for selecting, according to outputs of the likelihood determining means, certain of the words and/or phrases and combining them to form a new linguistic output so as to determine with greater accuracy the object of interest of the user.

The linguistic output generating means may be arranged to select initially one class, to construct a question from the stored words and/or phrases, to provide a corresponding linguistic output to the user, to receive the detected user response and to construct an initial set of objects of interest, which is the user interest structure, using combination operations based on the fuzzy sets. The output generating means may further be arranged to select a class repeatedly, which class in the appropriate combination with the user interest structure provides a new and more well-defined user interest structure, using functions such as fuzzy set union and intersection, to construct a question from the stored words and/or phrases, taking the dialogue history into account, to provide a corresponding linguistic output to the user, to receive the detected user response and to construct a new user interest structure as a function of the old user interest structure, the selected class and the user response.

Preferably, the means for storing classes and objects include means for storing attributes relating to the said objects, the stored words and/or phrases including attribute words and/or phrases describing the attributes, and wherein the linguistic output means are arranged such that when the object of interest has been determined, linguistic outputs are generated containing the said attribute words and/or phrases to provide information to the user in response to the command signals. In the case of the telephone answeπng system, the selected call receiving means are selected as a result of their association with the object of interest

In this specification, "objects" means objects in the sense of things to which action is directed, abstract or mateπal things or persons of interest, or information of interest

The invention also includes a method of operating a computer to generate a command signal in response to a dialogue-based input sequence, as defined in the claims

In at least one of its preferred forms, the invention provides for the construction of dialogues for a computer-aided telephone answeπng system handling calls to a large organisation In particular, the apparatus is directed to handling ill-defined calls from callers unsure of the end destinations of their calls The dialogues are negotiative in nature and designed to question the callers to ascertain their goals in making a call to the organisation

Preferably, the words and/or phrases include grouping or basket words and/or phrases for use in dynamically building question sentences

It will be appreciated, then, that in the preferred embodiment of the invention, the caller hears dynamically-built questions These are used in preference to pre-recorded messages Indeed there may be no pre-recorded messages Given that the elements of the associative network are linked by weightings, the processing of the apparatus makes use of fuzzy set operations

The dialogue proceeds on the basis of hypotheses which the apparatus seeks to confirm or refute by appropπately built questions The apparatus is also capable, unlike pπor art systems, of changing its hypothesis so that if a caller's responses are falsely interpreted as indicating one class of objects of interest, the system can recover and follow an alternative hypothesis to reach a more appropnate class Put a different way the apparatus is operable to assign weightings to the classes of objects, the weightings being indicative of user interest, and to select a class which contains a sufficient number of sufficiently weighted members to be of interest to the user. However, it should be noted that during the input sequence the weightings can change to the extent that although, initially, a first class may contain a relatively high number of relatively highly weighted members, subsequently a different, second class can be selected, also having a relatively high number of relatively highly weighted members, according to user responses.

According to a further aspect of the invention there is provided computer apparatus having a dialogue-based input system for generating a command signal which depends on user responses to a plurality of linguistic outputs provided to the user by the computer apparatus, wherein the apparatus comprises:- a dialogue generator for assembling linguistic outputs and for providing them to the user; a user response detector for detecting responses to the linguistic outputs; means for storing at least one selected class of objects as hereinbefore defined; means for storing a variable user interest structure dependent on earlier linguistic outputs provided to the user and of detected user responses; and an inference system coupled to the selected object class storing means and the user interest structure storing means, and operable repeatedly to select different object classes for storage and repeatedly to modify the user interest structure in response to the detected user responses; the dialogue generator including vocabulary storing means for storing a plurality of words and/or phrases and a message assembler for generating the linguistic outputs in response to the selected class stored in the selected object class storing means; the inference system further comprising evaluation means for evaluating the user interest structure according to a predetermined measure and for generating the command signal when the evaluation measure is of a predetermined value.

The apparatus may constitute or form part of a telephone answering system, with the user response detector comprising speech recognition means configured to recognise a plurality of predetermined spoken utterances such as "Yes", "No", and "don't know" Preferably, the apparatus includes means for storing a set of objects, a set of object classes, and a plurality of object/class relationships in the form of a fuzzy set or a plurality of fuzzy sets Normally, this data is constant duπng a dialogue

The user interest structure storage means is preferably arranged to store the user interest structure as at least one dynamically variable fuzzy set relating different classes of objects as a function of the user responses Thus, the contents of the user interest structure fuzzy set or sets change as the input sequence or dialogue progresses, the structure being updated together with the selected object class in response to at least some of the detected user responses

The dialogue generator is preferably concerned with the semantics of the linguistic outputs, and assembles messages on the basis not only of the existing content of the selected object class storing means, but also on the basis of stored user response information using a fixed vocabulary of words and/or phrases

The dialogue generator may be responsive to the command signal to generate an output information message for the user

The inference system typically includes combining means operable to perform combinauon operations such as fuzzy logic unions and intersections in order to update the user interest structure

The invention also includes a further method aspect as defined in claim 28

The invention will now be descπbed by way of example with reference to the drawings in which -

Figure 1 is a block diagram of a telephone answeπng system in accordance with the invention, Figures 2A, 2B, 2C, and 2D are a goal and question matrix set for determining user goal variability;

Figure 3 is a diagram illustrating a fragment of an associative network;

Figure 4 is a fuzzy set representation of relationships between classes and objects; and

Figure 5 is a block diagram of a portion of the telephone answering system of Figure 1.

Referring to Figure 1 , a computerised telephone answering system in accordance with the invention has a call switching circuit 10 coupled to a processor 12 and storage means 14. The switching circuit has an input portion 10A with several (here four) telephone line inputs 16 and a port 18 coupled to the processor 12. The switching circuit includes an output switching portion 10B having an input port 20 coupled to an output port 22 of the processor 12. The output switching portion has a large number of outputs 24 coupled to a corresponding number of call receivers (not shown).

In use, calls received on lines 16 are initially routed to the processor 12 by the input switching portion 10A, the processor 12 including speech recognition means for detecting and decoding user responses to dynamically built sentences generated by the processor 12 and fed back to the user in a manner which will be described in more detail below.

Once the object of interest of the user has been determined, the processor 12 may cause the switching circuit to connect the caller through to one of the output lines 24, the selection of line being performed according to the determined object of interest. Alternatively, a message can be generated giving information to the caller.

The apparatus of Figure 1 forms the basis of a novel answering system involving dynamically creating dialogue responses which depend on a caller's utterances. In particular, the system aims to assess the object of interest of a caller who is allowed to answer dialogue questions only using a limited set of responses such as "Yes", "No", and "don't know" The apparatus can operate in this way simultaneously on several calls on the respective line inputs 16

The question sentences are dynamically built from a number of possible words and/or phrases in fuzzy sets based on a possible endpoint. A caller's goal can fall into several of these sets and move between them.

As an example of an ill-defined call to a large organisation, the case of a call to a large hospital is considered, the caller wishing to find out the time of an event such as a Fun Run in support of a particular charity, with the caller being unable to name the person he or she wishes to speak to.

The organisation is represented by an associative network of organisational data which will be explained below. For the purpose of this part of the description, it is sufficient to know that the network consists of a series of interconnected nodes. Each node contains information about an individual entity or class or entities within the organisation. Facts about the organisation are decomposed into a set of nodes and relationships between those nodes. At any one time during an input sequence or dialogue, the system is focused on several nodes. In an inverse of the decomposition of facts, the system takes these several nodes to construct a question sentence. Thus, if one node represents "classes", as in "evening classes", and another one represents "organiser" a reasonable question might be "Do you wish to speak to an organiser of classes⁹" If there is a node representing "organisation" and one representing "finances", it may not be clear whether the appropriate question is "do you want information about organisation of finances?" or "do you want information about finance of organisations?" The dialogue history thus far contributes to a weighting of nodes which helps both to select and structure the elements for a question sentence.

Question sentences are constructed to produce responses which indicate what the user's goal is The approach is to construct questions that incorporate basket terms for groups of goals. For example, the caller calling a hospital to determine the time of the "Fun Run" for their favouπte chanty might be asked "is it a medical matter⁷" The term "medical" is a "basket" word which includes some references to medical staff and many other things

Callers will have different perceptions of whether their call is "medical" or not Some may think that "medical" implies illness and only "ill" people, and others might think it applies to everything in hospitals Both are equally πght, but are dependent on the caller's perception of word usage and their perception of the context of the word usage

Underlying the interaction between user and system is the system's model of the organisation Essentially this is a network with endpoints The procedural role of the system is to build a model of the user goals and map it onto an endpoint Seen from the system's model, the same stated user goal - e g speaking to the person organising maternity classes - may be vaπable between users Two users may have two distinct specific goals in mind when both state "I wish to speak to the person organising maternity classes" One may mean the person who is in charge of the content of the classes, the class plan and so The other may mean the administrator, responsible for the paperwork, handling of finances etc Of course it may be that both roles meet in the same person but if not, there may be a difference between the actual, as opposed to stated, goals Hence the desired endpoints of the system's models may also be different, although the path through the network to them may be common for the greater pan So this variability between actual goals, associated with the same stated goal, is taken into account, in the specification and design of the apparatus

There is also vaπability in the wider contexts that the user sees the goals set in One user may see the maternity class question as being concerned with a medical matter, while the other does not The first user may not feel strongly either way on whether it is an administrative issue, whereas the second feels that it is, perhaps because he or she is cunently engaged in organising their timetable for the next few weeks and wants dates and times Referring to Figures 2A to 2D, a goal and question matπx set is used to obtain a measure of the possible vaπabihty, as mentioned above, of user goals First of all, a set of questions, involving basket terms as mentioned above, and a set of possible stated goals is taken From the system point of view, the stated user goals will involve one or more of top level sets of objects within the system The user is modelled as seeing that his or her goals also involve membership of these top level sets, via the basket terms Of course, it is not anticipated that any one user's perceived pattern of membership matches the membership pattern or that of any other user, necessaπly The response from a number of users can be used to get a measure of the membership of these top level sets

The following is a prototype expenment earned out on 12 student users, from a variety of disciplines Naturally, we do not present such a group as being representative of the possible caller population, rather as a sufficiently heterogeneous group to provide an empirical example of user goal vaπability We used the examples of calls to a large hospital Each subject panicipated in 6 tπals In each tπal the subject was given a goal - e g to find out who organises maternity classes - and then was asked to reply "Yes", "No" or "don't know" to six question, each involving a basket term - e g medical matter The order of goals was vaπed between subjects, and the order of questions varied within the goals for each subject

The goals were as follows -

A - to find out who organises maternity classes

B - to find out the date and time of the Fun Run for Children in Need C - to find out how to get to the hospital

D - to find out which ward a patient would be in

E to find out visiting hours

F - to contact the Kidney Disease Research Group

The questions asked were as follows - 1 - are you calling about a medical matter?

2 - are you calling about a personal matter?

3 - do you have an administrative query?

4 - is your call about financial matters? 5 - do you need access information?

6 - do you want transport information?

The responses are tabulated as the matrices of Figures 2A to 2D. Figure 2A tabulates "Yes" responses, Figure 2B tabulates "No" responses, and Figure 2C tabulates "don't know" responses. Note that the figures for each cell position across the three matrices of Figures 2A to 2C add to 12, any response other than "Yes" or "No" being taken as "don't know" The fourth matrix, that of Figure 2D shows how the model of the organisational data would score using a straight yes/no format and interpreting relevance through links in the network representation. The complete set of matrices can be used to assess responses to other questions, where the system uses the same relevance interpretation.

Associative networks, otherwise known as conceptual, propositional, or semantic networks, have a long history in artificial intelligence, as well as in logic and reasoning. Strictly speaking, an associative network is distinct from its graphical representation, which is how they are usually represented. However, since the only other representations tend to be in pseudo-code or actual program code, we present in Figure 3 a graphical representation which is most strongly suggestive of the concept of an associative network.

In modelling the organisation data, classes of objects have been identified. Each class of objects may be a member - in which case it is called an instance -, a subset or superset of another. Each instance has attributes, which may be a simple property of the object, or a relationship with another object. In a diagram of fragment of the model for the hospital, in Figure 3, the direction of the relationship is shown by the black disk at the end. So it can be seen that the class of doctors is concerned with medical matters. Reflexive relationships, such as is are shown with a disk at each end of the connector. Most relations have some sort of inverse Here it is seen that the Fun Run has an organiser, who is organiser o/the Fun Run

The network in Figure 3 shows how some simple facts are decomposed For example the fact that "Jane Jones is a doctor" is represented by the instance node G833, together with the ιs_a link to the doctor class node, and the has name link to a property node Nodes may be involved in several facts Jane Jones is organiser of a Fun Run on 20/11/94" centres on the three instance nodes G833, G942 and G27 The facts available may be more or less precise, and may overlap "Dr Jane Jones is organising a Fun Run" tell us both a little more and a little less than the previous fact

From the goal and question matπx set it is possible to attach initial weightings to the leaf classes, I e those classes which are not super classes of any other At any stage these classes are the source of the weighting used to build question sentences However, the questions will not necessaπly involve these classes, at least initially It is possible to combine the weightings, regarded as fuzzy measures, using, for example Dempster's rule of combination (Gordon, J & Shortliffe, E (1984) The Dempster-Shafer Theory of Evidence, Uncertain Reasoning (ed Shatter, G & Pearl, J ) 1990, Morgan-Kaufmann) This produces measures for the (recursive) supersets As well as using the subset relation, shown by ιs_a, weights are allowed to "flow" along other links, such as is concerned with, but weighting the flow to reflect some assessment of the importance

In this way, the caller's perception of the meaning of computer generated sentences is dealt with by employing empincal user-based data generated by a Goal Question Matπx Set Organisational data is incorporated into an associative network These two models are then integrated in order to build a questioning dialogue by relating the likelihood of defined classes being the object of interest to the classes which form their supersets, or by relating the likelihood of defined instances being the object of interest to classes of which they are members The top layer of classes defines the "basket" words or phrases The weightings of the subset classes or the instances as determined by the responses to the questions are combined using the relationships of the network to produce likelihood weightings for the subset classes or instances so as to build subsequent questions.

The associative network is generalised in one respect by abstracting out the is_a links in such as way that the is_a links for instances to classes receive weightings which are fuzzy set membership weightings, and the is a links between classes become implicit in the fuzzy set structure using the notion of fuzzy subsethood. Thus, the relationships between objects and classes of objects is expressed in terms of degrees of membership or weightings. Thus, an object which is unmistakably a member of a particular class can be allocated membership degree " 1 ", and one which is unmistakably not a member of that class has a membership degree "0" with respect to the class. In many cases, however, the membership degree is between "0" and "1 ". It follows that each class can be represented as a fuzzy set. Fuzzy sets are the basis of fuzzy logic. In fuzzy logic, a signal can adopt the state " 1 ", "0", or a plurality of intermediate states such as 0.2, 0.6, etc., unlike binary logic in which, generally, only logic states " 1 " and "0" are permitted. Combinations of fuzzy sets can be performed in different ways.

Fuzzy set theory and fuzzy logic are explained in Neural Networks and Fuzzy Systems by Bart, Kosko, Prentice-Hall International, 1992.

To illustrate the use of fuzzy sets in the telephone answering system described in this specification. Figure 4 contains a fragment of a fuzzy set representative of tourist activities in Oxford, England. This representation would be stored in the storage device 14 of the apparatus of Figure 1 if it is used as part of a telephone answering system for providing tourist information in Oxford.

Referring to Figure 4, objects of interest 30 are related to classes 32 of tourist activities. each class taking the form of a fuzzy set in that the objects 30 have different degrees of membership 34 or weightings in the different sets, as shown by the numerical values associated with the links between objects and classes. Note that in a number of cases, an object has several links, each to a different class In the general case, each object has links to all of the classes

The diagram of Figure 4 is, in some respects, a more generalised representation of the association between objects and classes than the associative network of Figure 3, but could equally be applied to the hospital telephone answeπng system descπbed above with reference to Figures 2A to 2D and Figure 3 by the above-descnbed integration of the weightings deπved from the Goal Question Matnx Set into the associative network

It will be appreciated that, in practice, many more objects 30 would be included in the stored data represented by Figure 4, so that there would be many more links 34 representing degrees of membership

Referring to Figure 5, the computer apparatus represented as processor 12 and storage means 14 in Figure 1 can be represented in more detail as an inference system 12IS and a dialogue generator 12DG both controlled by a controller 12C Telephone calls arπving on lines 16 (Figure 1) are routed to a caller response detector 12CD which, in practice, constitutes speech recognition means for detecting and decoding a limited number of caller responses such as "Yes", "No", and "don't know" Typically, the number of permitted responses is 10 or less, preferably 5 or less, to obtain reliable operation with a variety of callers having different voice patterns, accents, etc , and in view of the limited reliability of cunent speech recognition systems in dealing with unknown callers, as is required of the present system

The storage means 14 (Figure 1) includes, as shown in Figure 5, means for stoπng the data shown in Figure 4, that is an objects store 14OS for stoπng objects 30, and object classes store 14CS for stoπng classes 32, and an object/class relationship store MRS for stoπng the membership degrees 34 The storage means also includes a selected class store 14SC for classes selected during the dialogue, and a caller mterest structure (CIS) store 14CI for stoπng a caller interest structure m the form of a fuzzv set or sets representing a summary of the dialogue history As part of the inference system 12IS is a combiner 12ISC which processes parameters of the caller interest structure CIS(QC) at any given stage QC (question count) within the dialogue in combination with the cunently selected class of interest C(QC) and the caller response code CR(QC) resulting from the most recent utterance from the caller, together with the membership degree data 34 from the object class relationship store 14RS to produce an updated CIS, refened to here as CISNext.

Associated with the combiner 12ISC in the inference system IS is an object class locator 12ISCL, the function of which is to select a new class C(QC + 1) potentially of interest to the caller in view of a combination of the cunent CIS and the set of object classes obtained from class store 14CS.

Both combiner 12ISC and class locator 12ISCL operate by performing union and intersection functions suitable for fuzzy logic processing, as will be described in more detail below.

Combiner 12ISC also periodically tests the CIS using fuzzy set measures such as fuzzy entropy which, when it attains a predetermined threshold, results in generation of a command signal COMMAND which is fed to the controller 12C for connecting the caller to. for example, a selected telephone receiver (not shown in Figure 5) or to actuate generation of an information message in the dialogue generator 12DG via command actuate line CA, this being transmitted to the caller as an audio signal by an audio output device 12OD comprising a speech synthesiser driven by the message generator 12DGM. Audio output device 12OD has an output (not shown) connectible to the relevant telephone line on which the incoming call is present.

The primary function of the dialogue generator 12DG is to generate messages in the form of questions aimed at determining the caller's interest or otherwise in the selected class of objects C(QC) stored in selected class store 14SC, making use of the dialogue history so far as stored or summarised in the caller interest structure CIS. A message is assembled in message assembler 12DGM using words and phrases stored as a vocabulary in phraseology store 12DGP. and the output device 12OD synthesises it as an audio signal for transmission to the caller

By way of further explanation, the computer apparatus has two main components, the inference system 12IS and the dialogue generator 12DG

The two main roles of the inference system 12IS are (a) to determine which class of objects C(QC) to ask about next, pπor to the dialogue generator generating question number QC, and (b) to work out the next generation of the caller interest structure CIS(QC), given the reply to question number QC

The caller interest structure CIS(QC), is a summary, for the purposes of the inference system, of the dialogue history so far - I e after QC questions have been asked Thus it does not contain such items as the forms of question, etc It summarises in some form, the sequence of classes objects enquired about, and the responses to those enquiπes To use an analogy, a summary of a set of figures relating to, say, student course marks, could vary from a simple average, through a sequence of histograms, based on different class sizes, to the oπginal set of raw data A very simple form of the caller interest structure is a fuzzv set, formed from appropπate unions and intersections of the classes of objects enquired about More information about the dialogue history is retained if the CIS is a summary set of (weighted) fuzzy sets, which is a more robust form for a general purpose system Note that CIS(QC) is a function of the dialogue history after QC questions, and could, in pπnciple, be computed from that dialogue history Accordingly, in generating each new version of the CIS, the system has the ability to evaluate or take in account in real-time (l e in each question message generating operation) a plurality of previous user responses, generally in the form of the summary or condensed version that is the CIS Consequently, the inference system is able to update the CIS in such a way that questions can be constructed dynamically on the basis of the dialogue history so that the CIS is progressively refined, I e the CIS is progressively built up as a simple model of the goals of the caller It will be appreciated from the above that the linguistic outputs or questions put to the user do not generally or necessanly follow one of a predetermined seπes of paths in a tree as in the prior art hierarchical systems. Once one question has been answered in a certain way, the system is not necessarily constrained thereafter to a predetermined more limited set of questions as a result. The dialogue progresses as an elaboration of a search path using a dynamically developed sequence of linguistic outputs, neither the search path nor the linguistic outputs beings explicitly built into the system at the start of the dialogue. Typically, the complete set of linguistic outputs does not appear anywhere in the controlling software code or files.

The main roles of the dialogue generator are:- (i) given a class of objects C(QC), the dialogue history so far, semantic links between the classes of objects, as well as the (implicit) structural information contained in the fuzzy set representation, to generate a question message which is aimed at determining the caller's interest or otherwise in C(QC), which fits in with the rest of the dialogue, and which is designed to obtain as much information as possible through variation in (non- subject) content and style, and through reference to previously used classes of objects where appropriate; and

(ii) to detect and, subsequently, either to draw to a close or restart dialogues which are making no apparent progress.

A more detailed descπption of the operation of the system will now be set forth in pseudocode form. As described here the system is seen as set in a wider system offering alternative facilities, not explicitly described in this application. For this reason, the pseudocode contains references to "...exit with appropriate exit code..". Note that this means a complete exit and that this may result in a non-standard termination of a loop.

PSEUDOCODE DESCRIPTION OF OPERATION

1 The dialogue generator welcomes the caller, explaining that there is a choice between using the system, or the alternatives of being put in a queue, ringing off. etc.

2.1 IF the caller chooses any of the alternatives THEN

2.2 This system is exited with appropriate exit code. ENDIF

3.1 IF there are alternative dialogue systems THEN 3.2 The dialogue generator explains this

3.3 The dialogue generator asks the caller whether to run a sub -dialogue A to detail and offer the alternatives.' 3.4 The caller response with CAlt.

3.5.1 IF CAlt indicates that sub-dialogue A is required THEN

3.5.2 Sub -dialogue A is performed ENDIF

3.6.1 IF the result of sub-dialogue A impl ies that an alternative system is required THEN

3.6.2 This system is exited with appropriate exit code ENDIF ENDIF

4 The dialogue generator explains what the system will do for the caller.

5.1 IF there are options about the form of the final goal THEN 5.2 The dialogue generator explains what the default settings are

5.3 The dialogue generator asks the caller whether to run a sub-dialogue B to detail and offer the options²

5.4 The caller responds with COpt.

5.5.1 IF COpt indicates that sub-dialogue B is required THEN

5.5.2 Sub-dialogue B is performed, during the course of which the default values concerning the final goal form may be altered.

END IF ENDIF

6 The dialogue generator passes control to the inference system.

7 Using the goal form values, the inference system locates a plasible first class of objects C(0) for a question.

8 The initial caller interest structure CISCO) is constructed from C(0)

9 The question count QC is set to 0.

10 KeepAsking is set to TRUE

11.1 WHILE KeepAsking 11.2.1 IF an analysis of CIS(QC) reveals a sufficiently interesting subject SIS to report to the caller, where the measure of interest is based on goal form settings and measures such as fuzzy set entropy³ THEN 11.2.2 A sub-dialogue C is entered⁴

• providing the caller with options of levels of detail, about any, none or all of the members of SIS

• obtaining caller's responses to the options

• providing information at appropriate level of detail as requested

• allowing the caller to request further search 11.2.3.1 IF further search is not requested THEN 11.2.3.2 This system is exited with appropriate exit code

ENDIF ENDIF

11.3 The inference system passes control to the dialogue generator

11.4 NoDefiniteResponse is set to TRUE 11.5.1 WHILE NoDefiniteResponse

11.5.2.2 IF the dialogue generator considers that the dialogue i s "stuck" or "aimless"^s in any way THEN

A sub-dialogue D is performed, as a result of which C(QC) may be changed prior to return from D, or the system may be exited

ENDIF

11.5.3 The dialogue generator constructs a question, based on C(QC). and the previous dialogue history, and this question is put to the caller.

11.5.4 The caller responds with CR(QC).

11.5.5.1 IF CR(QC) = EXIT THEN The system provides an appropriate courtesy message

KeepAsking is set to FALSE ENDIF

11.5.6.1 IF CR(QC) = BACK THEN QC is set to QC -1

ENDIF

11.5.7.1 IF CR(QC) = D0NT_KN0W THEN

C(QC + 1) is set to C(QC) AND QC is set to QC + 1

ENDIF

11.5.8.1 IF CR(QC) - YES OR CR(QC) = NO THEN

11.5.8.2 The dialogue generator passes control to the inference system 11.5.8.3 Using the response, CIS(QC). C(QC) and the previous dialogue history the inference system computes a new caller interest group CISNext

11.5.8.4 QC is set to QC + 1

11.5.8.5 CIS(QC)is set to CISNext 11.5.8.6 The inference system locates a plausible next class of objects, together with a combination function for that class and CIS(QC). C(QC) for a query 11.5.8.7 NoDefiniteResponse is set to FALSE

ENDIF

ENDWHILE ENDWHILE END [ NOTES : -

1 For example, in some domain areas, the system may attempt to establish if the caller is experienced and wishes to work through an alternative system - e.g. a hierarchical menu structure For example in some domain areas, the caller may be able to select how many objects make up the goal set and the degree of homogeneity cr heterogeneity in the goal set

3 Fuzzy set entropy is a measure of the definiteness of the membeship degrees of a fuzzy. Thus, a set having membership degrees near "1" and "0", e.g. (0.90, 0.15, 0.80, 0.05] has a higher entropy than one having membership degrees on average nearer 0.5, e.g. (0.65, 0.80, 0.40, 0.351. In the present example, when the entropy value of the CIS exceeds a predetermined threshold, the command signal is generated and an appropriate corresponding operation is performed, such as switching the call cr generating an information message (represented here as sub-dialogue C) .

4 The subsystem providing sub-dialogue C is conceptionally distinct from the mam dialogue system in the dialogue generator. It provides a straigt orward information service with a number of options. For example, t may inform the caller that half a dozen items of potential interest have been found and ask if the caller wants a summary, or to step through the l st, opting for full information on selected items, or have all the infomration with the option of cutting short the information provision. It should be noted that the system for generating sub-dialogue C is a relatively unintelligent system, in which the caller can select options regarding information delivery, the items of interest having been selected by the relatively intelligent activities of the inference system and the main dialogue system m the dialogue generator.

5 The two scenarios that come to m nd are a run of DONT_KNOWs, and a long dialogue with no visible development of a sufficiently interesting subset of the caller interest set.

It will be seen that in steps 1 - 5.5.2 of the pseudocode the system accepts inputs and provides linguistic outputs which form a series of preliminary dialogue exchanges prior to fuzzy logic operations to determine the true interest of the caller. The first of these operations is locating a plausible first class of objects C(0) for a query (step 7), and an initial caller interest set CIS(0) is constructed from C(0). Once KeepAsking is set to TRUE (step 10), the inference system and dialogue generator operate within outer and inner loops until KeepAsking is set to FALSE (step 1 1.5.5.3). The outer loop begins at step 1 1.1, while the inner loop starts at 1 1.5.1 and is followed so long as NoDefiniteResponse is TRUE (step 1 1.4). Both loops end with step 11.5 ^'8.7. Accordingly, once the initial object class C(0) and caller interest structure CIS(O) have been computed, the outer loop is entered and the CIS is evaluated to see whether it has reached the fuzzy entropy threshold (step 11 2 1) If not, control is passed to the dialogue generator (step 11 3) and the inner loop is entered, which first checks whether the dialogue is not progressing (step 11 5 2 2), and then the system constructs a question based on C(QC) using the dialogue history and the vocabulary store to construct a user-friendly question aimed at finding out in the most efficient possible way whether the caller is interested in the class C(QC)

The caller's reponse CR(QC) is then checked to see whether it demands an exit (step 1 1 5 5 1 ) (in which case the system is exited), whether the caller has asked to backtrack ( step 1 1 5 6 1 ) (in which case the last-but-one question is repeated), or whether the response is "don't know" (step 1 1 5 7 1), (in which case the dialogue generator puts the question based on C(QC) in a different way) In the case of the last two response options, the NoDefiniteResponse flag remains TRUE and the system reverts to the beginning of the inner loop at step 1 1 5 1 If, however, a definite "Yes" or "No" response is received (step 1 1 5 8 1), the dialogue generator passes control to the inference system and two new fuzzy logic combinauon operations are performed, firstly, to generate a new caller interest structure CISNext and, secondly, to locate a plausible next class of objects C(QC)for a query At the same time, the question count QC is incremented to QC + 1 (step 1 1 5 8 4)

As explained above, the CIS may be a single fuzzy set or a super set of weighted fuzzy sets To illustrate the generauon of CISNext, the case of the CIS being a simple fuzzy set is considered This exemplary simple fuzzy set is over the same universal set as the fuzzy sets in the object class domain 34 (Figure 4), and it has four elements from the universal class set, which are, taking the Oxford tounst guide example, {"The Biggs Museum", "The Lamb and Flag", "The Krump Tea Rooms", "Higgs Academy"}

Suppose then that the CIS is {0 3, 0 1, 0 0, 0 8}, indicating that "The Biggs Museum" has membership degree 0 3, "The Lamb and Flag" has membership degree 0 6, "The Krump Tea Rooms" has membership degree 0 0, and "Higgs Academy" has membership degree

0 8

The inference system output indicates that the dialogue generator should generate a question aimed at a logical union with the class of Cultural Activities which is {0 9, 0 2, 0 1, 0 6} If the response is "Yes" then the CISNext is {0 93, 0 28, 0 1, 0 92}, using the union function x ^■*- y -xy If the response is "No", then the CIS is not changed

Note that the new CIS for "Yes" is "sharper" (l e has a higher fuzzy entropy) than the old CIS and , if the goal structure indicates the caller has about two items of interest, this is taken as indicating it is appropriate to ask the caller about the two highly weighted items

This is very simple example in several ways Firstly, the number of items involved is very small (4) , secondly, the operations of union and intersection can be generalised and moderated by other functions and, thirdly, the information retained in the CIS, as shown here, is below the optimal level for the reasons descπbed above

Selection of a new object class C(QC) can be performed as follows

Taking the Oxford tounst guide example again, suppose that we have the situation as descπbed above, where the CIS is {0 3, 0 1, 0 0, 0 8}, and that there is also the class of Food and Drink, with membership degrees (0 4, 1 0, 0 9, 0 0} Using fuzzy union and intersection, where the intersection function is xy

CIS union Cultural Activities is {0 93, 0 28, 0 1 , 0 92 }

CIS intersection Cultural Activities is {0 27, 0 02, 0 0, 0 48}

CIS union Food and Dπnk is {0 58, 1 0, 0 9, 0 8}

CIS intersection Cultural Activities is {0 12, 0 01, 0 0, 0 0}

The inference system will infer that the most informative action is a "Yes" answer to a question posing a union of the cunent CIS and Cultural Activities Note that this is a very simple example in that the number of classes involved is very small (2) and the operations of union and intersection can be generalised and moderated by other functions

These simple examples illustrate the pπnciple of the main operations earned out by the inference system In practice, the data used and the combination operations, although still based on union and intersection operations, are considerably more complex and have not been set out in this specificaiton for reasons of clanty

The above system may be summaπsed as an automatic telephone answeπng system uses fuzzy set combinations to generate a command signal which causes switching of a caller to a telephone receiver associated with a person most likely to give the information required by the caller or to generate automatically an audio message signal containing information useful to the caller The system produces, as part of a telephone dialogue with the caller, linguistic outputs which are dynamically vaπable, each output being assembled according to real-time processing of dialogue history data based on a plurality of the previous caller responses in the dialogue The system progressively generates a user interest structure representing a model of the caller's goals The use of fuzzy logic operations results in a system which is capable of connecting the caller to a suitable information source more quickly and with a greater possibility of avoiding human intervention than with pnor automated answering systems

Claims

Computer apparatus having a dialogue-based input system for generating a command signal which depends on user responses to a plurality of linguistic outputs provided to the user by the computer apparatus, wherein the computer apparatus compπses means for stoπng a plurality of words and/or phrases, a user response detector, means for generating at least some of the linguistic outputs dynamically from the stored words and/or phrases as a function of user responses which are detected by the deteαor and are in response to earlier linguistic outputs provided to the user, wherein the generating means is operable, in generating each of a plurality of the dynamically generated linguistic outputs, to process an electrical representation of the user responses to a plurality of the respective earlier linguistic outputs in the dialogue in order to determine the content of the linguistic output

Apparatus according to claim 1, further compπsing an audio output device connectible to the linguistic output generating means for providing the linguistic outputs to the user as audio signals, and speech recognition means for detecting spoken user responses

Apparatus according to claim 1 or claim 2, in the form of a telephone answenng system having a call switching circuit coupled to the command signal generating means and arranged to route a telephone call to call receiving means selected in response to the command signal and according to the dialogue

Apparatus according to claim 1 or claim 2, including a message generator coupled to the command signal generating means for providing the user with an information message

Apparatus according to any preceding claim, including means for stoπng classes of objects and objects as hereinbefore defined together with relationship information linking the said classes and objects in an associative network, the said stored words and/or phrases being related to the classes and objects, and means for storing a response history representing user responses, wherein the means for generating linguistic outputs includes means for determining the likelihood of each of a plurality of the said objects being of interest to the user according to the stored response history in combination with the associative network, and means for selecting, according to outputs of the likelihood determining means, certain of the said words and/or phrases and combining them to form a new linguistic output so as to detemiine with greater accuracy the object of interest of the user.

Apparatus according to claim 5, call or enquiry including means for calculating and storing a relevance weighting for each of a plurality of the said classes of objects and for each user response, wherein the weightings are altered periodically during the call or enquiry according to the response history as it develops.

Apparatus according to claim 5 or claim 6, wherein the said means for storing classes and objects includes means for storing attributes relating to the said objects, the stored words and/or phrases including attribute words and/or phrases describing the attributes, and wherein the linguistic output means are ananged such that when the object of interest has been determined, linguistic outputs are generated containing the said attribute words and/or phrases to provide information to the user in response to the command signal.

8. Apparatus according to claim 5 or claim 6, in the form of a telephone answering system having a call switching circuit coupled to the command signal generating means and ananged to route a telephone call to call receiving means selected in response to the command signal and according to the said dialogue, wherein the selected call receiving means is selected as a result of its association with the object of interest.

A method of operating a computer to generate a command signal in response to a dialogue-based input sequence, compnsing generating a plurality of linguistic outputs for a user, detecting user responses, which are in response to the linguistic outputs, and generating the command signal according to the dialogue constituted by the said outputs and responses, wherein each of at least some of the linguistic outputs is dynamically generated from a plurality of stored words and/or phrases as a function of the detected user responses to respectively earlier linguistic outputs provided to the user, each said dynamic generation including processing an electπcal representation of the earlier linguistic outputs

A method according to claim 9, wherein the linguistic outputs are generated as audio signals and the user responses are detected by speech recognition

A method according to claim 9 or claim 10, wherein the computer forms part of a telephone answeπng system having a call switching circuit, the command signal compnsing a control signal for the switching circuit to route a telephone call to call receiving means coupled to the answeπng system

A method according to any of claims 9 to 1 1, wherein the computer is used to store data representing an associative network of objects and classes of objects as hereinbefore defined and wherein the stored words and/or phrases relate to the said objects and classes, the process of dynamically generating linguistic outputs including stoπng a response history representing previous user responses in the input sequence, determining the probability of each of a plurality of the said classes being of interest to the user according to the response history and, according to the said determination, selecting certain of the said words and/or phrases and combining them to form a new linguistic output so as to determine with greater accuracy the class of interest to the user

A method according to claim 12, wherein each of a plurality of the said classes of objects is assigned a relevance weighting for a given user input sequence, and wherein the weightings are altered according to the response history as it develops

A method according to claim 12 or claim 13, including storing attributes relating to the said classes, the stored words and/or phrases including attnbute words and/or phrases describing the attributes, and, when the class of interest has been determined, generating linguistic outputs containing the said attribute words and/or phrases to provide information to the user on receipt of the said command signal

A method according to claim 12 or claim 13 and claim 1 1 , including routing the telephone call to a receiver associated with a determined object of interest

Computer apparatus having a dialogue-based input system for generating a command signal which depends on user responses to a plurality of linguistic outputs provided to the user by the computer apparatus, wherein the apparatus comprises - a dialogue generator for assembling linguistic outputs and for providing them to the user, a user response detector for detecting responses to the linguistic outputs, means for stonng at least one selected class of objects, means for stoπng a variable user interest structure to act as a representation of earlier linguistic outputs provided to the user and of detected user responses, and an inference system coupled to the selected object class stoπng means and the user interest structure stoπng means, and operable repeatedly to select different object classes for storage and modify the user interest structure in response to the detected user responses; the dialogue generator including vocabulary stoπng means for stoπng a plurality of words and/or phrases, and a message assembler for the linguistic output assembling in response to the selected class stored in the selected object class storing means; the inference system further comprising evaluation means for evaluating the user interest structure according to a predetermined measure and for generating 5 the command signal when the evaluation measure is of a predetermined value.

17. Apparatus according to claim 16, further comprising an audio output device ananged to receive the linguistic outputs and to provide them to the user as audio signals, and wherein the user response detector comprises speech recognition

10 means.

18. Apparatus according to claim 17, wherein the audio output device includes a speech synthesiser

15 19. Apparatus according to any of claims 16 to 18, further comprising means for storing a set of objects, a set of object classes, and a plurality of object/class relationships in the form of at least one fuzzy set.

20. Apparatus according to any of claims 16 to 19. wherein the user interest structure 20 storage means is ananged to store the user interest structure as at least one fuzzy set relating different classes of objects as a function of the user responses.

21. Apparatus according to any of claims 16 to 20, wherein the inference system is arranged to update the stored user interest structure and selected object class in

25 response to each of the at least some detected user responses according to the contents of the user interest structure.

22. Apparatus according to any of claims 16 to 21, wherein the dialogue generator is ananged to assemble at least some of the linguistic outputs on the basis not only

30 of the existing content of the selected object class storing means, but also on the basis of stored user response information using a fixed vocabulary of words and/or phrases stored in the vocabulary storing means.

23. Apparatus according to any of claims 16 to 23. wherein the dialogue generator is 5 responsive to the command signal to generate an output information message for the user.

24. Apparatus according to any of claims 16 to 23, wherein the inference system includes combining means operable to perform combination operations based on

10 fuzzy sets in order to update the user interest structure.

25. Apparatus according to claim 24, wherein the combination operations include fuzzy logic unions and intersections.

15 26. A telephone answering system including computer apparatus according to any of claims 16 to 25.

27. A system according to claim 26, further comprising call switching means responsive to the command signal to connect the user to telephone apparatus

20 associated with the selected object.

28. A method of operating a computer to generate a command signal in response to a dialogue-based input sequence comprising generating a plurality of linguistic outputs for a user, detecting user responses which are in response to the linguistic

25 outputs, and generating the command signal according to the dialogue constituted by the said outputs and responses, characterised by

(a) storing at least one selected class of objects,

(b) storing a user interest structure which is a variable representation of the linguistic outputs provided to the user and the detected user responses contained

30 in the input sequence, the user interest structure comprising at least one fuzzy set containing combinations of a plurality of classes of objects. (c) generating a linguistic output dependent on the selected class,

(d) detecting a user response which is in response to the said linguistic output and is in the form of one of a predetermined set of possible responses,

(e) varying the user interest structure according to the content of the 5 detected user response,

(f) selecting and storing a different class of objects to replace the said one selected class,

(g) evaluating the user interest structure according to a predetermined measure,

10 (h) repeating steps (a) to (g) until the evaluation is positive, and

(i) generating the command signal in response to the positive evaluation.

29. A method according to claim 28, wherein the computer forms part of a telephone answering system having a call switching circuit, the command signal comprising

15 a control signal for the switching circuit to route a telephone call to call receiving means coupled to the answering system.

30. A method according to claim 28 or claim 29, including generating an output information message for the user in response to the command signal.

^**) 0

31 A method according to any of claims 28 to 30. wherein the user interest structure is a primary fuzzy set of weighted secondary fuzzy sets containing combinations of a plurality of classes of objects.

25 32. A method according to any of claims 28 to 31, wherein the predetermined measure is fuzzy set entropy.

33. A method according to any of claims 28 to 32, wherein the variation of the user interest structure includes combining the selected object class with the user 30 interest structure using combination operations based on fuzzy sets.

34. A method according to claim 33, wherein the combination operations include fuzzy logic unions and intersections.

35 A method of routing a telephone call including a method according to any of claims 28 to 34, and the further step of connecting an incoming line used by the user to telephone apparatus associated with the selected object of interest.