WO2000043935A2

WO2000043935A2 - System and method and articles of manufacture for automated advisory decision and control services using decision systems with model license protection

Info

Publication number: WO2000043935A2
Application number: PCT/US2000/000335
Authority: WO
Inventors: Mehdi Aftahi; Pierre Boudreault; Perry Drobetsky; Donald J. Lobley; Edward S. Robins; Salim Tharani
Original assignee: Technologyevaluation.Com (Tec)
Priority date: 1999-01-08
Filing date: 2000-01-07
Publication date: 2000-07-27
Also published as: WO2000043935A8; AU2493500A; CA2258383A1

Abstract

A decision support system in which data structures (elementary models) consisting of a hierarchical arrangement of criteria and containing textual meta-data and numerical values, may be reduced to form a sub-set of criteria in which the sub-set of criteria (the aggregate model) represents a meaningful sample of the information content of the descendant criteria, tailored to the needs of specific client. A process is described wherein one expert in the art of tailoring of a large hierarchical structure customizes said elementary model to create the aggregate model. In addition, the expert may provide a set of charts and explanatory text - called decision objects, that may be clustered into visual decision dictionaries (VDD's). Said VDD's may further be clustered into decision procedures. Said decision procedures may then be used to guide a client through systematic and thorough analysis on the tailored model data in order to reach a well-informed and often negotiated decision.

Description

SYSTEM AND METHOD AND ARΗCLES OF MANUFACTURE FOR AUTOMATED ADVISORY DECISION AND CONTROL SERVICES USING DECISION SYSTEMS WITH MODEL LICENSE PROTECTION

FIELD OF THE INVENTION

The present invention relates to automated and assisted decision making. Automated assisted decision making is a relatively new field made possible by the advent and large-scale use of powerful general purpose and special purpose processors, and advances in the fields of decision science, game and utility theories, and graphical user interface technologies. Automated and assisted decision making may require initial input information from human experts and automated data collection systems in order to assist less knowledgeable or time restricted human operators or systems to make decisions. Such methods require detailed information gathered by knowledgeable experts in a particular field and formulated by said experts into a detailed knowledge base, where said knowledge base may be represented by a model consisting of criteria, and alternative strategies or items for selection, and criteria weights, evaluation information in the form of numeric values and textual and multimedia meta-data, and other such attributes as related to the criteria, the relationships amongst said attributes, and alternatives and alternative specific attributes and relationship of the said alternative attributes with the criteria attributes. Summarized and specifically tailored information, including attributes and attribute relationship, may be required by those less knowledgeable in the field but who may need to make a complex decision relying on said information, where said information can be very large. Assistance by means of human intervention and/or automated machine and automated processes can be applied to best tailor the complex information required for a specific decision and make the decision manageable and well informed. The detailed information underlying the specifically tailored data may need to be aggregated, hidden, removed, or appropriately filtered to create a summarized decision model for said decision makers in order to achieve a rapid, cost-saving, informed decision. Currently said data is generated by human experts and manually reduced, or said information is arbitrarily reduced without a specific process and is subject to human error. In cases where sufficient detail is deemed to exist, said detail may also relate to a decision maker's conditional acceptance of any given solution, thereby providing means for negotiation in reducing cost of the selection, and means to assess the benefit and challenges represented by the selected alternative. Said means, because of the overwhelming complexity of a typical problem, does not often provide adequate detail for detailed negotiation between parties. This can be important to the eventual implementation project resulting from a decision, an important final part of an advisory service.

More specifically, the present invention relates to a novel decision support process that determines decision making procedures through a series of appropriate graphical and verbal prompts, and provides adequate detailed meta-data without necessarily revealing other attributes which may be judged inapplicable, confidential, proprietary or simply overwhelming in volume and complexity to a decision-maker. In addition, such a system could in of itself create objects to enable a means to evaluate multiple variables within a multicriteria decision problem, and cause from the results of the method an action or provide reasons for a selection of at least one alternative amongst at least two alternatives, or issue a signal thereto to a human operator or device for decision and control action. Such systems are known as automated advisory and control systems, and constitute a decision support system and process means for customizing a process and aggregating, disaggregating and analyzing data which may result in conditional decision making. In a control process, this can provide for conditional systems control.

DESCRIPTION OF THE PRIOR ART Decision support systems are well known, and many methods for their use are known. Commonly, in decision support systems information is entered which may consist of a plurality of criteria arranged in a hierarchical tree to demonstrate relationships amongst the criteria, and one attribute related to each criterion to determine the importance of the criterion, commonly called a weight. At least two alternatives which are to be compared and rank ordered are scored (sometimes referred to as rated) usually against the lowest level leaf criteria in the hierarchy. The ratings and weights are aggregated in a mathematical formula to give a final score to rank the alternatives. In addition, other meta-data attributes (which may include text and links to objects over a computer network) with comparative qualitative information are added to assist a human decision maker in qualifying a decision, or selecting an alternative that may not be the first rank by the final score and may be selected when meta-data is provided that mitigates the score and causes preference for another alternative. Generally, such hierarchical decision models are evaluated on the basis of a single mathematical formula, requiring either the decision maker to understand the formula so presented, or taking in blind faith that the formula directly represents the aspirations of the decision maker, and sum-totals the decision. Often this is not the case and rule-based or other means are required to better represent the evaluation of alternatives.

To mitigate some of these problems, rule based expert systems such as that disclosed by Rhonda L. Alexander et. al. in US patent 5182793 "Computer-aided decision making with a symbolic spreadsheet" have been developed. However, such rule-based systems are classically complex, often requiring many rules that can confuse and do not address the clarity of the decision process. Obviousness of the process is masked by the complex number of rules, and rigidity of the process where such rules are essentially "Yes/No", precluding gray areas of understanding or negotiable points. F. N. Burt in US patent 4829426 "Computer-implemented expert system and method for decision-making" discloses a method of eliciting qualitative and verbal information by a mentor from a decision maker, and through a method of reduction by comparing alternatives a criterion at a time, arrives at a conclusion to select one, using Boolean, ordinal, cardinal, and qualitative information, from an expert human or data from a storage device and using equations provided and elicited by direct involvement of a human mentor (facilitator) to evaluate the alternatives. Such a process is well known among professional facilitators and has been criticized since the influence of the mentor can result in bias from mentor influence, and can also obscure the final result from the decision maker, since the mentor or facilitator must reduce the information for presentation and simplification purposes, often forcing "yes/no" responses to many questions that have more complexity to them.

Decision making software such as Expert Choice [manufactured by Expert Choice Inc.,5001 Baum Blvd., Suite 650, Pittsburg, PA, USA, 15213] and ERGO [manufactured by Arlington Software Corporation, 740 Saint Maurice Street, Suite 410, Montreal, Quebec] also provide the means to evaluate alternatives to arrive at a decision, including only text information as meta-data. However, no means is provided by which the decision maker is assisted through the decision process based on prior knowledge of other decision makers, and prior knowledge can also provide qualification information for selecting alternatives other than the top ranked alternatives. The volume of such information can be very large, and summarized information is a painstaking process of peeling down the data, as F. N. Burt demonstrates in US patent 4829426.

A decision support system for analyzing client satisfaction with vendor choices is disclosed by Richard H. Case et. al. of Gartner in US patent 5734890. The patent discloses a means to aggregate ratings and weights from a number of decision makers using a plurality of criteria with the intent on enhancing client satisfaction through a systematic and thorough methodology for collecting information and data related to weights and criteria. However, such methods are based on spreadsheet techniques, and the volume of data such methods can handle is limited by practical considerations. In addition, Gartner Group subsidiary Decision Drivers Incorporated [56 Top Gallant Road, P.O. Box 10212, Stamford, Connecticut CT 06904-2212] has developed a systematic and automated process based upon Expert Choice decision support software in which data is accumulated separately in data bases and spreadsheets, then the aggregate data is passed to the Expert Choice decision aid tool by manual cut and paste techniques and tools available in Microsoft Windows™ operating systems. This process can take several hours to days or even weeks. Further, not all the information is made available due to space considerations, technological inability, the physical impracticality of providing all information through 'cut and paste' methods, or for proprietary reasons. The current methods are thus very limited, and data integrity is easily compromised leading to potential loss of faith by clients in the efficacy of the method. Further, a full analysis can be limited by the size of models as judged from the number of criteria and alternatives the software can accommodate, for example models with 16,000 criteria and a dozen alternatives or more are likely, and may often be composed of component models which in themselves can be viewed as complete decision models for specific applications. Further, there are significant limits due to numerical computation time in lεirge models, model size and structure limits imposed by the decision support tool and additionally, the problem of determining all pairwise comparisons due to its pairwise methodology (a problem well documented and noted by many authors - for example, Millet and Harker, "Globally effective questioning in the Analytic Hierarchy Process", European Journal of Operational Research, 1990, 48, 88-9 and Olson and Dorai, "Implementation of the centroid method of Solymosi and Dombi", European Journal of Operational Research , 1992, 60, 117-129]. The simplification process causes the "invisibility" of all the underlying data as it is removed, and manual summarizing processes are easily subject to human error where employed, nor can the models be easily tailored to specific clients - indeed, it can become physically and economically impractical to do so with the manual process. The "data invisibility" can further make the process appear as a 'black box' process, and leaves many questions unanswered. Prior art has therefore given cause to doubt in the efficacy of the process, resulting in limited applications and low market penetration of currently available systems. Automating this process would therefore provide a great advantage, and enable automated decision tools to be more widely accepted, as well as proving the efficacy of the process. Automating such decision tools gives a major advantage to such a player in the new - industry called the Automated Advisory Services Industry.

In Expert Choice a means to compare alternatives exists in which the weighted differences of value are summed to produce a single number designating the advantage of one alternative over another from a selected parent node. Such a method is also described by by Richard H. Case et. al. of Gartner in US patent 5734890 in which the summed differences between products is called the 'Competitive Edge' of one alternative over a less valued alternative. This is an obvious process and is used in such processes as providing final quality rating of one alternative compared to another that is assembled from comparing component criteria. However, the competitive edge calculation is unrelated to the worth and cannot, for example, provide a direct costed negotiating point with a vendor in order to obtain a best price. This is a point completely missed by Case et. al. Also, other comparisons are unavailable, and only two alternatives at a time are compared, limiting the use of the process. Further, many decisions are not 'unidimensional' and require examination of worth in respect to target alternatives such as industry standards, and ideal alternatives; such means is not described in the method. Additionally, simple differences are inadequate to describe more complex decision strategies available, for example, in Arlington Software's ERGO 4.0 product. (ERGO 4.0 has no aggregate equivalent method for analyzing worth by summed differences across alternatives: it can use a more complex methodology based on pattern analysis as well as several scenario and sensitivity schema such as the addition and removal of criteria based on weight, which gives rise to decision strategies beyond simple direct cost and benefit comparisons). A more sophisticated methodology is required to translate differences that are not only linear summed differences in alternatives against a simple linear weighted sum decision strategy, but against a decision strategy obtained from a cluster of appropriate questions, and represented by a set of easily interpreted charts and numbers by one familiar with the art. Said decision strategy may be performed by those less knowledgeable in the art, and yet who may be the major decision makers. Providing assistance on an as-needed-just-in-time (AN JIT) basis can be provided with through networked technologies. Providing said AN JIT services would give significant advantage to a company providing services in the automated advisory services industry, particularly if said services are integrated with an integrated approach to the decision support system and the generated model. Further, a model containing sufficient but not overwhelming information that may dynamically provide adequately accurate scenarios by providing a client with simple procedural interfaces would enable different strategic decision scenarios to be examined. Such a tool would need to integrate decision support and decision making tools such as ERGO and Expert Choice with optimization and data compression methods to achieve rapid and sufficiently accurate results. Currently, no such system exists.

Since the selection among alternatives can require thousands of criteria in any given model, aggregating data is a necessity in order to make an informed decision feasible within a reasonable timeframe for an expert and non-expert alike. No decision aid is available which can adequately cover all known available data and aggregate the information automatically to enable thorough decision analysis. Further, no guidance is given in how to build a decision strategy that mitigates risk, and often there is a requirement to seek expert assistance in order to make such decisions, often adding significantly to the expense of the process as well as delaying the decision. It is tacitly assumed the expert has a thorough knowledge of the area, and can present an unbiased view. However, it is not possible for a human or even group of humans to handle the volume of data available without an automated tool given the complexity of many decisions. Additionally, 'unbiased' data implies the reconciliation of differing views and experiences from at least two sources of information, set in the context of the current decision, and this can represent a problem to said expert in terms of retention of past experience, maintenance of information concerning the field, and frequent reconciliation of differing and contradictory information. Without an appropriate tool, said expert cannot claim to be unbiased. Often said experts are linked with one or two preferred vendors, and have deeper knowledge of said preferred vendors than other vendors. Thus it can be said that most experts in the field cannot provide an unbiased view of all available information, and have no tool to back up such a claim. As well, aspects of the detailed analysis may be proprietary or confidential or not relevant in a specific decision, or necessary in order to compute aggregate values, and consequently a means to tailor decision models 'on-the-fly' may present significant advantage. Tools such as Expert Choice have limited capabilities in this respect - indeed, it is not even possible for some attributes - particularly in regard to hiding/revealing specific data and. aggregating data in at least one node in the hierarchical tree. ERGO has several tools which provide flexibility to the model building process such as Append and Save sub-trees, but none for automatically creating aggregate and hidden nodes, with meta-data information embedded in the aggregate fields for example. For automated advisory services and automated decision and control process, such knowledge based aggregating processes and knowledge aggregating systems would add a significant competitive advantage, providing clients an unbiased and systematic process for making decisions in highly complex environments. Currently, no such system is known to exist.

In addition, systematic and learned methods maintained in a knowledge base may mean that valuable techniques and methods are passed on to those who may make, or may assist in making, similar decisions. Such methods may be scripted, and said scripts customized to enable both a client and an expert of an automated service to follow set procedures, and benefit from a store of past experience by others. As well, such processes enable the systematic training of decision makers and experts more amenable since case studies are well documented. Controlled access to said knowledge bases and tailored scripted procedures also provides additional revenue leverage for an automated advisory service. Thus an integrated approach to knowledge base generation, script customization, training of clients and experts, centered around means to make decisions, would be highly beneficial. As well, no decision support tool provides a systematic means to aggregate, accumulate, and store the methods and procedures by which decision makers reach decisions, including the practical experience and true costs of, for example, a missing vendor feature, and the constraints employed to assist in price negotiations as another example, or methods by which specific issues are resolved with particular vendors and clients. In providing such means to systematically describe decision processes, and storing said decision processes in a knowledge base, and providing a decision maker the means to utilize this knowledge in a structured fashion, would result in a significant advantage for any decision maker. It would also apply to any process making decisions where consequences can measurably result, and where said processes can benefit by prior knowledge of procedures used for making similar decisions. Said process is also of value to vendors seeking to improve performance, and provide direction for future product development in competitive markets since it would provide the ability of knowing how clients select their products and competitive products. The information for the said knowledge base may be garnered from non-proprietary and proprietary sources, from clients, public literature, vendors, and other sources general by at least one expert in the field.

Said knowledge base may include attributes specific to at least one alternative, and may provide at least one significant summary information a client of an advisory service may find relevant. Such information as the tradeoff a vendor may have made, knowingly or unknowingly, between feature capability and complexity of the product, compared to other vendors, may be of significant value in understanding the challenges that may be faced by the client, and may be of interest to a vendor who may find at a glance the summarized data and position of the vendor product with respect to the industry. Other attributes may be related to costs and business risk, vendor market positioning and future vendor development goals, and so forth. In some cases, some products may be composed of a cluster of products, and hence a hierarchy of products, possibly designated in terms of an alternative hierarchy, may provide a breakdown of a vendor's offerings in component form. To date, no such decision aid capable of examining hierarchical alternative structures exists.

In another aspect of said disclosed system and process herein, such data and meta-data may consist of engineering information related to risk in a control process such as that in a nuclear reactor where many thousands of nodes represent a hierarchical control structure, the accumulated risk as components of a system deviate from a norm (constraint) being so estimated by the decision support system, and possibly assisted by prior knowledge from similar situations in other systems. Data and meta-data for and aggregated node in the , hierarchy can be continuously updated, the advantage thereby being that operators are not inundated by information in a large complex system, and only such data is revealed by a process of node disaggregation when for example a local constraint is exceeded or a particular pattern of information occurs below the node. Decision support processes for automated advisory and automated decision and control systems are of the same process in either case in terms of the required automated aggregation process. In automated decision support processes, however, disaggregation may not be available, or available by special license, thereby providing added service feature options by the automated advisory service. Again in current systems, said service feature options are not known.

Data and meta-data for a knowledge base may be garnered from experience, statistical reliability data, and so forth. It is also obvious that it would be advantageous to minimize the intervention of a human expert, who may have specific biases from limited knowledge and other interests, through a systematic process of model customization and use of the accumulated systematic knowledge in said knowledge database.

In all current decision support systems text information alone is provided, and direct connections "on-the-click of a mouse" to expert opinion provided by means of direct object linking to a source of information which may be local or remote to the user, are not directly incorporated as part of the decision model, and updating of said information is not available by any automated process.

In another aspect, aggregation of numeric data would mitigate problems related to rapid and accurate dynamic feedback to develop what-if scenarios 'on-the-fly' for large and complex systems. Said dynamic analysis uses values and parameters of the model - such as weights and ratings - which are changed 'on-the-fly' to evaluate the effects on a particular decision or evaluation of at least one alternative, usually compared to at least one other alternative. Because computation times can require at least several seconds or even minutes for such large models, such dynamic analysis is not feasible for interactive use, and is difficult to perform even with the highest speed desk-top systems once model sizes and number of data values, plus updating of chart information on a graphical display for example, are included. If the information is aggregated, the computing problem would significantly be mitigated. For some parametrics such as Arlington Software's Matching Index and Composite Index, disclosed in US patent number US5844817 the calculations require sophisticated aggregation techniques not possible with any current decision support system. A means to mitigate the computational time of numerical calculation for large numbers of criteria through pre-calculation and storage of aggregated computed parameters at specific nodes in a given model would provide a significant advantage in reducing computational effort, and significantly extend the use of an aggregate model where data has been aggregated to high level nodes. In advising said client about possible scenarios, interactive graphical procedures would provide rapid answers not available in any current system.

In another aspect of deficiency, all models created by decision support systems have unlimited use in terms of time, and can be distributed without restriction to any who may have the appropriate machine readable code to read said model. Protection of proprietary information is compromised, and unlicensed use of the model content is therefore at risk. Currently, only machine readable code that executes on a processor may be time limited or restricted to a particular processor. A means to limit the period of availability of a model, its data and meta-data content, features available to a decision maker, and to limit its distribution, would provide greater control over the model use, and enhance the revenue generation for an automated advisory or automated control system. In addition, prevention of unauthorized input and output of said models is required, preferably by encryption and decryption method. In addition, a customized set-by-step guideline for making a decision in the context of a particular client would provide the said client and advisory service with a clear procedure and process. A means to determine a step-by-step procedure, and execute it, incorporating in at least one of the means to mitigate said problems described herein, would be highly advantageous to an automated advisory service.

To mitigate at least some of the deficiencies in current systems and processes, a novel and comprehensive decision support system and process for knowledge accumulation, knowledge aggregation and disaggregation, and process for its controlled use with knowledge based systems in automated decision processing, is disclosed herein. The decision support system provides a means to aggregate data and meta-data, determine decision procedures and related costs and risks, and disaggregation techniques. Means to prevent unauthorized use of model and model data is further described. As well, a method and process is described for extending these means and processes to automated large decision and control systems, and process means to provide a customized step-by-step customized and packaged procedure.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a novel decision support system and method which obviates or mitigates at least some of the above-mentioned disadvantages of the prior art. It is a further object of the present invention to provide a novel article of manufacture that obviates or mitigates at least one of the above-mentioned disadvantages of the prior art, and provide process means to take advantage of the present invention.

According to one aspect of the present invention, there is provided a computer- implemented decision support machine for comparing two alternatives, comprising: memory means for storing at least one decision data structure with each said decision structure having a plurality of decision criteria organized in parent category nodes with child (descendant) nodes; input means to assign to each said decision node a weight value to represent the importance of each said node to the decision, the plurality of weights and nodes comprising a predetermined two-dimensional benchmark pattern; input means for inputting a first plurality of scores for a first competing alternative to the decision factors of said decision data structure, and input means for inputting a second plurality of scores for a second competing alternative to the decision factors of said decision data structure; input means for entering meta-data for at least one node, wherein said meta-data consists of text information and can further include means for multimedia links to local objects and can also contain links to remote objects and experts located on a computer network or telecommunications network; input means for entering at least one raw score which can be ordinal or cardinal data; input means for determining how at least one raw score is translated into a utility value representing the worth of the raw score toward said decision for at least one said alternative; processing means to compute the utility value from the raw score value for at least one of said alternatives; input means for identifying at least one node as an aggregate node into which descendant criteria attributes below the aggregate node are aggregated into attributes in the aggregate node; input means for identifying at least one node as an aggregate hidden node into which descendant criteria attributes below the aggregate node are aggregated into attributes in the aggregate hidden node; input means to determine an aggregate hidden node and all of its descendants is not to used in determining the decision outcome; input means to expose a aggregate hidden node including all its descendants in the hierarchical tree, causing said hidden nodes to be used in determining a decision outcome; input means to determine at least one descendant node below an aggregate parent or aggregate hidden parent node is included in the aggregation to the parent aggregate node; input means to determine at least one meta-data attribute to be aggregated for at least one descendant node; input means to determine the aggregate meta-data field where at least one meta-data from at least one descendant node is aggregated; output means to indicate the status of a node and distinguish by means of a visual marker on a graphical display the determined aggregation status of the node; processing means to aggregate at least one attribute of at least one descendant node in the aggregate node in accordance with the aggregate node status; input means to assign at least one code determining the limitations on access rights to said aggregate model, said availability being determined by an authorized site; input means to assign at least one unique identification code to said aggregate model; input means to remove at least one node and node attributes from said aggregate model; input means to determine at least one attribute fir at least one alternative; input means to cause said at least one alternative attribute to be retained in the model following aggregation; input means to cause at least one alternative attribute to be dependent on the value of at least one other model attribute; processing means to determine at least one parameter that determines at least one aggregate score for an aggregate parent node; processing means for transforming (i) the first plurality of scores into a final aggregate score including using at least one aggregate and aggregate hidden node score parameter and (ii) the second plurality of scores into a final aggregate score including using at least one aggregate and aggregate hidden node score parameter; input means to cause said decision support system to select from the available alternatives a 'best set' of alternatives determined to best meet client requirements; input means to determine an industry standard alternative, said alternative having a plurality of scores determined from the plurality of scores of all scored alternatives; input means to determine an alternative-specific standard, said alternative specific having a plurality of scores determined from the plurality of scores of at least two selected alternatives; input means to determine at least one report template; output means to store at least one report template; input means to input at least one report template; output means to output at least one output signal corresponding to at least one of the first and second final aggregate scores to provide a ranking of said at least two competing alternatives. input means to determine at least one constraint for at least one decision factor; processing means to determine comparison between at least one alternative score attribute value and at least one constraint; input means to determine at least one constraint from at least one score of at least one alternative; processing means to select at least one 'worst constraint' condition and at lesat one 'best constraint' condition where at least one worst constraint and at least one best constraint may be computed from input data and from a selected set of alternative data; storage means to store at least one decision object with at least one constraint attribute as a decision scenario object with constraints; processing means to compare the score of at least one decision factor for at least one alternative against at least one constraint; processing means to determine an aggregate of all values of comparison to provide at least one final value representing the overall difference value for at least two alternatives when compared to the said at least one constraint; output means to output at least one output signal corresponding to at least one aggregate value differences to provide a conditional comparison value or worth of said competing alternatives. input means to input at least one code, said code determining license conditions on said model; output means to store model as a data structure on a storage device, said model being designated as an aggregate model; and processing means to encrypt and decrypt said aggregate model.

Preferably, the aggregation procedures are executed by at least one Analyst who is knowledgeable in the field of the model and its applications. The at least one Analyst is a trained expert the model and determines the aggregate model from a larger model.

Preferably, the aggregate model outcome determination method represents as closely as possible the required means by the at least one client. Determination of said decision outcome by means of a multiattribute value such as the linear sum of weights and scores is one preferred embodiment. A second preferred embodiment is the use of pattern match algorithms such as Arlington Software Corporation's Percent Match pattern matching algorithm. Supporting meta-data for a decision outcome, in one preferred embodiment, is customized to meet client requirements.

In one preferred embodiment, said at least one aggregation parameter are obtained by machine readable executable code on the general purpose processor on which the model aggregation procedure is executed. In one preferred embodiment, said aggregation parameters enable aggregate values to closely estimate exact values without requiring the larger data set to obtain the exact values, said estimation being necessary in the event of changes of weights, scores and other values during investigation of various scenarios using said aggregated model by said at least one client. In another preferred embodiment, the Analyst provides the data of the customized Analyst model prior to aggregation of the said at least one determined aggregate node, and uses machine readable executable code on at least one processor such as a neural net, thereby generating said at least one aggregation parameter. Said at least one aggregation parameter is then stored with the said aggregate model as at least one attribute of the said at least one aggregate node.

In another preferred embodiment, the Analyst determines that the aggregation parameters are not required, and does not generate said aggregation parameters. In one preferred implementation, the aggregate data and meta-data is selectively output to a printer or disk file. In another embodiment, said aggregate model is encapsulated in an electronic message to generate a request for proposal (RFP) to elicit responses from the 'best set' of alternative vendors. In another embodiment, said vendors are replaced by alternative strategies, actions, project alternatives, and personnel, and other such choices as would occur to, and be evaluated by, the at least one Analyst.

In another preferred embodiment, the aggregate data and meta-data is selectively output to a printer. In another embodiment said selected data and meta-data is saved as a disk file such as a spreadsheet or as a word processor document as is commonly known, and other known disk file formats as is available at the time. Said selected data and meta-data in one embodiment is transmitted over a computer network. The determined format, however, contains a summarized report on a limited number of preferred vendors most likely to satisfy the requirements of a client. Said preferred vendors can, in another preferred embodiment, be selected by automatic process means, said selection being performed by said decision support system. In another embodiment, said preferred vendors are manually selected from the list in the full Analyst model by the said Analyst prior to storing the aggregate model. A report on said selected vendor can in one embodiment include exception reporting and summarized vendor strengths and weaknesses, and other data and meta-data that is deemed relevant by one knowledgeable in the art.

In another aspect of the current invention, the at least one Analyst determines at least one attribute to assign to at least one alternative, said attribute having at least one value summarizing at least one aspect of the alternative. At least two attributes can be arranged in a hierarchical tree indicating hereditary relationships amongst the said attributes. In one embodiment, said hierarchical arrangement determines sub-alternatives. Said sub-alternatives are scored as independent alternatives, and said scores are used to determine the outcome score of the parent alternative. In another embodiment, said attributes represent a hierarchical- relationship of attributes for a single alternative. In either case, said at least one Analyst indicates by input means said means by which at least two attributes are to be aggregated and retained in the aggregate model. In another embodiment, said at least one Analyst causes said at least one attribute for at least one alternative to be provided in the form of a chart. In a further embodiment, said chart is given in the form of meta-data, thereby providing a client with the summarized values and meta-data information applicable to the specific requirements of the client. In another aspect of the current invention, the said limits on access and distribution at a client site for a provided model may include, but not exclusively, timed availability, availability of different features of the model and the decision support system, model storage location limitations, limitation of access by specific at least one individual, and number of licensees with access to the model in a file server environment, and limitation on the number of scenarios a client is allowed to generate. In one preferred implementation the first reading from storage of said model by a decision support system supplied to a client causes said decision support system to prompt for a second licensing key code to initiate time and feature availability. An authorizing site may then be requested to provide the second key code, and may be provided on determining said client is authorized. On inclusion of the second key code, the rights and privileges of access and distribution to said aggregate mode are accorded for the said provided aggregated model. In a preferred embodiment, means to protect from the unauthorized determination of the internal data structure of said model is provided by means of encryption and decryption methods incorporated in said decision support system. In another aspect of the current invention, the decision support system is provided to the client. In one embodiment of said client provided decision support system, at least one feature is removed. In one preferred embodiment, the said removed feature is the ability to determine aggregate nodes. In another preferred embodiment, said decision support system model access code limitation input/output features are removed. In another embodiment, said client provided decision support system includes means to determine access rights and limitations to at least one said customized model.

In another embodiment, the aggregate model is provided with a code to prevent further aggregation by said decision support system. In another preferred aspect of the current invention, the said provided model license may be transferred by a process of license transfer from one processor to another, thereby maintaining the number of licenses to a specified number.

In yet another preferred aspect of the current invention, said license may require renewal from time-to-time. In this process a validation of the expiration status of the said provided model is determined by the client provided decision support system. If the expiration status indicates the said provided model has exceeded its limited period of use, processor implemented code in the said decision support system may initiate a request for license renewal, said process in one embodiment being by automated means over a computer network such as the Internet. Another embodiment of said process is an appropriate indication to the licensee of the expiration status of the said provided model. In another embodiment, such expiration warnings may appear prior to complete expiration as a warning to the licensee, thereby providing the licensee the opportunity to ensure continued use of the model. In another preferred embodiment, the at least one conditional difference value to compare at least two alternatives is determined by relating at least one cost attribute to at least one criterion node score difference value, said cost usually being expressed, but not exclusively, in terms of a monetary unit. In another preferred embodiment, said conditional value difference may be in terms of a project success risk value, and in another embodiment said success risk value is expressed in terms of a success probability.

According to another aspect of the present invention, there is provided a computer- implemented decision support system for comparing two alternatives, comprising: memory means for storing at least one decision model data structure having a plurality of decision factors organized in parent category nodes with child (descendant) nodes, input means to assign at least one weight to at least one decision factor input means to input at least one alternative; input means to input at least one plurality of scores, said at least one score being assigned to at least one decision criterion for at least one alternative processing means to determine at least one multiattribute score for at least one alternative from said at least one plurality of scores for said alternative; input means to input meta-data into said model; input means to determine at least one chart comprising at least one data value in said model; input means to determine at least one report template; input means to assign at least one meta-data attribute to said chart; input means to input meta-data to at least one meta-data attribute field, where said meta-data includes at least one instruction on using said chart, thereby causing said chart to be determined as a decision object; input means to input meta-data in at least one other met-data attribute field of said chart; input means to assign said at least one decision object to a visual decision dictionary, said visual decision dictionary comprising of at least one decision object; input means to input meta-data with at least one instruction on the use of said visual decision dictionary; input means to determine at least one visual decision dictionary with said input meta- data as a decision procedure; input means combine at least two decision procedures, thereby determining a decision process, wherein said at least two decision procedures are executed sequentially as in an at least two step process; input means to input meta-data into decsion process wherein said meta-data provides guidance on said at least two steps; input means to remove at least one decision object from said model; input means to delete at least one decision procedure from a decision process; output means to output at least one decision object in a data structure; output means to output at least one visual decision dictionary in a data structure; output means to output at least one decision procedure in a data structure; input means to input at least one decision object into said model; input means to input at least visual decision dictionary into said model; input means to input at least one decision procedure into said model; output means to output at least one output signal representing said at least one assigned code enabling access to external meta-data transmitted over a computer network; input means to input at least one signal representing a request for at least one decision object; input means to input at least one signal representing at least one decision object; output means to output at least one signal representing at least one decision object. input means to assign at least one decision object, visual decision dictionary, and decision procedure to at least one said report template; output means to output at least one report template as a report data structure; input means to input at least one report data structure;

Preferably said assignment of meta-data instructions and associated meta-data for charts, decision objects, visual decision dictionaries and decision procedures is performed by an Analyst who is an Expert in the decision area to which the model is applied. In one preferred embodiment, said definition and assignment is performed when aggregating the complete model, and in one embodiment is saved in the aggregate model. In a second embodiment, the Analyst edits the aggregate model to add, modify and delete decision objects, visual decision dictionaries and decision procedures. In another preferred embodiment, said at least one decision object, visual decision dictionary and decision procedure, report template, customized report is transmitted by the decision support system over a computer network, and are sent from one decision support system operating on one processor on the computer network, and read by a second processor on said same computer network executing a second decision support system.

In one embodiment of said process, it is contemplated that a decision procedure, visual decision dictionary and decision object are identical. In the preferred embodiment, said decision procedure contains a plurality of visual decision dictionaries with meta-data guidance to guide in the use combined use of said plurality of visual decision dictionaries, wherein each said visual decision dictionary consists of a plurality of decision objects, said visual decision dictionary containing meta-data instructions concerning use of said plurality of decision objects in combination. Said meta-data guidance can be considered in this embodiment as a set of scripts.

According to another aspect of the present invention, there is provided a computer- implemented decision support system for comparing two alternatives comprising: storage means for an aggregate model data structure; output means to output meta-data guidance in selecting said at least one decision procedure; output means to output meta-data guidance in selecting said at least one visual decision dictionary; output means to output meta-data guidance in selecting said at least one decision object; input means to select at least one decision procedure; input means to select at least one visual decision dictionary; input means to select at least one decision object; input means to input meta-data in said at least one decision procedure; input means to input meta-data in said at least one decision object, at least one visual dictionary, at least one decision procedure; storage means to store at least one decision object; input means to select at least one decision object; and output means to store said aggregate model.

Preferably, in one implementation of the process, an Analyst who is an Expert in the field of the model determines at least one decision object, at least one visual decision dictionary and at least one decision procedure for the decision making process based on prior knowledge acquired by facilitation with said client and from experience of the Expert. Said customized aggregate model then incorporates client requirements and said client constraints.

In another embodiment, said client further customizes the aggregate model as determined here. Said at least one Analyst in one embodiment assists in said customization process. Preferably, aggregate score and value differences represent as closely as possible the aspirations of the decision-maker. Aggregation by linear sums of weights and ratings is one preferred embodiment. A second preferred embodiment is the use of pattern match algorithms such as Arlington Software Corporation's Percent Match and Composite Index pattern matching algorithms. In another preferred embodiment, said at least two decision procedures are bound together to form a complete guide to at least one decision process. Said guide may be determined by at least one knowledgeable in the art of making said decision. Input means may be provided by said decision support system to build said guide where and said guide may include at least one of, but not exclusively, report template, request for proposal template, decision procedure, visual decision dictionary, decision procedure library containing at least two decision procedures, bid letter template, and may include meta-data and support meta-data indicating the use of each decision procedure constituting said guide. In one preferred embodiment, said guide consists of machine readable code executed on a special purpose or general purpose processor. In another preferred embodiment, said guide resides on a computer network and is accessed by means of at least one output signal from a device such as an intelligent paper clip, said clip being provided with an appropriate code that can be read by an infra-red or radio frequency device, and said code containing information on location of the said guide on the computer network.

According to another aspect of the present invention, there is provided a computer- implemented decision support system for comparing two alternatives, comprising: input means to select at least one aggregate model; output means to store all attribute numeric data set of said aggregate model in as a structured data set; output means to determine a copy of said all attribute data set; input means to select at least one chart; input means to select at least one object on said chart, said at least one chart object representing at least one numeric attribute data value; input means where input causes said selected at least one chart object to change size, position, shape; processing means to determine said at least one attribute value resulting from change of size, position, shape of said at least one chart object; processing means to determine at least one attribute value at predetermined at least one interval, said at least one interval being based on size, shape, position of said at least one chart object; output means to output at least one signal representing said at least one changed position, shape and size of the at least one chart object; output means to transmit at least one signal to at least one other chart in the current chart set, causing said chart to determine at least one attribute value assigned to said at least one other chart; output means to update the display on a graphical display device of said at least one chart objects currently displayed; input means to select one copy of a chart, said copy having the unchanged attribute data set determined from stored set of data; input means to determine said copy of a chart attribute data is not caused to change as a result of at least one change made to at least one chart object in at least one other chart, said copy of chart is then said to be locked; processing means to determine at least one value representative of the difference between at least one selected locked chart attribute data and at least one said changed chart data attribute; input means for meta-data input to at least one decision procedure; output means to insert date and time data in said decision procedure to determine a history of said changes, said changes being thereby stored for future reference; output means to output at least one signal representing the changed data; temporary storage means to store set of changed attribute data as a structured data set; input means to store at least one set of said changed attribute data; storage means to store at least one set of said changed attribute data as a scenario of said aggregate model;.

Preferably in one implementation of the process, the client uses the method to devise and compare different decision scenarios. In one preferred embodiment, the attribute data and meta-data are assigned to report templates to determine at least one report, said at least one report in one embodiment being output on a device such as a printer. In another embodiment of said process said at least one report is stored in a disk file, and said disk file sent as an encapsulated item in an electronic message over a computer network..

In another embodiment, said at least one scenario is stored as a distinct model in a data structure, where said model represents one scenario. Said data structure in another embodiment enables selective access to at least one attribute datum, thereby providing means to develop further scenarios consisting of attribute data from at least two sets of attribute data. In a preferred embodiment, said further scenario consists of one set of decision factor weights, and one set of alternative scores from at least one other scenario.

According to another aspect of the present invention, there is provided an automated advisory service process comprising: means to store data in a knowledge base, said knowledge base being a structured data set determined for a specific type of decision and consisting of at least one decision object; stored data structure consisting of at least one non-aggregate model, said model consisting of a plurality of decision factors arrange in a hierarchical relationship, a plurality of alternatives, a plurality of alternative attributes, a plurality of reconciled scores where reconciled scores are determined from scores from at least two sources, for each of said plurality of alternatives, and a plurality of meta-data indicating at least said contents of said model; means to determine industry standard attribute values from said at least one non- agregate model; means to determine the performance of individual alternatives; means to determine client requirements; means to provide client with at least one decision procedure; means to provide client with at least one decision process; means to determine at least one best alternative to meet said client requirements; means to customize said at least one non-aggregate model; means to provide alternative overviews to assist client in a learning process in said decision; means to license said aggregate model for said client, wherein said license imposes limits of access on said aggregate model; means to input at least one decision procedure in said aggregate model; means to customize reports for said client means to assist said client in report generation; means to assist said client in generating a request for proposal; means to assist said client in management review processes; means to assist said client in letter of bid; means to assist said client in negotiation procedures and tactics; means to assess said selected alternative in terms of alternative performance; means to provide continuous update to said knowledge base and said non-aggregate model; means to distribute said updates to subscription clients;

In the preferred embodiment said at least one Analyst uses said decision support system to aggregate and customize an aggregate model for said client. Said means to provide services in the preferred embodiment uses at all stages said decision support system. In another embodiment, a separate knowledge base is used to store decision objects, constraint data, meta-data and other data as may occur to one knowledgeable in the field, said data being determined from prior experience of said at least one alternative. In the preferred embodiment, said alternatives are vendors. In this embodiment characterization of said vendors and said characterization of Client provides means to compare future clients against vendors, and can be used to determine expected success attributes and attributes wherein challenges are likely to be faced by said client on the selection of at least one specific vendor. According to another aspect of the present invention, there is provided a decision support system for use with a controlled process or system, comprising: storage means for a decision data structure having a plurality of decision criteria organized in parent category nodes with child (descendant) nodes; means to assign each said decision node a weight value representing its importance to the decision, the plurality of weights and nodes comprising a predetermined two-dimensional benchmark pattern; means to assign a first plurality of scores for a first competing alternative to the decision factors of said decision data structure, and means to assign a second plurality of scores for a second competing alternative to the decision factors of said decision data structure and; means to assign reference meta-data for each node; means to assign a method to translate at least one score into a utility value representing the worth of the said at least one score toward said decision for at least one alternative; processing means to compute at least one utility value from at least one score value for at least one of said alternatives; means to determine at least one node as an aggregate node; means to determine at least one node as an hidden aggregate node; means to determine at least one hidden aggregate node as not to be included in determining at least one decision multiattribute value for at least one alternative; means to expose at least one hidden aggregate node including all its descendants; means to determine at least one descendant node below said aggregate parent node as a node whose at least one attribute is aggregated in the parent aggregate node; means determine at least one meta-data attribute to be aggregated; means to assign at least one meta-data field determined to be aggregated to at least one meta-data aggregate field in the aggregate parent node; processing means to aggregate attributes of selected descendant nodes.; storage means to store aggregate model as a structured data structure; processing means to determine at least one parameter that determines at least one aggregate score for said at least one aggregate node; processing means for transforming (i) the first plurality of scores including said determined scores determined from at least one aggregate node parameter to a multiattribute value for the first alternative and (ii) the second plurality of scores including said determined scores determined from at least one aggregate node parameter multiattribute value for the second alternative; output means to output at least one output signal corresponding to at least one of the first and second multiattribute scores to provide a comparison of said competing alternatives; means to determine at least one decision procedure; means to store at least one decision procedure; means to select one decision object; means to input at least one constraint for at least one aggregate model attribute; means to determine at least one dependent relationship between said at least one constraint and at least one aggregate model attribute; means to determine said at least one constraint from at least one attribute value of at least one alternative; means to determine at least one 'worst constraint' condition and at least one 'best constraint' condition where worst and best constraints represent extreme acceptance conditions; processing means to compare at least one score for at least one alternative to at least one constraint condition and output at least one value for the comparison; processing means to determine an aggregate of all said values of comparisons to provide a final value representing the overall difference value of at least one alternative when compared to the at least one constraint; output means to output at least one output signal corresponding to at least one of the value differences; input means wherein input of at least one signal causes said at least one hidden aggregate node to expose said at least one hidden descendant node; input means wherein input of at least one signal causes said at least one hidden node descendant nodes to be hidden; input means wherein input at least one signal causes generation of at least one decision factor, said at least one decision factor representing an input element in said control system for said decision support system; input means wherein at least one signal causes the removal from said model of at least one decision factor, wherein said decision factor represents said output source of said input signal.

In one preferred embodiment of the process, at least one Analyst or persons knowledgeable in the field determines at least one decision procedure and at least one constraint for at least one decision object. Said at least one procedure and at least one constraint for the control process is based on prior knowledge which is acquired from facilitation with a client, and is additionally determined from experience with same or similar automated processes. In another embodiment an automated system provides as output for at least one human operator at least one decision procedure wherein attribute values and metadata represent the aggregate information and the decision making precepts and status of the automated system. In yet another embodiment, at least one exception datum is output when an attribute value exceeds said at least one constraint. In one embodiment said exception output is to a graphical display device. In another embodiment said output signal is directed to an automated process or system. In a further embodiment, said at least one exception is output to a printer, and can further be output to a disk file, and further, said disk file can be encapsulated in an electronic mail message, said message being transmitted over a computer network.

BRIEF DESCRIPTION OF THE DRAWINGS Embodiments of the present invention will be described, by way of example only, with reference to the accompanying drawings, in which:

Figure 1 shows a block diagram of computing hardware embodying a preferred embodiment of the present invention;

Figure 2 shows a pictorial representation of an embodiment of the computing hardware of Figure 1 indicating a hardware automated decision and control process; Figure 3 shows a pictorial representation of a second embodiment of the computing hardware of Figure 1, indicating computer network client/server capability and input/output devices; Figure 4 indicates the aggregation and tailoring procedure of reducing a large model to a client specific aggregate model with aggregate and hidden nodes;

Figure 5 shows the procedure used to construct decision objects with meta-data and numeric data; Figure 6 shows the procedure to compute conditional values such as costs from constraint information specific to client and to the alternatives;

Figure 7 shows an example of the different fields and visual cues for the aggregate status of criteria and preview of meta-data aggregated text fields;

Figure 8 illustrates means to preview aggregate text fields, and graphic images that may be saved as objects in the meta-data;

Figure 9 shows means to assign aggregate status for the node;

Figure 10 shows the means to customise the aggregation process by selecting the different fields to aggregate, what is to be aggregated, and where the aggregate information is to go. It also indicates how the aggregated evaluation is to be represented and evaluated; Figure 11 shows the way in which different chart objects may be dynamically changed in order to produce dynamic evaluations and scenarios;

Figure 12 shows a decision procedure in which rating values are adjusted, and how a chart value may be locked, refreshed and in order to control the comparison;

Figure 13 shows the report generation stages of the procedures to generate RFP's for example, and final reports;

Figure 14 shows the licensing and license management process at a client site, and at an authorisation site;

Figure 15 shows the license transfer process to transfer a license from one processor to another. Figure 16 shows one part of a visual decision dictionary / decision procedure composed of decision objects and means by which criteria attributes may be used in chart data;

Figure 17 shows a graph of weighted average value against the absolute sum difference function, indicating the approximation method results from the optimised selection of a fitting function;

Figure 18 shows a visual decision dictionary in which chart objects are divided between data modification charts and impact charts, and where meta-data technology is used to provide assistance and guidance to the client; Figure 19 shows an alternative hierarchy and associated attributes showing how attributes are inherited for a composite alternative;

Figure 20 shows means to assign attributes and attribute values and properties such as equations relating to other attribute values and attributes of factors in the model tree; Figure 21 shows the attribute summary table with attribute values for the at least one alternative, and means to filter alternatives by said attributes;

Figure 22 shows an overview of a scripted process for assisting a client through a procurement selection process, and means by which continuous improvement and updating of model information is obtained; Figure 23 shows the part 1 of the procedure for initial client needs specification and procedure for tailoring model and decision procedure scripts;

Figure 24 shows part 2 of the procedure for further detailed analysis of vendors with assistance from at least one Analyst;

Figure 25 shows parts 3 and 4 of the procedure wherein vendor selection is finalised, bid letters are provided to vendors, and proposal-counter-proposal procedures may be used. Part 4 indicates means for post- vendor selection and follow-up on execution of project in order to provide not only added revenue, but means to update information in the area of the decision model;

Figure 26 shows means a decision process guide by which a decision support system may provide a guide through one or more steps in a decision process

Figure 27 shows a decision procedure governing an automated process in which decision objects may be employed to reveal or hide data regarding the condition of a system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Figure 1 shows a block diagram of a decision support system 20 in accordance with a preferred embodiment of the present invention. System 20 includes processor means 24, input means 28, output means 32 and storage means 36.

In a first preferred embodiment, as shown in Figure 2, processor means 24 is an embedded control processor, such as a Motorola 68HC16 and associated circuitry 40; storage means 36 comprises a ROM memory 44; input means 28 comprises a series of sensors 48; and output means 32 comprises a known controller 52 for a process which produces appropriate output signals 56. In the embodiment illustrated in Figure 2, controller 52 can, for example, be a rocket vehicle launch controller and sensors 48 may comprise launch status sensors, type of rocket (manned, unmanned) containing meta-data and data input signals, service request counters, timers, etc. and radio receivers for remote site data input. This specific implementation of the embodiment of Figure 2 is discussed in more detail below. In otherwise similar preferred embodiments, wherein system 20 is used to control a number of different activities in an industrial process, for example the control of different chemical process reactors, system 20 may be implemented from appropriate discrete components. In such an embodiment, input means 28 can comprise one or more appropriate sensors 48, for example position, load, demand, velocity, pressure and/or temperature sensors, and database information for each of a plurality of reactors, and output means 32 will comprise one or more means to generate control signal outputs, for example variable voltage signals to control dc motor speeds, solenoid valve or brake actuation signals, etc. If required, the signals from sensors 48 can be translated into an appropriate format for processing by processor and associated circuitry 40 by, for example, an analog to digital converter, a protocol converter (in the event that the sensor signals are provided over a network), or other suitable conversion means as would occur to those of skill in the art. In addition, output means 32 may provide at least one input to a second decision support system wherein the input may control data and meta-data presentation to human operators.

As used herein, the term process is intended to encompass commercial, industrial and other processes and to include, without limitation: mechanical and/or electromechanical operations; chemical process control; HVAC (heating ventilation and air conditioning) systems; robotic systems; aerospace flight control; biological injection systems; medical monitoring, control and/or alarm systems; power station and electrical power delivery and supply systems; vehicle systems and transportation control or any system where a pattern of inputs is analysed and a decision process is invoked to deliver a pattern of actions and outputs to an apparatus or system through mechanical, electrical or other means.

In another preferred embodiment, illustrated in Figure 3, processor means 24 is a general purpose processor and related circuitry 60, such as an Intel Pentium family processor; storage means 36 comprises a mass storage device 64, such as a Winchester-style disk drive, a removable media storage device 68, such as a 3.5 inch high density disk drive, and RAM and/or ROM memory 72 which is operably connected to processor 60; input means 28 comprises a keyboard 76 and/or pointing device such as a mouse 80; and output means 32 comprises a video display terminal 84, such as a SVGA graphic display, and/or a printer 88, such as a HP LaserJet IV. In some circumstances, it is contemplated that input means 28 and/or output means 32 may comprise a communications link 90, either in addition to or in place of the above-mentioned components, and communications link 90 may be in the form of a local or wide area network, a radio link, etc. which is also operably connected to processor 60. In the embodiment illustrated in Figure 3, system 20 can also embody a graphical user interface, provided by an operating system such as Microsoft Windows 95 operating system, executing on processor 60. It is further contemplated that multimedia input means comprising camera 91, sound recording microphone 92, and output means of graphic images displayed on at least one graphical display terminal 84 and sound from at least one speaker 93 connected to said general purpose processor 60, may be used to record and may be used to play images and sounds accordingly. It is also contemplated that input and output means may be provided by transmission signals 94, 97. Said transmission may consist of electromagnetic emissions such as visible light, radio frequency and infra-red, and may consist of sound waves. Said transmission may be passive from said monitor 84 and may be converted and stored into meta-data as provided by an instrument such as a C-Pen™ 98. It is further contemplated said input and output may be realised using a video camera 91 , or special purpose input/output device such as a transceiver 95 and device 96. In one contemplation of 96, said device may be an intelligent paper clip providing direction to at least one data source on a storage device. Said device 96 may also provide data to at least one data file located on a storage device. It is also contemplated said data and meta-data input and output means may be used over a computer network using said network connection interface 90 for providing for at least one output signal and at least one input signal with at least one other node on said computer network.

In either of the above-mentioned preferred embodiments, system 20 receives various inputs through input means 28, processor 60 acting on these inputs according to instructions stored in storage means 36 and providing one or more outputs, via output means 32, which outputs recommend and/or implement a desired selection between alternatives and/or the operating state of the process or apparatus under consideration.

The term decision factor is used herein to refer to a factor or criterion used as an element to construct a hierarchically related tree. Said tree contains a root decision factor that is the base of the tree from which all other decision factors are descendant decision factors. A parent decision factor has at least one descendant decision factor below it. A leaf decision factor has no descendant decision factor. A decision factor with a parent is called a child of he parent decision factor. Decision factors which directly share the same parent are called herein sibling decision factors. Said decision factor tree elements may also be referred to as nodes and as criteria, said meaning and all respective definitions within this paragraph remaining the same as decision factor.

The term "aggregate node" is used herein to determine a designated decision factor that has at least one descendant decision factor and where at least one attribute is determined from at least one directly descendant (child) node attribute to be aggregated in said aggregate node attribute. Said descendant decision factors and all said attributes of said descendant child factors are removed when an aggregate model is stored or the aggregate node is caused to aggregate. Said determined attributes from descendant decision factors are retained in the selected at least one parent aggregate node at least one aggregate attribute.

An aggregate parameter as used herein is a parameter used to determine at least one attribute or multiattribute value to a determined adequate degree of accuracy when aggregated data is removed during an aggregation process. Said at least one aggregation parameter provides means of representing lost attribute data, and may be regarded as providing means for a compression algorithm representing said missing data. In one embodiment, said at least one aggregate parameter may be assigned to an individual aggregate node following aggregation and removal of descendant nodes of said aggregate node. In another embodiment, said at least one aggregation parameter may be related to at least one alternative-specific attribute. Said aggregation parameter is useful in dynamic data changing where said aggregation parameters provide means to represent missing node attributes such as ratings and weights of said missing descendant nodes. In another embodiment, said aggregation parameter may determine a mathematical formula, set of rules relating attribute values, and at least one table of values. In another embodiment, said aggregation parameters may represent a set of coefficients to be used in a default mathematical equation determined as most likely to accurately represent missing attribute data in an aggregate model.

As used herein, a hidden aggregate node is a node that is present and may be displayed on an output device such a graphical display, and may provide visibility for selected data and meta-data from at least one descendant node, where said at least one descendant node is hidden (i.e. is not exposed and is invisible) on any display device such as a graphical display. Attempting to expand the tree below the hidden aggregate node cannot expose said descendant hidden nodes. The descendant nodes are physically present in the aggregate model. The hidden node may appear like an aggregate node and have meta-data fields containing aggregate data. A hidden descendant node may be used for purposes of accurate numeric computations and may be revealed to selectively show data and meta-data ^■ when at least one specific event occurs to expose the node. Said event may be the result of at least one manually inserted key, and may be by means of at least one manually input value through a graphical user interface, and may be from at least one input signal over a network. Said hidden descendant node may contain proprietary data and may further contain secret data where it is desired to have said data available for purposes of determining a value. By this means said proprietary and secret information is prevented from disclosure, while use is made of said data in value aggregation. A trimmed node is a parent node from which all descendant decision factors have been removed. A typical trimmed node becomes a leaf node on the tree of the output aggregate model whose aggregate rating for at least one alternative is determined from the normalised weights and ratings of descendant decision factors that are removed on trimming. In one embodiment said aggregate rating is the normalised weighted average determined from descendant node ratings and normalised weights. No other information may be presented or retained in a trimmed node.

A rating or score as used herein is a value assigned to a decision factor to represent the worth, impact, direct value, or relative amount of the factor for at least one alternative toward the decision. Said value may be a cardinal or an ordinal value, or a verbal expression identified with at least one numeric value. Generally, a rating is a value applied to leaf decision factor, and a score is the value determined from combining weight and rating value of at least one decision factor. In another embodiment, a rating value may be translated into a utility value by means of a utility function or rating method. A 'standard' score or rating may be an industry standard score or rating obtained from published industry standard target and requirement data. In another embodiment, said standard may be determined by the decision support system from data among all alternatives rated in the model. In another embodiment a sub-set of alternatives may be selected to provide a 'decision-specific standard'. If the alternatives are vendors, said sub-set standard values are referred to as vendor-specific standards. In one embodiment, said standard may be an average value. In another embodiment, said standard may be determined as for example a statistical mean or mode. The terms utility method and rating method as used herein define a computational means of transforming at least one raw input value for at least one criterion to at least one utility value, thereby providing a sense of worth of the at least one raw input value toward the goal of the current model on the basis of a common scale of utility. Said raw rating is applied for at least one alternative for which said criterion is applicable. The said raw rating value and utility value may further be translated into a verbal statement to indicate in more familiar terms the worthiness of the assigned value toward the model goal. As used herein, the term alternative represents a physical item, or may mean a choice between at least two actions to perform. Said definition is intended to include the case wherein no physical alternative or action is selected, and may represent for example at least one state of a system against which a performance measurement is made, and said selection of an alternative represents a means to measure a system's performance for said alternative. In one preferred embodiment, said alternative may represent a vendor product, and said vendor product may be considered composed of a set of at least two vendors component products, said at least two vendor component products being thereby evaluated individually and said vendor component products may then be combined accordingly to provide a composite solution, and said composite solution may incorporate at least one other vendor product or component product. In another embodiment, said alternative may be at least one system state, said at least one system state may be composed of a set of at least two subsystem states. The performance of a set of inputs may then be rated for each said system state to determine by numeric means the validity of said system state and said at least one subsystem state. The term decision factor attribute is used herein to designate a value or meta-data field that is directly related to at least one decision factor. An alternative-specific attribute is an attribute that is directly related to at least one alternative. In one embodiment said alternative specific attribute may be determined from a rating or score value of at least one decision factor in the hierarchical model tree. In another embodiment, said rating and weight attribute of a decision factor for at least one alternative may be determined from at least one alternative-specific attribute.

As used herein, a constraint is a value assigned to node to which at least one node attribute for at least one alternative is compared, the outcome of which generates a numeric output. In one embodiment said output may be in the form of meta-data which in of itself may be useful and may provide information to assist in determining the suitability of an alternative. In one preferred embodiment, said output may not of itself eliminate an alternative in a selection process, and is therefore a weak constraint in this sense. In another preferred embodiment, said output may become a strong constraint wherein in of itself said output may eliminate an alternative from the decision process. A typical strong constraint may be a mandatory requirement.

Meta-data as used herein refers to non-numeric information that is required to qualify rating, weight or other numeric data, and may provide other non-quantitative information useful in determining the suitability of an alternative in the context of a decision, and may contain instructive information on the usefulness of the model, and may further contain explanatory text, sound, graphic image and animated and moving images concerning a decision factor and attributes such as weight for the decision factor. Meta-data may also be applied to result information, alternative-specific attributes, charts and decision procedures, import/export procedures of data, and report templates and report generation. Meta-data may contain embedded links to other documents consisting of text and multimedia reference objects such as sound images and animation and movie files that may be located locally and at locations on a computer network, and may contain direct links to expert advisors over a telecommunications network which may be by means of electronic mail and may include direct interactive voice and image connections such as those provided by products like Microsoft NetMeeting™ and Whiteboard™ [ESR1] and may use direct voice and images through at least one microphone 92 and at least one local camera 91, and at least one speaker 93 connected to at least one general purpose processor 60.

An Analyst as used herein may be at least one expert knowledgeable in the art of building decision models, and may be knowledgeable in the areas of application of the model. A client is defined herein as one who may use an aggregate decision model in order to reach a decision, and may take input from the aggregate decision model to reach a decision independent, of or with assistance from one knowledgeable in making decisions in the field to which the aggregate model applies. Said client may be a decision-maker, and may be knowledgeable in the application area of the model.

As used herein, the term "to tailor" means the act of reducing and customizing the data set of a model through a process including node aggregation, elimination of criteria and alternatives and said attributes of said criteria and alternatives, causing to store the reduced model as a customised aggregate model. One preferred embodiment of tailoring is by intervention of one knowledgeable in the art of tailoring a model for specific applications. Another preferred embodiment is the process of reduction and aggregation by means of at least one signal representing an automated message which may cause at least one node to aggregate selected descendant data in the said model, and may cause at least one node to hide descendant children and their attributes, and at least one second signal that may cause at least one node to disaggregate, and may cause at least one hidden node to be made visible. Said at least one signal may also cause at least one attribute of at least one criterion to change, said attribute being, but not limited to, the weight of the criterion, score for an alternative, the indicated relevance of the criterion in the model, and the presence or absence, inclusion or exclusion of said criterion in a decision and control process.

As used herein, a decision model is comprised of a plurality of decision factors organized in a hierarchical fashion, and containing information including values and other decision factor related data therein, and a plurality of alternatives with associated data including rating values assigned to decision factors, weights and meta-data, model goal information and other related meta-data as deemed necessary to assist in making a decision by one knowledgeable in the art.

A chart is defined herein as a means of representing numeric data and meta-data in the form of a table or a graph or combination of both thereof. Typically, a chart contains numeric and text data of at least two attributes, said attributes consisting of at least one numeric value and may contain at least one meta-data datum attribute contained in the model. Said chart may further comprise of a table of rating and score values expressed as verbal or numeric data.

A chart object as used herein may be a bar or point or other symbol or shape representing at least one data value.

As used herein, the term 'dragging' refers to the action of focusing on at least one chart object presented on a graphic display with an input pointing device such as a mouse, selecting said at least one chart object and causing said at least one chart object to change in size, position, shape by moving said pointer. Generally, such dragging may cause on a time delayed basis re-determination of the at least one value represented by the at least one new size, position, shape of said at least chart one object. "Simultaneous dragging" refers to the act of selecting at least two chart objects such as bars or points representing at least one data value in the chart, and changing each independently prior to re-determination of at least one value represented by the said at least one change in position, size, shape of said simultaneously dragged objects. Such simultaneous dragging selections and changes may be performed by providing sufficient time to select a subsequent object prior to re-determination of said at least one value. In another embodiment, said chart objects may be pre-selected as a group and caused to change by dragging simultaneously. Multiple points on a chart may therefore be changed in one dragging operation.

As used herein a decision object may consist of at least one meta-data field containing at least one instruction on the use of said decision object, and may include at least one chart and at least one meta-data instruction on the use of said chart. In one aspect said decision object provides output means to instruct a client in the method of proceeding through a step in the decision process using said representation provided by a chart or at least one numeric value or meta-data value. In another aspect, the said decision object may include meta-data identifying purpose of said at least one chart, and may include at least one instruction on the use of said at least one chart.

As used herein a visual decision dictionary consists of at least one decision object and at least one associated meta-data field wherein is entered at least one instruction pertaining to the use of the said visual decision dictionary.

A decision procedure consists of at least one visual decision dictionary and at at least one associated meta-data field wherein is entered at least one instruction on the use of said decision procedure.

A decision process consists of at least one decision procedure and at least one metadata field wherein is entered at least one instruction pertaining to the use of said decision process. In one embodiment of said process, it is contemplated that a decision procedure, visual decision dictionary and decision object are identical. In the preferred embodiment, said decision procedure contains a plurality of visual decision dictionaries with meta-data guidance to guide in the use combined use of said plurality of visual decision dictionaries, wherein each said visual decision dictionary consists of a plurality of decision objects, said visual decision dictionary containing at least one meta-data instruction concerning use of said plurality of decision objects in combination. Said meta-data guidance can be considered in this embodiment as a set of scripts.

As used herein, a decision process consists of at least two steps taken in sequence, the said steps being determined by at least one Analyst. In another embodiment of said process, said steps may be determined by at least one other who may be knowledgeable in the art of making the said decision. A decision process guide is a guide wherein said decision steps are provided in an outline format, said each step being provided with means to execute said step and complete said step as best as may be possible, as may be determined by at least one expert in the step procedure. Said guide may provide guidance for the use of means to execute and proceed through said step.

As used herein the terms to store or output in reference to a data structure consists of the action of processor means in writing said model consisting of a determined data structure from temporary storage medium such as random access memory attached to a computer processor to a medium regarded as permanent storage such as a CD-ROM, Winchester style hard disk, programmable read only memory (PROM), removable flexible diskette, magnetic tape and any other medium deemed by one knowledgeable in the art as a permanent storage for said data structure. Said storage process in one preferred embodiment consists of means to encrypt said data structure on output by means of an encryption process such as PGP (a public domain encryption/decryption procedure) or any such decryption technique as may occur to one knowledgeable in the art When said encrypted data structure is input into the decision support system as disclosed herein, the decision support system may use PGP decryption techniques to decrypt said model structure. An Authorising site is an authorized entity able to service license key code requests from client sites, and that has access to information to validate the eligibility of a client to a license. In one embodiment, said entity may provide service through a phone-in help desk system. In another embodiment, the entity may be an automated system resident on a computer network such as the Internet. In another embodiment, said service may be on an internal corporate network. Said automated system may be able to review client license requests automatically, and transmit at least one message with a response in regard to said license request, where said response may be to provide said license key code, reject said license key code request, and send a request for additional and repeat information.

A Client is an entity serviced by an automated advisory service and may make use of an aggregate model. In one preferred embodiment, the entity may be a corporation, a government organisation, or at least one individual. In another preferred embodiment, the entity may be a machine consisting of machine readable code executed on at least one processor, said processor having storage means such as random access memory and read only memory, and at least one permanent storage device such as a computer readable disk. A licensee as used herein refers to a Client who has applied for and been granted a valid license by at least one authorising site.

In one preferred embodiment of the invention, a decision support system is a processor implemented code that, in accordance with Figure 1, the processor 20 reads the code from a storage device such as a Winchester type disk 64 or from a removable storage medium such as a 3.5" disk or CD ROM 68. An Analyst using said code causes said processor to read data representing at least one decision model from at least one storage device. The decision model may have been designed, assembled and populated with data and meta-data by at least one Analyst familiar with creating such models in the field to which said model applies .In one embodiment, said populating may be done using said same decision support system. At least one Analyst knowledgeable in the art of tailoring said models, and knowing the purpose to be accomplished by current said model, proceeds through the process 101 indicated in Figure 4. The Analyst selects a node using a pointing device such as the mouse 80 in Figure 3 or by keyboard entry 76 using arrow keys and a selection key on the keyboard, and indicates at least one criterion node at which aggregation is to take place such as Manufacturing 250 in Figure 7. In the current preferred embodiment, the Analyst may select functions from a menu and sub-menus 350 and 351 of Figure 9 actions to designate said selected node as an aggregate node or as a hidden node, or may trim the current node, its status so indicated by a mark or symbol 250 beside the designated node. In another preferred embodiment icons designating said aggregation functions may be present in for example, at least one Microsoft Windows standard toolbar for the model tailoring window. Descendant nodes whereof the data is to be aggregated may be indicated by other symbols 251. The Analyst may select at least one other node such as 'Supply chain management' 253 wherein at least one descendant node and said descendant node's data and meta-data may be hidden but retained in the aggregated model. The Analyst may further select for each aggregate node specific descendant node data to be aggregated as indicated in figure 10 401 and apply further conditions to further filter descendant node attributes to be in or out of the aggregation procedure 404 and 405. Such conditions may include the descendant factors with the best and worst scores for at least one alternative, the descendant factors which most deviate from the average score; determining the level below the aggregate node at which descendant factors are represented in the aggregate node 408. If descendant node level does not extend to the lowest descendant node, the meta-data from the lowest descendant nodes may not be aggregated included in said aggregation node. In the present preferred embodiment, only descendant node attributes directly descendant (the level below) from the aggregate node are aggregated as a default condition, all descendant data from the lower nodes being aggregated transparently (i.e. aggregated and removed on aggregation without choice of preservation of specific parameters for calculation and choice of aggregating meta-data information other than the, on one preferred embodiment, the weighted average and other parameters as required time-to-time by the decision support system). Aggregation may be selected to go to lower levels 408. The Analyst may select the aggregate node field 403 into which meta-data and summary numeric data are placed for client review. The Analyst may select any of the available data and meta-data fields 401 for aggregation into at least one of the aggregate node fields selected in 403, and include specific items such as multimedia files and direct ('hot') links to experts via E-mail or dial-up line 410, and may include a specific tutorial related to the node. A restriction of selecting only one aggregate field for each sub-factor meta-data field may be imposed as a preferred embodiment. In addition, the Analyst may select how the scores and weights may be represented in the aggregate field by selecting specific rating and weight interpretation methods 405, 406. For example, a score of '90' may be translated into the verbal score of 'Excellent', and a score of '30' may indicate a 'Poor' score. A weight which is over 5% may be regarded as 'very important', while a weight of 0.2% may be considered 'Unimportant'; the most important factor may be identified by the verbal statement 'Most Important', and the least by 'Least Important', and so on. Said rating and weight interpretation methods may be created and edited as part of the customisation process 407. The verbal scores and weights may be combined into sentences such as indicated in 256. Numeric values may also be included 409 to show at least one score and at least one weight, and at least one verbal statement to indicate the sense of the value 255. A sentence in a selected aggregated meta-data field 403 may then read

"Production Planning is the most important factor [2.1% of total decision] in Manufacturing and scored poor [ 30 out of 100] for Barndon Plus Corp.'

Score information may be written, for example, a typical output may read: 'Manufacturing scored Average for alternative Softwares Inc. [53.8], Good for Barndon Plus Corp. [68.2] and Poor for Pertall Office Suite [42.4] and carries a weight of 5.75% toward the decision. For further source information contact Harry.Biggs@arlingsoft.com or at URL www.manuf.com/~Biggs or call 999-123-4567'

'Design and Engineering is the least important factor and scored Excellent for alternative Softwares Inc. [93.8], Fair for Barndon Plus Corp. [62.3] and Very Good for Pertall Office Suite [82.4] and carries a weight of 0.75% toward the decision.' The major three weaknesses of Softwares are in order of importance:

Master Production Planning - Poor [48.4] moderate weight [0.75%] Inventory Management - Very Poor [32.3] average weight [0.58%] Production Configuration Management - Average [51.9] weight [0.55%] The major three strengths of SofWares Inc. are in order of importance:

Computer Aided Manufacturing - Very Good [78.9] high weight [0.82%] Computer Aided Design - Excellent [87.9] moderate weight [0.70%] Product Formulation - Good [65.4], low weight [0.35%]

For specific information consult document [X]' [X] may be an embedded icon in the document and whereupon pressing said icon results in the document being opened by the appropriate application assigned to the file type such as a file with extension '.doc' may be opened Microsoft Word. Said document may be a text word processing document, and may contain sound and images for further explanation and additional contact information. Such information is of use to clients that may need summary information prepared in the form of a report 803 in Figure 13 for presentation purposes to senior officials or others responsible for such decisions at the client site. In another embodiment, a Request for Proposal (RFP) 804 may be generated by the Analyst for the client following aggregation. Said RFP may be generated from the aggregate model. Question and Instruction meta-data fields such as 'Score Question' 300 in the descendant factors and aggregation node meta-data fields may assist in generating questionnaires. Said reports may be generated by the decision support system as a meta-data output, and may include charts generated by the client. Reports may be generated and sent to an output device such as a printer 88, or encapsulated in an electronic message and sent by electronic mail over a communications network using a network interface 90.

In some instances, at least one minimum requirement may be established from at least one constraint. In this case, at least one exception report may be generated for at least one alternative in the aggregated data and said at least one exception report submitted to the at least one client. Such information may be selected 40 land assigned to the selected aggregate meta-data fields 403, and may be generated automatically by the decision support system for automatic presentation to the Analyst. The Analyst may then generate exception reports and may submit said at least one report to the at least one client.

In some circumstances, the Analyst may recognise an alternative may not have at least one feature, said at least one feature being represented by at least one decision factor, and these at least one feature may be compensated by at least one other related decision factor, and it may be desirable to avoid reducing the value of the alternative for a particular category of criteria containing the at least one decision factor representing the at least one missing feature. The Analyst may then mark the at least one criterion as 'Not Applicable', causing weight of said at least one criterion representing said at least one absent feature to be distributed to sibling and related criteria in the model parent category that compensate for the absent feature, thereby mitigating the undesirable result of reducing the value of the category containing said at least one missing feature for said alternative.

On completing the aggregation selection and configuration tasks the Analyst may store the aggregate model to a storage medium such as a Winchester drive 64 or removable medium such as a CD-ROM or removable disk 68. The model may also be exported to a spreadsheet or other formatted document 72, and may be encapsulated and transmitted in an electronic mail message.

In one preferred embodiment, said nodes designated as aggregate nodes in figure 4 104 may be caused to aggregate immediately 121, either before parameters are determined from 106, or after said parameters are determined from 107.

In the current preferred embodiment, storing the aggregated model 110 causes said selected descendant aggregated nodes to aggregate data according to the selection choices made by the Analyst prior to assigning a unique model identification code 109, and storing the aggregate model 122.

In a preferred embodiment, the Analyst may select to cause the aggregation process to occur on designating the currently selected node as an aggregate node 103, using default aggregation settings.

The Analyst may be unable to store the aggregated model under the same name as the originally opened model 110, and may be forced to save the aggregate model at a different location. Said model may be given a different unique code 109, and said unique code may provide indication of the parent model from which the model was tailored. Aggregation parameters may be required for estimating at least one multiattribute value used to assess the rank and value of at least one alternative. Said aggregation parameters may be used in certain decision procedures and may be used to generate different scenarios that may include changes to at least one weight and at least one score, yet such weight and score and other data may be removed through the aggregation process. Said parameters provide means to approximate values enabling calculation of at least one required multiattribute value. A multiattribute value may be any value incorporating a plurality of other values and used to determine the rank ordering of a plurality of alternatives. In the current preferred embodiment, said multiattribute values may include standard deviation, Weighted Average, Matching Index, Weighted Average Composite Index, and Percent Match as calculated in Arlington Software's product ERGO. In another preferred embodiment, other common multiattribute values may be calculated as required from time-to-time, and may include cost and risk values that may be related to at least one criterion attribute and at least one alternative attribute.

In one preferred embodiment, the executing machine readable code causes said processor to process said tailored model generated by the Analyst, but not yet aggregated, and causes said process to determine and store said aggregate parameters 107 for at least one alternative. In a second preferred embodiment, said general purpose processor may not be adequate, and it may be determined that at least one other processor executing machine readable code to read the Analyst tailored and not yet aggregated model data, thereby causing said other processor to generate at least one aggregation parameter 114. Said at least one aggregation parameter may then be stored with the aggregated model 108. In a third embodiment of the aggregation parameter process, the Analyst may determine that the aggregation parameters may not need to be processed, and thereby saves the aggregate model without determining said aggregation parameters 110.

In another aspect of the invention, the said at least one Analyst may determine relevant additional alternative-specific attributes (as in figure 19 and 20) of the Alternatives 1300, 1400 that are separate from decision factor attributes. Said alternative-specific attributes may be inherited in a hierarchical manner as indicated by 1302 and 1303 in figure 19. Attributes may be assigned a name 1401 and may be assigned meta-data information 1402. Other data may be given and may include at least one value 1403 selected from a list of at least two values 1404. An indication that the attribute may contain numeric data may be given 1405. In another embodiment, said numeric value may be related through an equation 1406 to other alternative attributes 1410 and may be related to at least one factor attribute, said at least one factor being selected from the model tree 1412. At least one factor attribute 1413 may be selected for inclusion in the equation 1409. Various mathematical functions may be selected 1411 for inclusion in the equation 1409. Said at least one factor and at least one factor attribute and said alternative and at least one alternative-specific attribute may be identified internally in the model data structure through an identification code. Machine readable code of the decision support system may then be executed to read and interpret said identification codes and mathematical functions, thereby causing said processor to determine the value outcome of the alternative attribute. In another embodiment, said value list 1404 may contain text and meta-data. In yet another embodiment, the default value may be selected from the list of values, and assigned to the attribute of that alternative if no other value is selected for the attribute of the alternative. In another embodiment the at least one assigned value and at least one calculated value may be assigned to all alternatives 1407. Said alternative attribute may be processed according to selected means 1414. Said values may be displayed as summary information for the alternative 1501, and text and meta-data information about a highlighted alternative attribute may be indicated in the same window 1502. Further, said at least one alternative attribute may then be used to sort and filter at least two alternatives according to mathematical or textual relationships 1504 and rules 1505, said rules combining said at least two alternative attributes into a common means to sort and filter at least said two alternatives.

In a preferred embodiment, a decision support system is provided to the client wherein said decision support system may have at least one feature rendered ineffective. In another embodiment said feature is removed from said decision support system. Said decision support system is referred to as a client decision support system. In one embodiment, said means to determine key codes and sub-keys may be removed. In another embodiment, means to determine aggregate state of at least one decision factor may be rendered ineffective or may be removed. Other feature limitations may be determined according to required product delivery functionality and may be determined by said at least one Analyst prior to delivery to the client. Said client decision support system may further be enabled to input only one determined at least one aggregate customised model In a preferred embodiment, said client decision support system may be determined as providing a limited set of features that have been determined as a common requirement by most clients, enabling a standard client decision support system to be provided.

In another preferred embodiment, the Analyst may determine prior to final storing and providing the aggregate model to the client, determine a security code key 120 for said model. Said security code key may include sub-keys to determine additional restrictions on distribution and availability of said aggregate model, said conditions and limitations being determined by said client decision support system on input of said aggregate model. In one preferred embodiment said security code key and sub-keys may determine the number of licensees, and may cause limits on the number of simultaneous readable copies that can be distributed at the client site over a computer network. In another embodiment, said key codes and sub-keys may limit the number of concurrent accesses to a single model, the model being located on a single storage device. In another embodiment, an additional sub-key may also be provided to limit the number of storage devices on which the model may reside at a client site, thereby allowing multiple concurrent copies to exist. In another embodiment, said code keys and sub-keys may enable only named users to input said aggregate model. Said key and sub-keys restrictions may be activated following delivery of said model, and by an additional process initiated between the client decision support system at the client site 900 and an authorising site 920, following initial reading of said model by the client decision support system. Once the authorisation codes have been input, the model may be used according to the conditions in the license as determined by the code key and sub-keys.

Processes of model customization may be by means of at least one facilitation process with at least one decision-maker 111. In another preferred embodiment of the process, specifications 112 from at least one expert in the field may be provided to at least one Analyst, said specifications providing the basis for the said at least one Analyst to tailor the decision model. One such embodiment of an expert may be a consultant in the service of the client.

In another aspect of the process, the Analyst or one Expert in the field of a specific aggregate model may execute the machine readable code and select at least one aggregate model. The Analyst or Expert may then select at least one chart from a set of charts 150 (Figure 5) or determine at least one chart therein. In combination with each chart an appropriate question may be formulated to ascertain whether a client may require the chart as an attribute of the chart 155, and explanatory and instructional text, meta-data and links may be entered and edited 156 to correspond to the said selected chart, thereby customising the meta-data for the client, and creating a decision object for said client. Multimedia embedded documents and animated figures may be included in the meta-data makeup 165 to provide additional information and clarification of the use of the chart. The meta-data may be used to assist said Analyst and Expert and client so that said Analyst/Expert/client may effectively utilize the chart to assist in the decision. Said meta-data can also serve in training sales staff and future experts and analysts, and provide means to effectively communicate the decision process to clients. Each decision object resulting from said chart and meta-data instructions thereby renders at least one complete decision aspect as relevant as possible toward the decision for the said client. Sometimes a single decision object is insufficient in dealing with all aspects of a decision. A visual decision dictionary may then be used where said visual decision dictionary consists of at least one decision object used respond to one aspect of the decision 158, and may have its own meta-data associated with it 159. Typically a visual decision dictionary may incorporate at least two charts in combination. An example of a visual decision dictionary is the sensitivity analysis 1200 whereby the sensitivity of a particular alternative to its ratings and weights may demand the simultaneous presence of three charts: one representing alternative ratings such as Option Rating Bars 1210 in Figure 18, one representing the model criteria weights such as the benchmark chart 1211, and one representing the final evaluation of the alternatives such as the score breakdown 1212. Preferred embodiments of said charts are further illustrated in Figure 16 as charts 501, 502, and 507 respectively. The sensitivity visual decision dictionary may be used to observe the effect of changing the weight and rating individually or together, to observe the outcome in terms of rank. An embodiment of said process is indicated in figure 11 whereby said at least one rating value may be changed by dragging 606, and at least one weight may be changed by dragging 603, and said effect noted in the score breakdown graph 609, representing a summary of the value outcome on said alternatives resulting from said changes of ratings and/or weights. Meta-data for the visual decision dictionary may be used for example to explain the implications of changing the rating of certain alternatives (for example, if the alternative is a vendor, in six months the vendor may have added new features and improved current features to its product, and hence the rating improvement impact may be assessed), and weight change implications of certain criteria may be explained from expert opinion, thereby making sensitivity analysis meaningful and realistic. Another such visual decision dictionary may be to observe the change in cost-effectiveness and rank of an alternative as rating and cost information are changed. A combination of visual decision dictionaries may be provided to determine a decision procedure 167 wherein different aspects of the decision - in this case sensitivity outcome and cost - may be examined, and the combination provided with meta-data instructions and guidance in use of said combination. In figure 18, for example the decision objects illustrated by 1210-1212 may be replaced by visual decision dictionary titles, and associated meta-data fields may then refer to the decision procedure and included visual decision dictionaries. Decision procedures may further be stored in a structured knowledge database 162 for later retrieval 161, 164 and incorporated into other customised decision models where they be edited and customised for the another client. Additional constraint data 163 may also be included and may be incorporated as separate object data sets 163, or may be bound with each decision object 165 in the knowledge base. The visual decision dictionary may be stored with the model 160.

In another aspect of the present invention, each dictionary may be accompanied by at least one equation 157 determined by the Analyst or one skilled in the art. Said at least one equation may use the at least one said numeric datum available in the accompanying chart and at least one result of the at least one said equation used to assist in determining at least one negotiation point for at least one alternative. Said at least one equation may use at least one constraint 165 applicable to the at least one chart composing the decision object.

In another embodiment of the present invention, the visual decision dictionary may be selected from a list of visual decision dictionaries in the knowledge base 161. Said selection may be assisted by at least one question requiring at least one response by one skilled in the art in order to select at least one visual decision dictionary from the knowledge database. Said visual decision dictionary knowledge base may also contain at least one constraint datum 164 for at least one chart contained in a visual decision dictionary. The at least one constraint datum may be determined from proprietary or confidential data received through interview processes with clients and vendors, and from publicly available material.

In another aspect of the art, the decision support system may provide the Analyst means to create a flow-through process for the client, said flow-through process being indicated by a set of instructions. In another aspect of the present invention, an Analyst or one knowledgeable in the art may have provided code keys and sub-keys as part of a license condition, said keys and sub- keys being stored with the aggregate model. On inputting the aggregate model, said client decision support system is caused to determine the license validation state of said model. If the client decision support system determines that the license requires a second key to continue inputting said model 901, said client decision support system issues a request for a second key, said request may be output to a graphical display. In another embodiment, said request may be transmitted by electronic mail, or may prompt for personal communication between requesting site 900 and authorizing site 920. Upon receipt of the request for authorization 921, the authorizing site verifies the request is eligible for license 922. If the requesting site is eligible, the authorizing site issues an appropriate at least one license code 925. Alternatively, it may provide a message indicating the request for license code is refused 924. Said information may be transmitted verbally or by an automated response system such as electronic mail 926, or automated signal over a computer network to said client decision support system. In one embodiment of the said process, receipt of an electronic message containing said at least one license code causes machine readable code at the authorizing site to transmit message to the said machine readable code currently executing on said processor at the client site, the said at least one license code may then be entered by automatic process through machine readable code at the client site. In another embodiment of the process, a licensee may enter the license code manually. In all cases, on acceptance of the said at least one license key, said client decision support system verifies said at least one second key is a valid key, prior to continuing input. Said second validation key is then stored 906 where in one embodiment said stored license may be with said model. In another embodiment, said validation key is stored in a location known to the client decision support system. In either case said model is licensed for use. If the at least one license key code is not valid, or no license key code is provided, the model is not read and the said model file is closed 907. If the license key code is valid, the model file data is read by the client decision support system 908. In another aspect of the invention, the license period is verified 904 for the said model, and if said license period is exceeded, said machine readable code may prompt the licensee to update the license 903, whereupon the process of authorization 921 through 926, and validation 905 would again take place. The status of the license may, in another embodiment, be indicated prior to license expiration, thereby warning the licensee of the time left before expiration of said model usage. In another aspect, said license verification process may determine that the license is violated during an attempt to input said model by a client decision support system. In this case, input is denied for the at least one client decision support system.

In another aspect of the invention, specific features of the model may be made available for limited time periods. At least one sub-key may be issued to activate at least one the feature of the model. For example, hidden aggregated data may be made available for a limited period, or the ability to use at least one report template and at least one specific visual decision dictionary. At least one sub-key may be issued to de-activate said features, and may be transmitted automatically over a computer network. Control of distribution sensitive information can be provided through this methodology by at least one central administrative center with means to provide said service. Such control may be provided as a service in the automated advisory services and decision and control products. In another aspect of the invention, machine readable code may be executed to transfer the model license as in figure 15 from the current processor on which the license resides 1001, to another processor 1007. The process may involve the following steps:

1. Transfer out of the current model license to an intermediate storage medium 1002 such as a disk file location on a network, or removable medium: the model may then be copied from the current processor, and the license code for the model marked invalid 1005. In another embodiment of the process, the model file may be deleted from the current processor 1006. In another embodiment of the process, only the license information may be transferred, the act of transferring said license to the intermediate medium then being followed by invalidation of the model on the first processor 1005. In yet another embodiment, the license may be transferred directly to the second machine over a computer network 1003, and said model may remain resident on a network server with a shared storage device 1020.

2. Transfer in of the license for the model to the second machine, wherein said license of the model may be copied. In one embodiment of the process the license may be copied into a copy of the said same model on the second machine which is not currently valid, said model having the name and content identical to the model on the first machine. In another embodiment, the said model file is directly copied to the new machine, and license information is modified appropriately on the said second machine to reflect the validity of the license on the said second machine.

3. The limiting on the number of licenses issued to a client is maintained by the transfer-out-transfer-in process by adding a restriction. In multiple use licenses, only the authorized administrator can initially provide licenses up to the limit set by the license agreement. In this procedure, a set number of license keys are provided and given one at a time by the administrator up to the limit allowed by the license. From then on, only this number of licenses can be transferred through the transfer in-transfer out process 1000.

4. Further restrictions may be applied to prevent the transfer process to unauthorized processors by requesting prior to model release the identification numbers of the computer processor whereon at least one client decision support is executed. Thence such transfers can be limited to specific machines, said processor identification number being encrypted into the said model file. In a second preferred embodiment, the processor identification number is read automatically on first installation of the model by the client decision support system executed on the specific processor authorized by the administrator, and may not be read by another client decision support system executing on a processor with an identification number not corresponding to the authorized processor identification number.

In another aspect of the process, the said client decision support system consisting of machine readable code is executed at a client site, causing said code to read at least one aggregate model. In one aspect of the current invention, the licensee may change at least one weight, and may change at least one rating, and may change at least one meta-data field. The change of at least one datum may cause said at least one processor to determine a new outcome, and output said data to a display device such as graphical display monitor. In so doing, aggregate values are utilised in the process.

In another aspect of the invention, a licensee may be provided with at least one Decision Procedure in the form of at least one decision object contained in at least one Visual Decision Dictionary (VDD) that may be contained in a decision procedure 181 by the said at least one Analyst with said aggregate model 182. Through a process of question and answer, which may include guided instruction in text and other meta-data formats within the said

Analyst provided decision procedure, as well as by direct links provided in the help system to at least one Analyst, said links consisting of electronic mail services and may further include interactive assistance using such methods provided by for example Microsoft NetMeeting™ over a computer network or similar, the licensee may select at least one decision object in the decision procedure and display the at least one decision object on a graphical display. Said client may then proceed to analyse the information contained in the model with said guidance of the meta-data, and may add additional information such as constraint data and meta-data comments and embedded data links. In another embodiment of the invention, the licensee may select at least one visual decision dictionary by pressing, for example, a tool button 600 in the window provided. Data from the visual decision dictionary may then be included in at least one report template, and the licensee may then select at least one alternative amongst the at least two alternatives for further examination and report generation.

In another aspect of the invention, separate at least one constraint information may be determined to characterise the requirements of a client. Said constraint information may be entered manually by means of a keyboard 76, thereby modifying or adding at least one constraint data item in the said provided model (183 and 185). In another embodiment, constraint information may come from a client database, data warehouse or other source available to the client 184. Constraint data from all sources may then be combined 187. Said constraint information may be, for example, the requirement that a product feature includes the ability to read files of a specific format. Said feature may be present to a limited extent - e.g. the file read supports only a version of the file format that is lower than that of the version used at the client site, thereby reducing the attractiveness of said item, and may cause a need for in-house customisation, and said customisation may cause increased cost, and said cost resulting in a negotiation requirement with the vendor (the alternative). In this way, such formulae as entered into the decision model by the Analyst in step 157 that may relate the rating value of the alternative to added cost can be customised by the Analyst for the client, and evaluated and noted as a negotiation point in steps 182 through 190. In another aspect of the invention, at least one decision object may be used to provide information of total cost of ownership. As indicated previously, if a value does not meet a specific constraint condition, cost of ownership may increase. In Figure 16, for example, chart 504 represents a quadrant analysis graph with weight of at least one decision factor plotted on the y-axis, and rating value of said at least one decision factor on the x-axis. Decision factor values on the left of the cross-hairs 506 in figure 16 are important decision factors judged as poor performers, where poor performance indicates a need for additional customisation costs of a vendor's selection. The related costs of the poor performance may be aggregated into a net cost, and added to the total cost of the alternative. Such costs may then be used for negotiation purposes. The formulae to determine said costs may be embedded with the attributes of the decision factor as part of the model data structure. Said formulae and associated values may be inherited by charts, or assigned and used in estimating costs and other values in a chart, as executed in step 157. Said assignment may be done by the Analyst, and may later be modified by the client using the client decision support system. Cost data may be determined from other clients, and vendor projects, and may be determined from similar experience at the client site.

In another aspect of the invention, the licensee may determine the need to operate at least one decision object in a dynamic mode to explore various scenarios of attribute values. In the current preferred embodiment, when the licensee selects this mode of operation, the data for said at least decision object is transferred to temporary storage. Said temporary storage may be a computer storage device such as a disk data structure, or it may be in the form of random access memory. In either case, the data is maintained separate and apart from that in the said provided model data. In dynamic mode, at least one data point in at least one decision object chart representing at least one variable in the decision may be changed by use of a pointing device and dragging at least one chart object to a new position on the chart, or entering new values in a table.

Figure 11 demonstrates at least one example of said dynamic mode. In figure 11 rating values for a plurality of alternatives and decision factors are represented by the chart bar objects 606, and said chart bars may be adjusted 610, and the effect observed as a change in position indicating a change in the rating, and said change may be transferred to another chart such as the quadrant chart where a new value is determined by a change in position 608. Similarly, the weight and rating of at least one decision factor may be simultaneously changed, and the change reflected in the length of the y- value bars 607 and 608. Changing both weights and ratings of a decision factor can be accomplished by moving points in the quadrant chart 608, thereby simultaneously changing the ratings of at least one alternative, and the weight of at least one decision factor. Other parameters such as costs may changed as in the Cost/Weighted Average chart 605 by means of moving the data point chart objects. The calculation update may be performed 'on-the-fly'. In another embodiment, the updates may be selectively applied to each chart. Said changes may then result in at least one what-if scenario to be established. Each new scenario may then be saved using a toolbar tool 602, and retrieved using a second toolbar button 601. In another embodiment, menu items may be added in for example the File menu item, as is standard practice in Microsoft Windows operating system applications. In a third embodiment, said scenarios may be stored in the same model structure, and presented by means of tabs, and special charts wherein said scenarios may be displayed at the same time, and may be compared. Said comparison process may follow a scripted procedure as determined by the Analyst or one knowledgeable in the art, and may provide grounds for negotiation input by, for example, increasing the rating of one feature such as Product Functionality to indicate at least one improvement of the vendor product feature set. Said required improvement may then lead to specific analysis of product features that are lacking, as in the absence of support for a particular operating system. Said vendor may then be required to reduce the price, or provide at least limited support for the operating system, the details of which may be provided by the at least one Analyst using the Analyst's more detailed model and information content. Said information content may for example indicate the cost to the vendor in upgrading said product to provide the support, and may include the anticipated cost to the client due to its absence. Either way, said point is available for negotiation, and negotiations may thereby be assisted by the at least one Analyst. In another embodiment, the value outcome of the said changes in ratings and weights may be detailed in table format, as indicated in 611. In this embodiment, values may be entered into the cells and may then change said values as though the points had been moved on the chart, and may cause said chart objects to change accordingly.

In another aspect to the invention, in order to determine the composite index, a proprietary algorithm developed by Arlington Software Corporation for its product ERGO, a special data aggregation technique is required. One component of the equation relies on the generation of summed absolute differences between the expected (desired) contribution of a criterion to the decision, and the actual contribution the item provided for each alternative score, for each alternative. The aggregate node may contain a linear sum of prior values, but in the case of the Composite Index, the final aggregation value cannot be known if any weight or rating is changed in the system. This is because the contribution of an aggregated criterion is based on the original weighted average of the original aggregated model supplied by the Analyst, not the new weighted average value arising from changes in the data. In order to understand this, we consider the determination of the Composite index.

The weighted average is a linear transform and it is easily aggregated. The Composite Index is different. For aggregation of the descendant data of node L from descendant nodes J for an alternative T we may write the composite index in the form

where w are the global weights normalized to node J, S_JT is the rating for node J and alternative T, and V r is the weighted average for alternative T which is the sum of the product of each weight and rating for each descendant leaf node and wherein the leaf node weights are normalized and add to 100% at the root node, that is, for _}τ leaf criteria the weights rfor alternative T are:

NJT

∑^r = l 2 =1

In the composite index equation, the summations can be divided between the contributions from the descendant nodes under the aggregate node and all other nodes. Equation 1 can then be rewritten in the form of the following parameters 1 - ^e T ^{+ e}X + h_Lr + h_x

CI_T 5F mm .'w- min 3: )

where the items with subscripts LT represent value contributions from descendant decision factors under an aggregate node L, and those with the subscript X are the value contributions from all other parts of the model besides L. The components for ERare determined by:

Vτ = ∑ w_JTS_JT 4 i

, 2 $LT

'LT W

VJcL ^JT I vΓ T ^wmin;LT = ^mln[ k^wmmiinn;;₇Tr;Y/.| .6

SLτ = ∑i 'JT .1

Wei

'LT ^VJT 1 - ^}JT

J zL V_τ where we consider all determined J descendant nodes to be aggregated as being under parent aggregate node L.

If a weight or rating elsewhere in the model is changed, said change is reflected in the weighted average of equation 4. It is seen that equations 5 and 8 only depend on the weighted average. In the case of 5, said transformation for the contribution is easily obtained. If V'_r is the new weighted average, and if the former weight of node L was W^rand the new weight is W'ιτ thorn equation 5 transforms as:

In equation 9 it is clear that the values outside the summation are all known. If b^on the right of the equation is retained as a parameter, then it is a simple matter to determine h '_LT . Similarly, the two other parameters in equations 6 and 7 can be preserved as aggregate node parameters and transform as simply as in equation 9. However, equation 8 presents a problem due to the absolute value calculation in the summation. It can be shown that an approximation to calculate e'z,r is given by

NL ^e τ ^≤ ∑^aLT^WJT \^cLτ\ + ^eLT 10

JeL where

JT VJT a JT N, V LT . 11

2 _Λvjτ

JeL

However, it is determined that this approximation yields generally errors of order 10% or worse, depending on the distribution of weights and ratings distributions of the descendant decision factors. This inaccuracy is unacceptable. In order to mitigate this problem a number of procedures are possible. In the present embodiment, said procedures comprising:

1. Data compression techniques in which the score data of said aggregated nodes are summarized in a set of parameters indicating the distribution of said rating values of the aggregated node. Such compression techniques seek to solve the following constraint optimization problem:

Given a discrete

where x, is a variable (a weighted score contribution in our current embodiment, but it may also include a weight or a cost distribution for example) over the range [x_mm, x_maxJ, then find a function g[f(x)J, such that for a given value of x, the value z(x) is given by

^Z(*l)= Σ *.•-?!- [/^■(*- )] n=0 and given that the exact solution is

then optimize g_n

)] subject to minimum(|y - z|)

In the current preferred embodiment, the variable x, is the weighted average Vγ of an alternative T. Typically, the range of Vγ is [0,100]. The function of concern is the weighted absolute sum difference of contributions, that is

The function g_n[f(Vτ)] is generated by evaluating f(V_T) at discrete intervals of Vγ. over the range of Vj. This provides a spectral graph 1100 as indicated in figure 17. In one preferred embodiment the expression for g_n[f(Vτ)] in calculating z(V ) 1102 may be

and one may provide means through an approximation method on estimating coefficients a„ through optimization and approximation methods as may be applied and may be available in the technical literature, and may be specially developed for certain classes of score distributions. Said coefficients are then stored as aggregate parameters for the aggregate node.

In some instances, if said rating value of the aggregate node is changed, said at least one rating value of the at least one descendant node must be considered modified. For a realistic view of the rating changes for an alternative, where for example the alternative is a vendor, said rating changes may be provided with a rule base to determine which of said descendant decision ratings may be changed for said alternative. For example, if a vendor is focusing on developing Computer Aided Design 306 in figure 8 rather than supporting Engineering Change Management 305, and said decision factors are under aggregate node as indicated, an aggregation rule would cause an increase in Computer Aided Design 306 rating prior to Engineering Change Management 305 rating, until said Computer Aided Design 306 rating reaches the maximum rating for said nodes, when the rating of Engineering Change

Management may then be changed. The process of rating change is subject to an optimization process wherein said rules may be applied and the value of equation 14 determined for determined values of the rating. Thence a function of the illustrative form in figure 17 is generated and a series of parameters for each new rating value determined . In this case, several sets of a„ coefficients may be generated, each set representing a specific rating, weight and weighted average scenarios for each alternative. In another embodiment, said at least one rating may determine at least one alternative attribute. In said case, determination of the alternative attribute value may further be represented as a function of the score value of the aggregate node. In this procedure the rating values and weights no longer appear in g_nLΥ(Vτ)], and the equation is purely dependent on the value of JN. The at least one set of coefficients a„ 1103 are stored for at least one alternative, and generally are specific for each alternative, and assigned to said aggregate node. It is normally the experience that the number of coefficients will be much fewer than storing the number of raw scores and weights, and hence the data compression is achieved. Further it is anticipated that calculation times may be greatly reduced, and further may significantly reduce demands on system resources, thereby further assisting in reducing the calculation time. Further, said compression may mean that smaller machines than otherwise may be able to perform calculations with sufficient accuracy without the requirement of the full data set. Figure 17 illustrates means of comparison of z(Vτ) with the exact solution. In the current embodiment, f(Vτ) = eoτ.(Vτ) where L=0 and where 0 may indicate the root of the tree.

In the current preferred embodiment, the set of coefficients for each alternative may be stored as attributes for each aggregation node. The said coefficients may be generated by the Analyst prior to data aggregation and saving the model in at least one of the steps 106, 107, 114. In the current preferred embodiment, the requirement for generation of at least one aggregation parameter may be determined by the Analyst

The selection and accuracy of the functions gn[f(Vτ)J may depend on the nature of the distribution function f(Vτ). Consequently, it is anticipated that at least one such function may be required, where the choice of said at least one function may depend on nature of the function/( r). In another preferred embodiment, said parameter Vγ may represent cost, benefit, risk and other values significant to the alternatives, and may further represent combinations of said other values. Summary scores may also be displayed in separate tables, as indicated by 604, and said summary scores may be dependent upon the identified node in the hierarchical tree 604. 2. Ignore the pattern of data in the descendant nodes, and use only the aggregate scores as if they were trimmed nodes. Parameters to enable determination of the Weighted Average Composite Index (as in equations 5-8), standard deviation, and Percent Match may be calculated, but cannot be used in dynamic mode, the aggregate node is treated as if it were a leaf node in dynamic mode. 3. Providing means to approximate changes in the distribution of contributions from the aggregate nodes by modifying the transformations required to aggregate the data. In this method, said transformations 5-8 may be modified and approximations sought by means of pre-changing the complete Analyst model prior to aggregation. Said resulting output of values is modeled by means of statistical distributions. However, said method is dependent on knowing which decision factors and which decision factor attributes are likely to be changed. In one embodiment, a table of values may be stored with the aggregate model as a result of said pre-aggregated generated scenarios. In another embodiment, at least one statistical distribution may be determined such as a Weibull distribution for the ratings and weight distributions, score distributions and final outcomes. This method is complex and not well understood because of the variety of possible distributions, and is not the preferred embodiment.

In all cases wherein aggregate parameters are determined, the impact on alternative attributes may further be assessed. Total cost of ownership of a vendor product, for example, may depend significantly on the development direction the vendor is taking for its products. Hence in addition to aggregate parameters for decision factors, aggregate parameters for alternative attributes may need to be determined, as by means of one of the three embodiments herein disclosed. As an example of an application of the automated advisory process making use of the model building, tailoring and advisory processes disclosed herein, Client X seeks to purchase an Enterprise Resource Planning (ERP) suite of products.

Many vendors offer ERP suites, and the complexity of the suites is large because of the many facets that must be understood by Client X. For example, said ERP system must support interfaces to a wide range of corporate databases, and be linked to the databases through customized interfaces. Said ERP must also be web enabled, and be able to provide reports for web display, for the sales force, for administrative personnel, and for reports for senior management. The ERP must also be capable in providing up-to-the-minute inventory information, maintain lists of clients and integrate sales inquiries, ordering systems, supply requests, purchase orders etc. Further, Help desk information is to be tied to product types, help desk personnel performance measures, and performance of sales personnel.

Many aspects of the ERP system are vendor specific, with some vendors being strong in some of these areas, and weak in others. Weak areas provide challenges to Client X, and may cause additional costs to Client X. Moreover, the implementation process is complex, and the ability of each vendor to deliver such an ERP system must be assessed. Further, the stability and likely future of said ERP vendor, and the future direction of the ERP vendor development, need to be taken into account. Client X may not have the resources or expertise for said research, or the time and reliable, systematic means to do so. An automated advisory service (AAS) is approached, said AAS being a company specializing in independent research on ERP systems. The AAS may have developed a model and database that may contain significant information available from public sources, private client installations for at least two vendors, and at least one vendor. The at least two vendors are independently assessed for their financial stability and likely future by at least one Analyst familiar with the field. Said at least one Analyst may have constructed the said multicriteria model for an ERP procurement decision process using the decision support system herein disclosed, said model containing all possible features of the said at least two vendors' ranges of product. Said model may have accumulated several thousand said criteria representing different aspects of the decision and may further contain privileged information concerning the vendor, and may include product features represented by criteria in the model. Ratings for said criteria and said vendor products may have been obtained from former client implementations, independent evaluations of vendor products, and other sources that may be found and included. From the said decision model, the said at least one Analyst may generate questionnaires and may provide said questionnaires to Client X that may be distributed by means of electronic mail, and by any other means as may be suitable at the time, including but not exclusively web based forms and automated response and analysis processes. An example of an automated response may be the establishment of a website document with controlled access, and response data processed and set into a database from which the information may then extracted and may be processed through machine readable code, and may cause said processor to tailor said ERP model. In a second embodiment, that may be additional to the first embodiment, from the return of said questionnaires the at least one Analyst may then assess the responses and may accordingly use the information gathered to assist in tailoring the said model. In another embodiment of the process, the Analyst may make use of interactive computer network group processes to interactively determine said decision model, thereby characterizing Client X. In the current preferred embodiment, the preferred interactive network product is NetMeeting™ manufactured and developed by Microsoft Corporation, Seattle, Washington. The Analyst may then interact with at least one member of Client X over the computer network, and may provide the use of the model to at least one Client X member through the interactive group process, thereby jointly editing the said model, and may determine the criteria required for Client X, and may remove those criteria not required by Client X, and may further tailor the said model with other information and determine decision procedures such as sensitivity analysis, as required by Client X, and may be available from the said model and at least one knowledge base. In another embodiment of the process, the said at least one Analyst may visit the client site and through a process of questioning and elicitation from at least one decision-maker of Client X and further characterize Client X. By said means Client X is duly characterized, and the at least one Analyst may then determine similar past clients who may have similar characterization profiles.

As an example of said characterization measure, a measure of said closeness of Client X to another client may be determined by means of estimating the Matching Index value (this value is determined as part of the composite index in equation 1) as determined from the common criteria count in the two aggregate models, and weights of said common decision factors as determined by Client X and that used by said other client of said advisory services. Other measures such as a linear regression fit may be applied, said means being determined by one knowledgeable in the art. Said characterization then assists the Analysis in determining the decision objects and procedures that may be required by Client X and have been stored in the automated advisory services knowledge base.

The at least one Analyst may have used the decision support engine to build and populate the model, and may at least one part of said model by executing machine readable code to translate the format of the stored model data structure stored in another structure data format into the format of the said model and may display said model for tailoring purposes. Said at least one Analyst may then proceed from the information provided by Client X, to tailor the model. Mandatory data may be tailored, and at least one mandatory component of the model may be disabled, and may be removed from the model, and others may remain enabled. For example, if Client X uses IBM computers but does not use DEC computers, then all relevant mandatory information for IBM computers may be enabled, and those for DEC computers may be disabled, and may be deleted from the model, or marked Not Applicable, the said at least one Analyst determining the course of action to be performed. Said at least one Analyst may then designate at least one node for aggregation. For example, it may be agreed by the at least one Analyst and at least one member of Client X that the criterion "Product Functionality" may be aggregated at the second level above the root, as the detail below the node is not required at this time. It may also be determined by the said at least one Analyst that the information below the said criterion "Product Functionality" is important, but it is desirable to remove them for simplification requirements, and therefore aggregation parameters will be calculated for the node. On the other hand, the item "Corporate Service and Support" is determined asunimportant in terms of the information content below the node, possibly because preferred vendors may be closely the same in this parameter, thence the said at least one Analyst may determine that there is no requirement to generate aggregation parameters for said node, and consequently no aggregation parameters are generated, and the node is simply trimmed, retaining only the normalized weighted average from all descendant nodes as representing the score for the said node for each alternative in the model. In another embodiment, the evaluation for the ability of the vendor to actually deliver, under the factor "Corporate Viability", has score distributions for at least some alternatives that are not amenable to current methods to calculate aggregation parameters of sufficient accuracy in the calculations. This may be indicated by prior comparisons of exact calculations and current methods as may exist at the time in calculating the aggregation parameters. The node "Corporate Viability" is determined by the at least one Analyst to be marked 'Hidden', preserving said data for calculation purposes, while simplifying the data presentation to Client X members, and retaining confidential any proprietary information therein.

Decision factor attributes such as weights may be determined from prior information as disclosed earlier. Some criteria weights may be reduced to zero as these criteria are determined by the said at least one Analyst to not be relevant to the needs of Client X. Similarly, costs related to the presence and the absence of a feature may have been obtained from prior experience of similar Clients of particular Vendors. The costs of ownership of particular solutions from each Vendor suite of ERP products may further have been garnered from other installation experiences. Said data may be present in the Analyst model, and may be aggregated as factor attributes and Alternative attributes. On the basis of required attributes and aggregated information, as well as the agreed at least one decision procedure to be used for the decision, said at least one decision procedure is applied and the at least one Analyst may then generate an initial ranking of the alternatives. According to one aspect of the invention, the at least one Analyst may determine a short list of vendors who best meet the Client requirements. This may include in one embodiment selecting those Vendors that fail the least set number of mandatory requirements, as well as those that may have the highest overall scores, according to the chosen decision method that may include, amongst others, lowest cost, best score, and best benefit per unit cost or best cost per unit benefit. Said at least one Analyst may then add instructions and other meta-data to the aggregated model, and this may include adding, removing and editing decision procedures, factor attributes and alternative attributes. Once said short list of preferred vendors is determined and decision procedures appropriately tailored, the aggregate model may be saved, and a report generated for Client X. Said report may be customized from a template and include aggregated information, and may be distributed to at least one member of Client X. Said at least one aggregate model may further be provided to Client X. In another embodiment of said process, a report may be provided to Client X, where said report may disclose aggregated model criteria and may show criteria weights, and may include preferred alternatives and determined alternative attributes, and may further include meta-data. Said report may be generated for management level presentation to Client X. In another aspect of the invention, Client X may require the generation of a Request for Proposal (RFP) to be sent to the preferred vendors. The Analyst may select at least one report template within the decision support system to automatically generate the said RFP, An RFP is generated by machine readable code that may be a component of the client decision support system, causing said processor to process said model andselect data from said model, and organize and format said selected data and output said data in the form of an RFP report. In one embodiment the output is in the form of a printed report executed on a printer. In another embodiment said report is saved in a known application format such as Microsoft MS Word, and may be stored on a storage medium such as a CD-ROM, and may be sent by electronic messaging system to at least one member of Client X. In another embodiment, said report may contain at least one decision object.

In another aspect of the invention, the Analyst may assign and store keys and sub- keys in the aggregate model in order to limit the distribution and time of availability of the model, as may be required according to at least one license agreement with at least one member of Client X. The analyst may then provide at least one member of Client X with at least one aggregate model and at least one client decision support system.

Client X upon receipt of the aggregate model, and having obtained the necessary authorization keys from an authorized site, may then proceed to further analyze the vendors using the aggregate model and dynamic decision procedures to generate at least one scenario. Said AAS may assist Client X to generate a short list of vendors for which RFP's may be generated from at least one report template and to whom said RFP's may be sent.

In another aspect of the process, when said RFP responses are returned, the said responses may be supplied to Client X and may be provided to the said AAS. Data may also be provided automatically over a computer network into which vendors have access, said computer network thereby providing means to automatically update data contained in the automated advisory services knowledge database. In this manner, the AAS benefits by updating its information base and content. As a consequence, the AAS may then provide a subscription update service to its clients, including Client X. Further, from the scenarios generated by Client X, weight and criteria attribute information may be obtained by the AAS and added to the knowledge database for future reference. Additional decision procedures may also be obtained and added to the decision knowledge base, said procedures obtained from the Client that may be by means of agreement and may be garnered from Client X's staff. In another aspect of the procedure, the AAS may provide further decision procedures and data obtained from other sources, thereby updating the client aggregate model, said process being made automatic by machine readable code that causes a general purpose processor to modify existing model attributes and content in the said tailored Client X aggregate model.

In another aspect of the process that may depend on the level of service agreed to between the AAS and Client X, assistance in bid evaluation by said AAS may be provided. Said assistance may be by presence of at least one Analyst at the client site. Selection of said vendor may follow proposal and counterproposal steps, and scripted decision procedures may be made available to Client X to provide assistance to Client X in selecting at least one vendor. Said decision procedures may require determination of scores, cost information, and challenges of said at least one selected vendor, and determination of business risk based on prior performance of the at least one selected vendor. When at least one preferred vendor is selected, the ASS may provide additional services to provide a detailed summary report generated from a detailed report template. Said report may include graphic, numeric and meta-data in a format suitable for presentation for senior management summary as well as detailed information for project management requirements that may include cost and project success risk estimations.

In a second embodiment, said assistance may be delivered by remote conferencing as earlier disclosed using customized group networking products such as NetMeeting™. The said at least one Analyst may assist Client X by using scripted decision support procedures, said script may have been determined for Client X prior to delivery of the said aggregate model and said aggregate report, and may be incorporated in the knowledge base of said AAS. Additional information and decision procedures, including negotiation approaches may be garnered from Client X against at least one vendor by the advisory service, enabling the storage of said methods for future reference in the knowledge base and within decision procedures. Said information may then provide the at least one Analyst with means to determine other scripts for other clients. Further, in another embodiment, said issues raised by Client X may provide vendors with information to determine reasons for lost and won sales, and thereby the impact of Client X views of the at least one preferred vendor's weaknesses and strengths may then allow at least one vendor to plan future development and market strategy.

In one embodiment, said script may provide to Client X prior analysis and procedures to enable negotiations with said at least one preferred vendor. For example, the lack of a feature such as the non-support for a particular E-mail system may result in an incurred cost in procurement and adaptation to the new E-mail system. Said cost may be assigned to the vendor as a cost attribute dependent on the rating value for the support of said E-mail system provided by the vendor. The sum of such costs may then be assessed and may constitute at least a component of a report on least one preferred vendor. A means to deduct such costs from the ERP implementation assists in reduction of final ERP cost implementation and better planning for said insured costs. Thus if the ERP system is assessed at $3,500,000, the average cost from prior experience at other similar ERP installations due to the lack of support for the E-mail system is estimated at $200,000. Hence negotiation may allow reduction of the ERP system cost to $3,300,000, or assist the vendor in determining that the cost of implementing support for the E-mail system would ultimately pay-back since a number of significant future sales may be affected by said feature absence. In another scenario, said vendor and Client X may determine that only partial support is required, hence the rating value may increase and indicate a cost to the Client is minimal if vendor supplies said partial support, and vendor and Client X may then share the risk. Statistics of failure of sales as a result of missing feature, or profit margin detriment, maintained in said ERP model, may provide vendors with adequate information to determine future directions of development.

As well, in another embodiment of the process, said advisory service may provide means to select outsourcing consultants through at least one other aggregated model, and may assist in selection of said outsourcing agency. In another embodiment said means to assist Client X in selecting project personnel may be offered to Client X as a service. In a third embodiment, said advisory service may provide key personnel to assist in decision support purposes, which may include but not exclusively procurement of required items, project management decisions and project, personnel and vendor performance analysis. In one preferred embodiment, each said assistance aspect may comprise of additional models, and a set of said models may then be described as comprising a model library, said library thereby consisting of at least two tailored models covering at least two aspects of a project process. Said library may then be treated as a single model in terms of licensing, and thence the licensing procedures applicable to models may then be applied to said model library.

In another embodiment, said advisory services may follow a detailed scripted process as indicated in figures 22, 23,24, 25, and 26 wherein the service may be divided into four parts, the first 1601 may be to determine the requirements of Client X. Within part I, as indicated in Figure 23, the market overview may be provided to Client X 1701-1703, wherein a broad view of the ERP process may be provided, and said major vendors 1704, 1705 and criteria 1706 are presented in broad terms to Client X. Said process thereby educates Client X as to the major issues and relative market placement of the vendors with respect to Client X's requirements. In so doing the advisory service analyst may obtain sufficient detailed information of Client X's broad requirements to begin the tailoring process through a needs definition process 1707, 1708 including mandatory requirements 1709 and other criteria 1710. Requirements definition templates that provide means to further identify the needs of the client may be generated 1707 and distributed to at least one Client X member. Responses to said document distribution may provide sufficient information to eliminate at least one vendor from the list of vendors through compliance analysis 1716-1718, 1715. Thence a tailored model may then provide a selection overview 1719 and first management review documents 1720, 1721. In one embodiment, at the stage 1719 Client X may receive a client decision support system and at least one aggregate model under a license agreement. Part II of the process may consist of further detailed examination of the reduced number of vendors 1602, and may consist of the steps indicated in figure 24 1800. Through this process, metadata and Analyst assistance is provided to Client X to evaluate vendors and further customize said aggregate model that may have been provided to at least one member of Client X, and interactive means process over a computer network may be employed as indicated herein to facilitate said detailed evaluation. Templates and decision processes 1804-1806 using visual decision dictionaries and decision procedures 1807-1811 may assist in further elimination of at least one vendor, and may provide a short list of vendors for which a further report and supporting documents may be provided to Client X. In another embodiment, Client X at least one member may also generate said supporting documents and reports from customized template reports, and may be assisted by at least one Analyst 1813-1815. Finally, decision justification is provided that may include the strengths and challenges in the selection of said at least one preferred vendor. Various scenarios may be explored, said scenarios being performed by use of visual decision dictionaries and decision procedures 1812-1818, and may be assisted by the Analyst. Decision justification documents may then be prepared for senior management and senior decision makers, 1819-1821. In the third part of the said process, the final selected at least one vendor may be provided a Letter of Bid that may be generated from templates within the decision support system using selected data from the tailored aggregate model, and may be prepared by the at least one Analyst who may assist at least one member of Client X . Negotiation procedures with final at least one vendor may then proceed 1901-1903, utilizing said procedures and information contained in the said tailored decision model, and may further be assisted from meta-data procedures known by the at least one Analyst, and may be contained within the knowledge base of said main ERP decision model 1904-1906. The Analyst may assist in the final selection through facilitation procedures between the at least one vendor and Client X, or through remote conferencing and multimedia interaction as disclosed herein, and as may be available at the time 1907, 1908, and occur to one knowledgeable in the art. Final management reports may be generated from templates concerning justification of the decision, and may contain details of decision issues, negotiation points and results and such items as may be required by senior management 1909-1911. In the fourth and final part of the advisory service process 1604, said analyst may assist project planning by identifying the detailed challenges that need to be overcome, and identifying deficiencies and strengths within vendor features that may or may not be compatible with Client X systems and processes. In so doing, said Analyst further obtains information and data for model improvement. Follow-up procedures may also be performed to assist Client X in measuring the performance of said vendor, and thereby this process may provide added information on cost of implementation, vendor performance and project risk at the detailed level, thence giving the opportunity for using said information 1605 in refining the knowledge base and Analyst model 1606 of the automated advisory service. Said refinement may be determined by one knowledgeable in the art and may include industry standards 1607where said industry standards may include industry averages and industry means, industry-accepted levels and legal requirements imposed on the industry. Said refinement may further include vendor performance standard values for at least one vendor and other vendor specific information 1608, and client characteristics 1608. Vendor information may be of great interest to prospective clients of the vendors as well as other competing vendors, and conversely, client characterization may be important for vendors. Thence advisory services may be realized through the process of systematic knowledge acquisition and utilization in assisting clients and vendors based on actual case studies and scenarios. Such scenarios may thereby be characterized by the aggregate model containing at least one decision factor and at least one decision procedure and at least one decision procedure attribute that was implemented, and said characterization may provide means of comparison between prospective clients of vendors, and of vendor performance in said client scenarios, and among all clients of the automated advisory service. In one embodiment of the decision support system, means may be provided to the said at least one Analyst to customize all scripts into a single decision process herein called a Decision Process Guide, as may be indicated in figure 26. At least one of the steps as may be determined by the Analyst may be provided 1951, and selected decision procedures as may be determined by the Analyst included 1953, with customized text and other meta-data 1954 that may assist in linking each step and sub-step in the process, and provide direct access to at least one Analyst or other information related to the decision process. Said combination of indicated steps, decision procedures, and link data and meta-data may be combined to comprise a customized decision process guide for the Client. Client X, for example, may require at least three steps, and thereby be presented on a graphical display an image as indicated in figure 26. Said image presents the major steps 1958, 1963 that may be selected in one preferred embodiment by means of buttons pressed using a pointing device such as a computer mouse. On selecting a sub-step which in the instance of the figure is Market Overview, said vendors and vendor products may be presented in a tree 1965, and details of at least one selected vendor displayed elsewhere as in, for example, frames 1962, 1969. Said views may have been determined by the Analyst prior to delivery of customized decision process. Link information in one embodiment may be displayed 1966, and contact means added for additional sites information that may include vendor web locations 1967 and expert contacts 1968. NetMeeting™ interactive procedures may be invoked as may be desired and arranged with the Analyst 1960. In the current preferred embodiment, said Decision Process Guide may be an independent computer readable executable code and may be supplied to the client, the execution of which may then read at least one selected aggregate model at the client site. In another embodiment, said tree may consist of the aggregate model decision factors, as may be determined by one knowledgeable in the art, and as may be relevant to the data displayed in the at least one chart in frame 1962. In another preferred embodiment, the Analyst may determine certain visibility features for the said frames, causing meta-data to appear in for example the chart frame 1962, and vice- versa. The analyst may, in another preferred embodiment, add or remove frames from said decision process guide, as may be determined for each process sub-step of the Decision Process.

In another embodiment, said Decision Process Guide may be integrated in with the decision Support system, and may only be licensed to read selected licensed aggregate models, as may be determined by the Analyst, and said licensing procedures as indicated in 900 may then be applied to the Decision Process Guide with the additional restriction of reading only specific aggregate models.

In one embodiment, it may not be desired to provide means for said Client X to further aggregate said model. Thence input means may be provided to the said at least one Analyst to disable said aggregation ability of the decision support system. Said means may be by inputting a code into the aggregate model, causing said decision support system to prevent further aggregation of said aggregate model. Said disabling may be preferred since the aggregation process may require external means to compute said aggregation parameters, and said means may not be available to said client, thereby compromising the accuracy of the aggregate model further since aggregation of an aggregated node into a higher node may cause loss of said parameters. In another embodiment, said Decision Process Guide may consist of at least one stored aggregate model and client decision support system, and may be provided as executable machine readable code, wherein said code upon execution on a processor causes said processor to read only the at least one stored aggregate model provided within said executable code's data structure. In another aspect of this invention, said means to estimate costs of implementation project may be determined from combined factor and alternative attributes within the aggregate model. In another aspect, said likelihood of success of a project may be determined from at least one measure of the degree of success accredited to at least one vendor known from prior experience of clients that have used and may still be using said vendor. Said at least one measure of the degree of success may be estimated in terms of numeric values representing the difference between targeted deliverables - representing a target alternative - in terms of functionality and solution quality, and estimates of actual functionality and quality of said deliverables. Said difference may be interpreted as a vendor success risk. Said example illustrates an advisory service process in a procurement procedure for an ERP system. Said process can be applied to any procurement process, and can further include any decision process open to a systematic decision making process. Said systematic decision making processes can include, and may not necessarily limited to, hiring, performance measurement, systems analysis, project performance measurement, medical decision making, and strategic planning.

In another embodiment of the process, said tailored model may be stored on at least one storage medium such as a Winchester hard disk drive 64 or CD-ROM 68, and may be read by at least one special purpose processor 2004 in figure 27, said processor thereby caused to read said model by machine readable code executing on said special purpose processor 2004. Said processor may receive input to from sensors 2001, 2002. For example, in a nuclear power plant such sensors may read temperature of coolant. In another embodiment, said sensors may measure coolant fluid flow. Said sensors may thereby be any sensor generating a calibrated signal as may be determined by one knowledgeable in the art, and said signal converted to a rating value of significance to the decision model, where said signal may be represented by a criterion within the decision model. From data that may be provided by former tests and similar active sites, said weights in regard to a critical situation developing may be determined. For example, sensor 1 (2001) near the reactor may provide temperature readings where it is known that such readings indicate a likely reactor failure. Sensor 2 (2002) is further away, and its readings toward a reactor failure may not be so significant. Sensor 1 is therefore given for a condition (alternative) called FAILURE CONDITION A a weight significantly higher than that of sensor 2 - i.e the local weight of sensor 1 may be much larger that of sensor 2. If the importance of sensor 1 is determined as ten times the importance of sensor 2 in regard to FAILURE CONDITION A (specified as an alternative with respect to the Goal of "Reactor Failure"), then the local weight of sensor 1 is 2012 and is determined to be (w, =1/11=9.09%), and that of sensor 2 is (w =10/11=90.91%) 2013. Thence the sensors, grouped under the criterion "Temperature Sensors" in the model 2009, may be provided ratings according to the temperature signal. Said ratings and weights may then be aggregated into the hidden node "Temperature Sensors", 2011 which represents the aggregated signal utility value from special processor 2004. In one embodiment of said process, the aggregated node attributes may be determined by a machine readable code executed on special purpose processor 2004, said aggregated attributes may then be transmitted to the higher level general processor 2007, where said aggregated attribute may be displayed to an operator on a graphic display device 2008, where only node 2011 may be seen. Said device and attribute data may be aggregated information indicating for example the status of the processor determined worst temperature reading by sensors under the hidden node "Temperature Sensors" 2011. In the event the aggregated value exceeds a preset threshold (said threshold may be stored locally in local storage device 2003 and read by the special purpose processor 2004), a signal may be generated by said general purpose processor, and said signal causes machine readable code to execute on a special purpose or general purpose computer 2007, causing said hidden node to expand and expose the underlying data of node 2011, said underlying data may then be displayed to at least one operator, and may cause the generation of at least one signal to indicate that at least one threshold may have been exceeded.

In another embodiment, said weights 2012 and 2013 may be changed to cause a new perspective to be examined. For example, said temperature sensors in respect to calibration signals may be measured against a target set of values, and the importance of said calibration may change. For example, sensor 1 may be calibrated against sensor 2, since sensor 2 is amenable to human access, and hence the importance of calibration of sensor 2 may be four times that of sensor 1, thence the local weights of the two may be sensor 1 80%, and sensor 2 20%, in confidence of the calibration performance of the system. Hence, the action of selecting CALIBRATION PERFORMANCE STATUS as the alternative, causes said processor to select and tailor said model of system for calibration performance status of the system for alternative CALIBRATION PERFORMANCE STATUS, including thereby "Temperature Sensors" criteria, but may not include other criteria such as mandatory items with discrete values such as valve closed/open signals. In another embodiment of the process, said selection of alternative CALIBRATION PERFORMANCE STATUS may cause machine readable code to execute on a general purpose or special purpose processor, causing said processor to disaggregate hidden nodes 2010 and may cause additional signals to cause automatic measurement of calibration of said sensors.

In another aspect of the invention, said signals may cause the enabling of at least one decision procedures, said at least one decision procedure being present according to the aggregated/disaggregated state of the aggregated model. Said procedures may for example provide instructions concerning the current state, and may indicate consequences of action by means of dynamic graphs and generation of what-if scenarios. In another embodiment of the process, processor 2007 may not have within its model the nodes known to 2004. Thence at least one output signal 2006 carries information pertaining to said unknown nodes to 2007, causing said processor 2007 to create said nodes 2010 with attributes that may be contained in said at least one signal 2006. Said newly created nodes may then be displayed on the graphical display 2008, and may include at least one attribute of said created nodes. In the example, said attributes may include weights related to a decision or analysis condition, and may include the temperature readouts of said sensors 2001 and 2002. In another embodiment, nodes 2001 and 2002 may be processors providing 2004 with aggregate information of sensors which are descendant nodes to said nodes 2001 and 2002, and so forth.

Said data for weights that may represent FAILURE CONDITION A may be generated from prior data determined from similar systems. In another embodiment, said data may be generated by means of simulation procedures run on a general purpose or special purpose processor. Additionally, said prior data may be processed by machine readable code to determine at least one utility function and at least one rating method for ratings, and weight of importance toward, for example, the goal of CALIBRATION PERFORMANCE STATUS of the system.

Claims

What is claimed is:

1. A computer-implemented decision support system for comparing two alternatives, comprising: memory means for storing at least one data structure having a plurality of decision factors; output means to store at least one data structure containing at least the plurality of said decision factors; input means to determine at least one decision factor with at least one descendant decision factor node as at least one parent aggregate node; output means to store at least one said aggregate node with at least one aggregate attribute as an aggregate model;

2. A decision support system as in 1 wherein input means to determine at least one descendant decision factor below at least one said parent aggregate node with at least one decision factor attribute is to be aggregated into at least one attribute in the said parent aggregate node;

3. A decision support system as in 1 wherein input means to determine at least one decision factor as an aggregate hidden parent node with at least one descendant decision factor;

4. A decision support system as in 3 wherein input means to determine a said hidden aggregate node as not to be included in a decision process and is included in the output aggregate model;

5. A decision support system as in 3 where input means causes at least one hidden node to expose at least one descendant node and at least one attribute of the said at least one descendant node;

6. A decision support system as in 4 wherein input means causes said hidden aggregate node and its hidden at least one descendant node to be used in determining the decision;

7. A decision support system as in 1 with input means to cause at least one descendant decision factor below an aggregate parent to be aggregated in the said parent aggregate node of said at least one descendant node;

8. A decision support system as in 1 where at least one input signal causes at least one said determined descendant node to aggregate into said determined parent aggregate node;

9. A decision support system as in 1 with input means to determine at least one meta-data attribute to be aggregated in at least one meta-data attribute field of said parent aggregate node;

10. A decision support system as in 1 with input means to cause at least one metadata item to be directed to aggregate in at least one meta-data aggregate data field of aggregation node;

11. A decision support system as in 1 , 2 and 3 with input means to cause said decision support system to indicate the aggregate status of a node;

12. A decision support system as in 1 2 and 3 wherein a visual marker displayed on a graphical display indicates the selected aggregation state of at least one node;

13. A decision support system as in 1, 2 and 3 with processing means to aggregate at least one attribute of at least one determined descendant node determined for aggregation in at least one determined aggregate node;

14. A decision support system as in 13 wherein at least one said attribute of said at least one determined descendant node contains meta-data;

15. A decision support system as in 14 wherein said meta-data provides input means to cause said decision support system to locate external assistance;

16. A decision support system as in 15 wherein said external assistance is an expert advisory service, and wherein external expert advisory service provides means for controlled access to said expert advisory service;

17. A computer-implemented decision support system for comparing two alternatives, comprising: input means for at least one aggregate model; memory means for storing a data structure containing at least one aggregate model; processing means to determine license validation status of at least one stored aggregate model ;

18. A decision support system as in 1 wherein input means causes assignment of at least one code in said stored aggregate model and causing at least one decision support system as in 17 executed on at least one processor to verify said code;

19. A decision support system as in 1 wherein input means causes said at least one limiting availability condition of said at least one stored aggregate node to require input of at least one other code through input means provided in decision support system as in 17;

20. A decision support system as in 18 wherein input means causes said at least one limiting availability condition of said at least one stored aggregate node to include at least one specified time limited access for at least one decision support system as in 17;

21. A decision support system as in 18 wherein input means causes said at least one limiting availability condition of said at least one stored aggregate node to include specific named individual access as verified by at least one decision support system as in 17;

22. A decision support system as in 18 wherein input means causes said at least one limiting availability condition of said at least one stored aggregate node to include access limitation from at least one decision support system as in 17 and causes said at least one decision support system as in 17 to determine if said decision support system as in 17 is permitted to input said at least one stored aggregate model;

23. A decision support system as in 18 wherein input means causes said at least one limiting availability condition of said at least one stored aggregate model to include limitation on the number of decision support systems as in 17 that may simultaneously access said at least one aggregate node;

24.. A decision support system as in 18 wherein input means causes said at least one limiting availability condition of said at least one stored aggregate model to include limitation to input storage location of the said at least one stored aggregate model for at least one decision support system as in 17;

25. A decision support system as in 1 with output means to assign a unique identification code to said at least one stored aggregate model;

26. A decision support system as in 1 and 17 with input means to input at least one plurality of scores for at least one alternative;

27. A decision support system as in 26 with input means to input at least one plurality of weights for said at least one aggregate node;

28. A decision support system as in 27 with input means to determine processing means to translate at least one plurality of scores into at least one utility value to be included in determining at least one multi-attribute value for at least one alternative;

29. A decision support system as in 25 and 28 with processing means to determine at least one multi-attribute score for at least one alternative;

30. A decision support system as in 28 with processing means to determine at least one multi-attribute difference between at least two alternatives;

31. A decision support system as in 28 with output means to indicate the at least one multi-attribute difference of the at least two alternatives;

32. A decision support system as in land 17 with input means to assign at least one alternative-specific attribute to at least one alternative;

33. A decision support system as in 32 with input means to determine if at least one alternative specific attribute is used to determine at least one alternative attribute value;

34. A decision support system as in 33 with input means to determine the relationship between at least two alternative attributes for at least one alternative;

35. A decision support system as in 34 wherein output means causes said processor to aggregate all nodes determined as aggregate nodes, and further causes said alternative specific attributes to be removed if said alternative-specific attributes are determined not to be used in said determination of at least one alternative attribute value as in 32;

36. A decision support system as in 35 wherein stored at least one aggregate model contains at least one hidden node with at least one aggregate attribute and at least one hidden descendant node;

37. A decision support system as in 29 with processing means to determine at least one parameter that is stored as at least one attribute for said aggregate model and used to determine at least one multiattribute score in said aggregate model;

38. A decision support system as in 29 with processing means to determine at least one parameter that is stored in said aggregate model and used to determine at least one alternative-specific attribute;

39. A decision support system as in 37 with input means to input at least one aggregation parameter, said at least one aggregation parameter being determined by means other than the decision support system as in 1 ;

40. A decision support system as in 1 and 17 with processing means for transforming (i) a first plurality of scores for a first alternative that includes at least one aggregate score into at least one multi-attribute score, and (ii) a second plurality of scores for a second alternative that includes at least one aggregate score into at least one second multi- attribute score;

41. A decision support system as in 40 with input means to cause said decision support system to select from the available alternatives a 'best set' of alternatives;

42. A decision support system as in 40 with output means to output at least one output signal corresponding to at least one of the first and second aggregate scores to provide a ranking of said at least two alternatives;

43. A decision support system as in 1 and 17 wherein is provided at least one chart representing at least one aspect of a decision process, said chart representing at least one decision parameter to be used to determine the decision;

44. A decision support system as in 43 wherein is provided input means to determine at least one meta-data field with said at least one chart, said at least one meta-data field including at least one customized instruction on the use of said at least one chart;

45. A decision support system as in 44 wherein input means is provided to include meta-data to provide means to locate at least one other assistance source for the use of the said at least one chart, and where said source is located on a computer network;

46. A decision support system as in 43 wherein input means is provided to determine said at least one decision factor attribute value from the value of at least one chart parameter;

47. A decision support system as in 43 wherein input means is provided to cause at least one parameter of the said chart to determine at least one alternative-specific attribute;

48. A decision support system as in 43, 44 ,45, 46 and 47 with input means to determine the at least one chart represents a decision object, said decision object containing at least one parameter that is used to compare at least two alternatives and at least one meta-data item in at least one meta-data field;

49. A decision support system as in 48 with input means to input meta-data to determine at least one decision object as at least one visual decision dictionary;

50. A decision support system as in 48 and 49 with output means for at least one said decision object to be stored in a data structure other than an aggregate model;

51. A decision support system as in 1 and 17 with input means to input at least one decision object;

52. A decision support system as in 51 wherein input means is provided to change at least one decision object parameter;

53. A decision support system as in 52 wherein output means displays at least one decision object parameter as a chart object on a graphical display device;

54. A decision support system as in 53 wherein processor means determines at least one changed parameter and causes said decision support system as in 1 and 17 to determine values determined dependent on said at least one changed parameter;

55. A decision support system as in 54 wherein said alternative attribute is determined as a measure of currency;

56. A decision support system as in 54 wherein said alternative attribute is determined to be a measure of probability representing risk;

57. A decision support system as in 32 wherein said at least one alternative attribute represents at least one vendor product attribute, said at least one attribute determining the market position of at least one vendor product in respect to at least one attribute significant to characterizing said at least one vendor product in the market place in respect to said attribute;

58. A decision support system as in 40 wherein input means causes said decision support system to determine at least one alternative with decision factor ratings and alternative-specific attributes representing at least one industry standard rating for said at least one decision factor and at least one alternative-specific attribute;

59. A decision support system as in 40 wherein processing means determines for said at least one decision factor at least one said decision factor score representing a standard score for at least two selected alternatives;

60 A decision support system as in 40 wherein input means causes said decision support system to determine at least one alternative with ratings and scores representing industry standard;

61. A decision support system as in 59 wherein said decision support system determines at least one alternative with scores determined as alternatives-specific values determined from scores of at least two selected alternatives;

62. A decision support system as in 43 with processing means to determine at least one difference of at least one attribute between an ideal or target alternative and at least one other alternative and from said at least one difference determine at least one alternative success risk parameter;

63. A decision support system as in 26 wherein said at least one alternative is divided into at least two sub-alternatives;

64. A decision support system as in 63 wherein said at least two sub-alternatives have at least one parent alternative attribute;

65. A decision support system as in 64 wherein said at least one sub-alternative attribute determines at least one attribute value for the alternative;

66. A decision support system as in 43 with input means to cause said at least one chart and at least one chart parameter to be locked and prevented from change when said at least one chart parameter is attempted to be changed by input means as in 50;

67. A decision support system as in 66 with input means to determine at least one other instance of at least one said locked chart and determine from the difference between the at least one locked chart and second unlocked instance of said locked chart the impact of change when said second unlocked instance of said chart is changed as in 50;

68. A decision support system as in 67 wherein said determined impact is measured in terms of at least one cost measure;

69. A decision support system as in 52 with output means to store said at least one changed datum in a data structure representing a scenario of said decision;

70. A decision support system as in 18 wherein input means is provided to cause a decision support system as in 17 to limit the number of saved scenarios as in 69;

71. A decision support system as in 52 wherein processing means causes timed delay prior to determining score values based on a first change, and where input within the delay time causes said delay to restart to allow at least one other change as in 52 prior to determining at least one value used in determining the value of at least one model attribute as in 54;

72. A decision support system as in 48 wherein input means is provided for metadata for said at least one decision object, and where said meta-data is stored with the aggregate model;

73. A decision support system as in 72 wherein said meta-data is stored with decision object in a data structure;

74. A decision support system as in 52 with output means to output at least one signal representing the changed data;

75. A decision support system as in 52 wherein temporary storage is provided for at least one changed datum, and said original data remaining unchanged by said changes;

76. A decision support system as in 1 and 17 wherein is provided at least one report template;

77. A decision support system as in 76 wherein input means is provided for said at least one template to be customized by a decision support system as in 1 and 17;

78. A decision support system as in 77 wherein said at least one template is a customized request for proposal (RFP) report;

79. A decision support system as in 78 wherein said at least one template includes at least one management report template, said at least one management report template containing at least one parameter from at least one decision object;

80. A decision support system as in 18 wherein input means is provided to transfer aggregate model license of at least one aggregate model as stored in 1 , said transfer of license being from one processor to at least one other processor, and said allowance of transfer being determined by license verification process as in 18;

81. A decision support system as in 1 with input means to determine at least two steps in a decision process;

82. A decision support system as in 81 with input means to provide meta-data for each of said at least two steps;

83. A decision support system as in 81 with input means to associate at least one decision procedure for each of said at least two steps, where said decision procedure consists of at least one visual decision dictionary as in 49;

84. A decision support system as in 81 with output means to provide computer processor executable code containing a decision support system as in 17 and at least one aggregate model;

85. A decision support system as in 81 and 17 with output means to store at least one aggregate model with license constraint as verifiable as in 18;

86. A decision support system as in 85 with output means to store a decision support system as executable machine readable code with at least one aggregate model with license constraint as verifiable as in 18;

87. A decision support system as in 1 with input means to prevent further aggregation for said aggregate model;

88. A decision support system as in 87 with input means to enable further aggregation of aggregate model;

89. A decision support machine as in 1 and 17 wherein output means causes said model data structure to be encrypted;

90. A decision support machine as inl and 17 wherein input means for said at least one aggregate model causes decryption of said model data structure;

91. An automated advisory service method using the decision support system 1 and at least one aggregate model, said automated advisory service method consisting of means for storing a data structure having at least one aggregate model; means to determine at least one requirement for at least one client; means to determine at least two decision process steps for at least one decision for at least one client; means to eliminate decision factors and mandatory items from a decision model based on requirements; means to aggregate data from a larger data set to a customized data set; means to store said customized data set as at least one aggregate model; means to determine at least one scripted decision procedure for said at least one decision step; means to provide at least one scripted decision procedure in said at least one aggregate node model;

92. An automated advisory service method as in 91 with input means to determine at least one report template for said at least one aggregate decision model;

93. An automated advisory service method as in 91 wherein means to determine at least one decision procedure script is by at least one interactive multimedia process between advisory service and client over a computer network;

94. An automated advisory service method as in 91 wherein means to determine at least one client requirement by at least one interactive multimedia process between advisory service and client over a computer network, said requirement providing at least one item to characterize said at least one client;

95. An automated advisory service method as in 91 with input means to input at least one decision procedure as in 89 into said at least one aggregate model;

96. An automated advisory service method as in 91 with output means to output at least one decision procedure;

97. An automated advisory service method as in 91 with input means to input at least one signal and said at least one signal causes said decision support system as in 46 to output at least one decision object from at least one aggregate model;

98. An automated advisory service method as in 91 wherein input of at least one signal causes at least one decision support system as in 1 and 17 to determine at least one alternative attribute, and cause said at least one decision support system to select at least one preferred alternative according to said at least one alternative attribute;

99. An automated advisory service method as in 92 wherein at least one report template in said aggregate model is a customized questionnaire script;

100. An automated advisory service method as in 99 wherein at least one response from at least one scripted questionnaire is used to characterize at least one client and determine at least one client requirement;

101. An automated advisory service method as in 100 and 94 wherein said client characterization is used to determine a list of preferred vendors, said vendors determined as having characteristics most likely to meet client requirements amongst all considered vendors;

102. An automated advisory service method as in 91 wherein at least one scripted assistance is determined by 94 and 100 and wherein input means inputs said at least one scripted assistance into at least one decision procedure;

103. An automated advisory service method as in 92 wherein it is determined that for at least one step at least one meta-data scripted assistance is provided for at least one report template to generate at least one report for a management review process as determined in 94 and 100;

104. An automated advisory service method as in 91 with input means for at least one aggregate model and at least one customized scripted decision procedure;

105. An automated advisory service method as in 92 wherein at least one report template with at least one scripted procedure provides means to customize and generate at least one Letter of Bid;

106. An automated advisory service method as in 92 wherein at least one report template with at least one scripted procedure provides means to customize and generate at least one Request for Proposal;

107. An automated advisory service method as in 91 wherein at least one customized script is determined to provide at least one negotiation point between vendor and - client;

108. An automated advisory service method as in 92 wherein at least one report template is determined and customized for a final management review process;

109. An automated advisory service method as in 92 wherein at least one detailed report is generated from at least one aggregate model to determine at least one project task for an implementation project determined from said decision;

110. An automated advisory service method as in 91 wherein input means provides means in the aggregate model to measure vendor project performance for at least one task within the implementation project for said client;

111. An automated advisory service method as in 110 wherein at least one vendor risk value assessments may be obtained for at least one task in said implementation project;

112. An automated advisory service method as in 112 wherein input means of at least one signal causes said at least one attribute to generate at least one output signal representing said at least one risk value;

113. An automated advisory service method as in 94 and 100 wherein processing means is provided to determine at least one attribute characteristic of said client;

114. An automated advisory service method as in 112 and 113 wherein said at least one attribute is contained in at least one decision procedure, and said decision procedure is stored in a structured knowledge database;

115. An automated advisory service method as in 114 wherein processing means by decision support system as in 1 has input means for said stored knowledge database, and determines industry standard and vendor and client specific data;

116. An automated advisory service method as in 115 wherein processing means has means to identify challenge issues between at least one client and at least one vendor, and determine vendor implementation performance based on client characteristics as determined in 94 and 100, and vendor characteristics as determined in 101;

117. An automated advisory service method wherein processing means compares at least one client with at least one other client by means of aggregate models of said clients, said processing means providing at least one value representing degree of similarity between said client aggregate models;

118. An automated advisory service method as in 115 wherein output means is provided to store said at least one determined industry standard in the knowledge database;

119. A computer-readable storage medium for storing machine readable code which, when executed by a processor, causes said processor to: store decision data structure having a plurality of decision factors each decision factor representing at least one input signal, each said decision factor having a weight assigned thereto, the plurality of weighted factors comprising a predetermined two-dimensional benchmark pattern that may be selectively used as a target pattern; input means for at least one signal for at least one decision factor; processing means to transform the at least one input signal into at least one utility value; input means for at least one value representing a target value for at least one decision factor, said at least one target value representing a target pattern; processing means to pattern-match the first contribution pattern and the target pattern to produce a value indicating the match between the at least one target value and the said at least one utility value; output means to output at least one output signal corresponding to at least the difference between the at least one target value and at least one input value

120. A storage medium as in 119 that causes said computer to output at least one signal when target pattern and input signal pattern meet a specific level of difference as measured by the said pattern match value;

121. A storage medium as in 119 wherein said output signal 120 causes the disaggregation of said aggregated model, said degree of aggregation thereby represents the status of the system the said aggregate model represents;

122. A storage medium as in 116 wherein said output signal 120 causes a decision support system as in 17 to disaggregate a second model stored at a different location, said second model having components representing at least the node signal structure of said source of signal 120;

123. A decision support system as in 122 with output means to output at least one signal to a graphical display device where at least one node of said aggregate model is displayed on a graphical display;

124. A decision support system as in 122 wherein input of at least one signal 117 causes said model to expose all nodes in said model;

125 An aggregated model as in 124 wherein disaggregation exposes underlying node values and meta-data related to said underlying nodes;

126. An aggregated model as in 122. and 122 wherein disaggregation of values and meta-data is made visible on said graphical display by a screen update procedure;

127 A disaggregated model as in 122 wherein input of at least one output signal 97 causes said model to aggregate;

128 An aggregated model as in 127 wherein aggregation aggregates underlying node values and meta-data related to said underlying nodes;

129. An aggregated model as in 122. and 127 wherein input signal causes aggregated values and meta-data to be made visible on said graphical display by a screen update procedure;

130. A decision support system as in 124 wherein reception of said at least one output signal 120 causes the execution of at least one decision procedure, said specific at least one decision procedure depending on the aggregation/disaggregation state of the aggregated model;

131. A decision support system as in 124 and 130 wherein output means provides output from the decision procedure to indicate at least one course of action;

132. A decision support system as in 131 where the said decision procedure can indicate the consequences of taking at least one preferred action, prior to taking said at least one action;

133. A computer-implemented decision support machine for comparing two alternatives, comprising: memory means for storing an aggregate model data structure having a plurality of decision factors, each said decision factor having a weight assigned thereto, the plurality of weighted factors comprising a predetermined two-dimensional benchmark pattern; means for inputting a first plurality of scores for a first competing alternative to the decision factors of said decision data structure, and for inputting a second plurality of scores for a second competing alternative to the decision factors of said decision data structure; processing means to transform at least one score into at least one attribute utility value; output means to output at least one output signal corresponding to at least one utility value to provide a comparison of said competing alternatives, and output means to store at least one data structure containing at least the plurality of said decision factors and at least one factor attribute value and at least one alternative and at least one alternative attribute value; memory means for storing said model structure; processing means to translate at least one raw score into a utility value representing the worth of the raw score toward said decision for at least one said alternative; means to compute the utility value from the raw score value for at least one of said alternatives and; means for identifying at least one node as an aggregate node into which descendant criteria attributes below the aggregate node are aggregated into attributes in the aggregate node output means to output an aggregate model;

134. A decision support machine as in 133 wherein input means for identifying at least one node as an aggregate hidden node into which descendant criteria attributes below the aggregate node are aggregated into attributes in the aggregate hidden node;

135. A decision support machine as in 133 wherein input means to cause a hidden node to be excluded from determining the attribute values for at least one alternative;

136. A decision support machine as in 134 where input means causes at least one hidden node to expose at least one descendant node and at least one attribute of the said at least one descendant node;

137. A decision support machine as in 134 wherein input means causes said hidden aggregate node and said hidden at least one descendant nodes to determine at least one attribute value for at least one alternatives;

138. A decision support machine as in 133 where input means determines which descendant decision factors below an aggregate parent decision factor are to be aggregated in the parent aggregate node;

139. A decision support machine as in 133 where at least one signal causes at least one descendant node attribute to aggregate into at least one attribute of the parent aggregate node;

140. A decision support machine as in 133 where input means determines the meta- data attributes to be aggregated for at least one descendant node;

141. A decision support machine as in 133 where input means causes at least one meta-data item to aggregate in at least one meta-data aggregate data field;

142. A decision support machine as in 133 with input means to cause said decision- support machine to output at least one signal representing the state of aggregation of at least one aggregate node;

143. A decision support machine as in 133 wherein a visual marker displayed on a graphical display indicates the aggregation state of the node;

144. A decision support machine as in 133 with processing means to aggregate at least one attribute of descendant nodes determined as nodes to aggregate;

145. A decision support machine as in 133 wherein the said attributes contain metadata;

146. A decision support machine as in 145 wherein at least one aggregated metadata item is displayed on a graphical display;

147. A decision support machine as in 133 wherein said meta-data may cause said machine to output at least one signal requesting access for at least one data item located in at least one known location on a computer network;

148. A decision support machine as in 146 wherein at least one automated advisory service is accessed;

149. A decision support machine as in 148 wherein said output signal causes at least one input signal in response, said input signal thereby determining at least one modification in the meta-data content of said model;

150. A decision support machine as in 148 wherein said output signal causes at least one input signal in response, said input signal thereby determining at least one modification in the numeric data content of said model;

151. A decision support machine as in 133 wherein said raw scores comprise at least one input signal from at least one sensor;

152. A decision support machine as in 133 wherein input means causes output of at least one scripted decision procedure;

153. A decision support machine as in 133 wherein at least one input signal causes at least one node to be determined as a hidden aggregate node, and causes said node to add hidden nodes;

154. A decision support machine as in 133 wherein at least one input signal causes at least one attribute to be assigned to at least one said hidden node;

155. A decision support machine as in 133 wherein at least one input signal causes said decision support system as in 130 to create at least one decision node at a specified location in said aggregate model, and assign at least two attributes to said node, said at least two attributes being determined as at least one weight attribute and at least one raw score attribute;

156. A decision support machine as in 133 wherein at least one input signal causes said decision support system as in 130 to eliminate at least one decision node and all said eliminated node's attributes ;

157. A decision support machine as in 133 wherein output means causes said model data structure to be encrypted;

158. A decision support machine as in 133 wherein input means for said at least one aggregate model causes decryption of said model data structure;