US20090271351A1 - Rules engine test harness - Google Patents
Rules engine test harness Download PDFInfo
- Publication number
- US20090271351A1 US20090271351A1 US12/323,707 US32370708A US2009271351A1 US 20090271351 A1 US20090271351 A1 US 20090271351A1 US 32370708 A US32370708 A US 32370708A US 2009271351 A1 US2009271351 A1 US 2009271351A1
- Authority
- US
- United States
- Prior art keywords
- test
- rules
- results
- database
- test cases
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0637—Strategic management or analysis, e.g. setting a goal or target of an organisation; Planning actions based on goals; Analysis or evaluation of effectiveness of goals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/08—Insurance
Definitions
- the United States Medicaid program was enacted in 1965 to provide a medical assistance program for individuals and families with low incomes.
- the Medicaid program comprises three main entities—the patients, the healthcare providers, and the agency administering the plan (i.e., the payer).
- the Medicaid program is financed through joint federal and state funding.
- the Medicaid program is administered by each state according to an approved state plan.
- the specifics of the Medicaid program differ from state to state. Differences may include covered healthcare procedures, allowable procedure costs, and patient eligibility criteria.
- the state administrators of the Medicaid program are required to have a Medicaid management information system (MMIS) that provides for mechanized and/or computerized Medicaid claims processing.
- MMIS Medicaid management information system
- Recently, the Medicaid information technology architecture (MITA) has been promulgated by the U.S. government to provide a blueprint and a set of standards that individual states are to follow in administering the Medicaid program and for developing the next generation MMIS.
- Rules engines may be pluggable software components that execute rules that have been externalized from application code.
- the rules define business rules and/or business logic that may change frequently.
- rules may be defined by nonprogrammers and may be provided to the rules engine in the form of data or data files.
- Using a rules engine to provide business rules to an application may reduce time to market and reduce total cost of ownership, with reference to the alternative of encoding the business logic in high level programming language code.
- Automation designs for example business rules to be executed by a rules engine and/or applications encoded in a high level programming language, are typically tested extensively before deployment in customer facing products and/or systems. Testing may take the form of executing a number of test cases on the target computer systems, where each test case comprises a single set of inputs to the subject rules engine and/or application. In addition to defining the inputs, the test cases may define the expected output and/or results of executing the rules engine and/or the application on the inputs. Because testing every possible combination of inputs is typically infeasible, an attempt is often made to identify a representative selection of test cases that may provide sufficient confidence in the automation design under test. Testing automation designs may be an expensive process that consumes substantial business resources including personnel, equipment, and schedule time. Insufficient testing can result in deployment of a flawed automation design that may fail inopportunely, possibly damaging the enterprise brand and/or incurring liability.
- a system comprising a computer system, a builder component, and a test execution component.
- the builder component promotes defining a plurality of test cases and a plurality of test scenarios.
- the text execution component simulates at least one service application, invokes execution of a plurality of rules on a rules engine with an input based on one of the test cases defined using the builder component, wherein the rules engine interacts with the simulated service application, and stores the result of the rule execution in a database.
- a method comprises providing a builder interface for building a plurality of test cases and a plurality of test scenarios, the interface comprising a list of rules and a plurality of input controls for defining a plurality of inputs for the test cases.
- the method also comprises storing the test cases and the test scenarios in a database, invoking the rules on a rules engine based on at least one of the test cases and the test scenarios, and simulating at least one service application, wherein the rules engine interacts with the service application when executing the rules.
- the method also comprises collecting the results of the invocation of the rules on the rules engine and storing the results in the database.
- an apparatus comprising a test case editor and a rule test manager.
- the test case editor When executed on a first computer system, the test case editor provides an environment for building a plurality of rule test cases by selecting an entry point from a list of available entry points, by defining a plurality of inputs for the rule, and by defining expected results for one of the test cases.
- the rule test manager executes the rule test cases by simulating at least one service program, invoking a plurality of rules on a rules engine based on the rule test cases built using the test case editor, receives results from the rules engine, and stores the results, wherein the rules engine interacts with the simulated service program when executing the rules.
- FIG. 1 is a schematic diagram of a rules engine test harness according to an embodiment of the disclosure.
- FIG. 2 is a flowchart of a method of testing rules according to an embodiment of the disclosure.
- FIG. 3 is a schematic diagram of an exemplary general-purpose computer system suitable for implementing some aspects of the several embodiments of the disclosure.
- a rules engine test harness for testing rules may promote testing rules without using a fully configured and/or duplicate execution environment.
- a rules engine framework is used to create and optionally to compile rules that may be executed by a rules engine. Further details about a rules engine framework are provided in U.S. patent application Ser. No. 12/257,782 filed Oct. 24, 2008 entitled “Rules Engine Framework” by Uma Kandasamy et al., which is hereby incorporated by reference in its entirety.
- the rules When deployed as a released product, the rules may be executed by a rules engine on a process server computer system in cooperation and coordination with a business process.
- the business process and rules engine cooperatively provide at least a portion of a healthcare management information system.
- a user of the healthcare management information system may interact with the business process and rules engine by entering information and selecting actions from a user interface presented at a workstation communicatively coupled to the process server computer system.
- the business process may promote enrolling healthcare providers into the healthcare system, for example enrolling physicians, therapists, hospitals, minor emergency medical centers, and other healthcare providers.
- the business process may promote enrolling healthcare recipients and/or patients.
- the business process may promote receiving and processing claims from enrolled healthcare providers for services provided to enrolled healthcare recipients.
- a function of the business process may be invoked by the user of the workstation, for example processing of a claim for healthcare services.
- the business process may process this claim by invoking a series of rules and/or a rule flow on the rules engine.
- the use of rules to provide automated processing may have the advantage of allowing for the rapid creation, modification, and deployment of rules with respect to processing based on specifically designed computer programs.
- the processing of the claim may involve the rules engine executing a sequence of rules that validate the enrollment of the healthcare provider, validate the enrollment of the healthcare recipient, determine a coverage status of the subject procedure, determines a maximum coverage amount, and other such processes.
- the rules engine may return a rule result object to the business process that invoked the rules engine, wherein the rule result object may identify one or more exceptions if something went wrong, or no exceptions if the rule processed successfully.
- the system 100 comprises a computer system 102 , a workstation 108 , a network 110 , and a database 112 .
- the computer system 102 comprises a builder component 104 and a test execution component 106 .
- the workstation 108 may execute a builder interface 126 and a test control interface 128 .
- the database 112 may also be referred to as a test case repository, and may comprise a plurality of test cases 114 , a plurality of test scenarios 116 , a plurality of test results 122 , and a plurality of success/fail reports 124 .
- the database 112 may be substantially empty and the test cases 114 , the test scenarios 116 , the test results 122 , and the success/fail reports 124 may accumulate in number over time. While the computer system 102 in FIG. 1 is shown to be in direct communication with the database 112 , in some embodiments the computer system 102 may communicate with the database 112 via the network 110 .
- the computer system 102 , the workstation 108 , and the database 112 along with their contained components, applications, and data, may be referred to in some contexts as a rules engine test harness.
- the network 110 may or may not be considered as a part of the rules engine test harness.
- the system 100 also comprises a server 118 and a rules engine 120 .
- the test execution component 106 then invokes the server 118 to execute the test cases.
- the computer system 102 , the workstation 108 , and the server 118 may each be implemented as a general-purpose computer system, which is discussed in greater detail hereinafter.
- the builder component 104 may comprise one or more components.
- the builder component 104 promotes defining a plurality of test cases and a plurality of test scenarios for testing rules, for example using the builder interface 126 on the workstation 108 .
- the builder interface 126 may be a web interface that is provided by the builder component 104 , for example as a hypertext mark-up language (HTML) page transmitted from the builder component 104 to the workstation 108 .
- the test cases and test scenarios may be stored in the database 112 .
- a test case 114 identifies a set of inputs, a rule, and a set of expected results from executing the rule based on the set of inputs.
- a rule may comprise a condition portion and an action portion.
- the condition may comprise a left hand variable associated with a right hand value by a generic template.
- the condition may comprise a left hand operand associated with a right hand operand by an operator.
- a rule may comprise a condition that evaluates the enrollment of an individual in a healthcare management information system and an action that indicates the rule succeeded if the condition evaluates to TRUE and sets an exception code if the condition evaluated to FALSE.
- a test scenario 116 comprises an ordered sequence of test cases 114 . Each test scenario 116 may be used to determine the correct operability of one or more rules.
- the set of inputs associated with a test case 114 may comprise a number of different input parameters and the values associated with those different input parameters.
- the set of expected results may comprise an output parameter and the values associated with the output parameter.
- as many as 6,000 or more rules may be deployed to be executed by a rules engine in an enterprise function, for example a portion of a healthcare management information system.
- a representative set of test cases 114 adequate to provide an acceptable level of confidence in the rules may exceed 100,000 test cases 114 .
- the builder component 104 provides drop down menus for composing test cases 114 , for example in a web interface provided to the workstation 108 .
- the builder interface 126 may be a web interface that is provided by the builder component 104 , for example as an HTML page transmitted from the builder component 104 to the workstation 108 .
- the drop down menus may comprise drop down menus for selecting invocation points and/or entry points, for defining inputs, and for defining results, for example rule return object contents and/or values.
- the builder component 104 may provide a way to begin defining a new test case 114 by first cloning or copying a previously defined test case 114 .
- the new test case 114 is then changed in some manner, for example by specifying a different value for an input parameter and a different expected value of the rule return object.
- the builder component 104 also provides means for composing a test scenario 116 as an ordered sequence of test cases 114 .
- the builder component 104 writes test cases 114 to the test cases 114 in the database 112 and writes test scenarios 116 to the test scenarios 116 in the database 112 .
- the builder component 104 writes test cases 114 and test scenarios 116 to the database 112 by using a test case data access object (DAO).
- DAO test case data access object
- the builder component 104 may be referred to as a test case editor.
- the test execution component 106 simulates one or more service applications that the rules engine 120 interacts with when executing rules.
- the test execution component 106 may simulate a portal server processor, a plurality of databases, a plurality of database management applications, a messaging application, and other applications and/or services.
- the test execution component 106 also invokes execution of rules on the rules engine 120 according to the test cases 114 and/or the test scenarios 116 stored in the database 112 , for example under the control of a web interface on the workstation 108 .
- the test control interface 128 may be a web interface that is provided by the builder component 104 , for example as a HTML page transmitted from the builder component 104 to the workstation 108 .
- the test execution component 106 may read the test cases 114 and/or test scenarios 116 from the database 112 and, based on the content of the test cases 114 and/or test scenarios 116 , invoke the execution of specific rules with appropriate input parameter values specified by the test cases 114 and/or test scenarios 116 .
- the test execution component 106 receives the results of the execution of the rules by the rules engine 120 , for example receiving a rules return object containing SUCCESS and/or exception code values.
- the test execution component 106 saves the results of executing the test cases 114 and/or the test scenarios 116 in the test results 122 of the database 112 .
- the test execution component 106 compares the rule return object received from the rules engine 120 to the expected rule return object defined in the test cases 114 and generates a report based on this comparison.
- the test execution component 106 may compare the rule return object from the current execution of a test case 114 with the expected result that was optionally provided as input to the test case. If the expected result is not given as an input, the result will be marked as not available (N/A) since there is no data for the comparison.
- the report may comprise a summary of numbers of test cases 114 that passed, numbers of test cases 114 that failed, numbers of test scenarios 116 that passed, numbers of test scenarios 116 that failed.
- the report may comprise trend analysis of pass/fail statistics over a plurality of testing sessions.
- the test execution component 106 stores the report in the success/fail report 124 in the database 112 .
- the test execution component 106 may communicate results information to the database 112 at least in part using a reporting data access object.
- the rule return object may be referred to as results and/or test results.
- the test execution component 106 may optionally perform timing of the execution of one or more of the test cases 114 , for example test cases 114 that include an elapsed time of execution expected value.
- a test case 114 may be directed to testing the timeliness of a response to an operator command input.
- the test execution component 106 may store the timing results along with other test results in the test results 122 of the database 112 .
- comparisons among timing results for executing the same or similar test cases 114 stored in the test results 122 of the database 112 may be performed to determine a trend of performance of the rules engine 120 .
- the network 110 may comprise any combination of private networks and public networks.
- the network 110 may comprise electrical or optical wired links as well as wireless links.
- the network 110 may be confined to a single facility or campus, but in another embodiment, the network 110 may be distributed across a wide geographical area, including across the entire globe.
- the rules engine 120 may include some portions of a commercial off the shelf (COTS) rules engine tool.
- COTS rules engine tool may be customized and/or encapsulated by a wrapper or a rules engine framework to provide functionality and convenience otherwise not provided by the COTS rules engine tool.
- a Fair Isaac BLAZE ADVISOR COTS rules engine package may be encapsulated in the rules engine 120 .
- the rules engine 120 may extend the COTS rules engine tool, for example, by providing automated input value validation between a left hand variable and a right hand input.
- the rules engine 120 also may feature other extensions of the COTS rules engine tool.
- a test builder interface comprising a list of rules to be tested.
- the tested rules may be all of the rules or only some of the rules.
- the test builder interface 126 may be provided as a web interface that may execute on the workstation 108 .
- the list of rules and/or entry points may be provided as one or more drop down menus in the test builder interface 126 .
- a plurality of test cases 114 are defined using the test builder interface. Defining test cases 114 may comprise identifying a specific rule, an entry point, one or more input parameter values associated with the rule, and one or more expected output parameter values associated with executing the rule based on the input parameter values.
- the rule test cases 114 and test scenarios 116 defined at block 208 above are stored, e.g., in the database 112 .
- a plurality of test case designers may be concurrently defining test cases 114 and test scenarios 116 by executing the test builder interface on separate workstations 108 , and may be concurrently storing the test cases 114 and the test scenarios 116 in the database 112 .
- the rules are invoked on the rules engine 120 based on the test cases 114 and based on the test scenarios 116 .
- a single invocation point may be invoked at one time, providing appropriate input parameter values based on the test case 114 .
- a plurality of invocation points may be serially invoked by a test scenario 116 , thereby providing appropriate input parameter values based on the test cases 114 composing the test scenario 116 .
- the rules engine 120 may access the rule and/or rules out of a rule repository database (not shown), or the rule definition may be provided along with the input parameter values.
- the rules are precompiled and/or preprocessed into a binary file format that is stored on the server 118 , for example in Fair Isaac binary (ADB) file format.
- ADB Fair Isaac binary
- one or more service applications are simulated by the test execution component 106 .
- the rules engine 120 may interact with the service applications during the course of executing the rule under test.
- the service applications may comprise a portal server processor, a database management system, a messaging service, and other services.
- the elapsed time of execution of the rule processing by the rules engine 120 is optionally timed by the test execution component 106 .
- the results of executing the rule and/or rules by the rules engine 120 for example output parameter values and optionally elapsed time information is collected.
- the results of executing the rule and/or rules are stored in the database 112 .
- the results may also be displayed by an interface, for example a web interface that may execute on the workstation 108 .
- a success/fail report is optionally generated based on the results of executing the rule and/or rules.
- the expected output parameter values defined in the test cases 114 are compared to the results to determine whether the subject test cases 114 succeeded or failed.
- Summary statistics for example total numbers of successes and failures, may be aggregated and placed in the success/fail report 124 .
- the success/fail report 124 may also include information comparing the current test session results with earlier test results to report a test success trend.
- the success/fail report 124 may be stored in the database 112 and may be displayed by an interface, for example a web interface that may execute on the workstation 108 . The process 200 then exits.
- FIG. 3 illustrates a typical, general-purpose computer system suitable for implementing one or more embodiments disclosed herein.
- the computer system 380 includes a processor 382 (which may be referred to as a central processor unit or CPU) that is in communication with memory devices including secondary storage 384 , read only memory (ROM) 386 , random access memory (RAM) 388 , input/output (I/O) devices 390 , and network connectivity devices 392 .
- the processor 382 may be implemented as one or more CPU chips.
- the secondary storage 384 is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM 388 is not large enough to hold all working data. Secondary storage 384 may be used to store programs that are loaded into RAM 388 when such programs are selected for execution.
- the ROM 386 is used to store instructions and perhaps data that are read during program execution. ROM 386 is a non-volatile memory device, which typically has a small memory capacity relative to the larger memory capacity of secondary storage 384 .
- the RAM 388 is used to store volatile data and perhaps to store instructions. Access to both ROM 386 and RAM 388 is typically faster than to secondary storage 384 .
- I/O devices 390 may include printers, video monitors, liquid crystal displays (LCDs), touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices.
- LCDs liquid crystal displays
- touch screen displays keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices.
- the network connectivity devices 392 may take the form of modems, modem banks, Ethernet cards, universal serial bus (USB) interface cards, serial interfaces, token ring cards, fiber distributed data interface (FDDI) cards, wireless local area network (WLAN) cards, radio transceiver cards such as code division multiple access (CDMA), global system for mobile communications (GSM), and/or worldwide interoperability for microwave access (WiMAX) radio transceiver cards, and other well-known network devices.
- These network connectivity devices 392 may enable the processor 382 to communicate with an Internet or one or more intranets. With such a network connection, it is contemplated that the processor 382 might receive information from the network, or might output information to the network in the course of performing the above-described method steps. Such information, which is often represented as a sequence of instructions to be executed using processor 382 , may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave.
- Such information may be received from and outputted to the network, for example, in the form of a computer data baseband signal or signal embodied in a carrier wave.
- the baseband signal or signal embodied in the carrier wave generated by the network connectivity devices 392 may propagate in or on the surface of electrical conductors, in coaxial cables, in waveguides, in optical media, for example optical fiber, or in the air or free space.
- the information contained in the baseband signal or signal embedded in the carrier wave may be ordered according to different sequences, as may be desirable for either processing or generating the information or transmitting or receiving the information.
- the baseband signal or signal embedded in the carrier wave, or other types of signals currently used or hereafter developed, referred to herein as the transmission medium may be generated according to several methods well known to one skilled in the art.
- the processor 382 executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage 384 ), ROM 386 , RAM 388 , or the network connectivity devices 392 . While only one processor 382 is shown, multiple processors may be present. Thus, while instructions may be discussed as executed by a processor 382 , the instructions may be executed simultaneously, serially, or otherwise executed by one or multiple processors 382 .
Abstract
Description
- This application claims priority to U.S. Provisional Application Ser. No. 61/048,810 filed Apr. 29, 2008, and entitled “MMIS Health Enterprise Solution,” by Jack Devos, et al., which is incorporated herein by reference for all purposes.
- Not applicable.
- Not applicable.
- The United States Medicaid program was enacted in 1965 to provide a medical assistance program for individuals and families with low incomes. The Medicaid program comprises three main entities—the patients, the healthcare providers, and the agency administering the plan (i.e., the payer). The Medicaid program is financed through joint federal and state funding. The Medicaid program is administered by each state according to an approved state plan. The specifics of the Medicaid program differ from state to state. Differences may include covered healthcare procedures, allowable procedure costs, and patient eligibility criteria. The state administrators of the Medicaid program are required to have a Medicaid management information system (MMIS) that provides for mechanized and/or computerized Medicaid claims processing. Recently, the Medicaid information technology architecture (MITA) has been promulgated by the U.S. government to provide a blueprint and a set of standards that individual states are to follow in administering the Medicaid program and for developing the next generation MMIS.
- Rules engines may be pluggable software components that execute rules that have been externalized from application code. The rules define business rules and/or business logic that may change frequently. Typically, rules may be defined by nonprogrammers and may be provided to the rules engine in the form of data or data files. Using a rules engine to provide business rules to an application may reduce time to market and reduce total cost of ownership, with reference to the alternative of encoding the business logic in high level programming language code.
- Automation designs, for example business rules to be executed by a rules engine and/or applications encoded in a high level programming language, are typically tested extensively before deployment in customer facing products and/or systems. Testing may take the form of executing a number of test cases on the target computer systems, where each test case comprises a single set of inputs to the subject rules engine and/or application. In addition to defining the inputs, the test cases may define the expected output and/or results of executing the rules engine and/or the application on the inputs. Because testing every possible combination of inputs is typically infeasible, an attempt is often made to identify a representative selection of test cases that may provide sufficient confidence in the automation design under test. Testing automation designs may be an expensive process that consumes substantial business resources including personnel, equipment, and schedule time. Insufficient testing can result in deployment of a flawed automation design that may fail inopportunely, possibly damaging the enterprise brand and/or incurring liability.
- In an embodiment, a system is provided. The system comprises a computer system, a builder component, and a test execution component. When executed by the computer system, the builder component promotes defining a plurality of test cases and a plurality of test scenarios. When executed by the computer system, the text execution component simulates at least one service application, invokes execution of a plurality of rules on a rules engine with an input based on one of the test cases defined using the builder component, wherein the rules engine interacts with the simulated service application, and stores the result of the rule execution in a database.
- In another embodiment, a method is provided. The method comprises providing a builder interface for building a plurality of test cases and a plurality of test scenarios, the interface comprising a list of rules and a plurality of input controls for defining a plurality of inputs for the test cases. The method also comprises storing the test cases and the test scenarios in a database, invoking the rules on a rules engine based on at least one of the test cases and the test scenarios, and simulating at least one service application, wherein the rules engine interacts with the service application when executing the rules. The method also comprises collecting the results of the invocation of the rules on the rules engine and storing the results in the database.
- In another embodiment, an apparatus is provided. The apparatus comprises a test case editor and a rule test manager. When executed on a first computer system, the test case editor provides an environment for building a plurality of rule test cases by selecting an entry point from a list of available entry points, by defining a plurality of inputs for the rule, and by defining expected results for one of the test cases. When executed on the first computer system, the rule test manager executes the rule test cases by simulating at least one service program, invoking a plurality of rules on a rules engine based on the rule test cases built using the test case editor, receives results from the rules engine, and stores the results, wherein the rules engine interacts with the simulated service program when executing the rules.
- These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims.
- For a more complete understanding of the present disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.
-
FIG. 1 is a schematic diagram of a rules engine test harness according to an embodiment of the disclosure. -
FIG. 2 is a flowchart of a method of testing rules according to an embodiment of the disclosure. -
FIG. 3 is a schematic diagram of an exemplary general-purpose computer system suitable for implementing some aspects of the several embodiments of the disclosure. - It should be understood at the outset that although illustrative implementations of one or more embodiments are illustrated below, the disclosed systems and methods may be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques illustrated below, but may be modified within the scope of the appended claims along with their full scope of equivalents.
- A rules engine test harness for testing rules is disclosed. The rules engine test harness may promote testing rules without using a fully configured and/or duplicate execution environment. In an embodiment, a rules engine framework is used to create and optionally to compile rules that may be executed by a rules engine. Further details about a rules engine framework are provided in U.S. patent application Ser. No. 12/257,782 filed Oct. 24, 2008 entitled “Rules Engine Framework” by Uma Kandasamy et al., which is hereby incorporated by reference in its entirety. When deployed as a released product, the rules may be executed by a rules engine on a process server computer system in cooperation and coordination with a business process. In an embodiment, the business process and rules engine cooperatively provide at least a portion of a healthcare management information system. A user of the healthcare management information system may interact with the business process and rules engine by entering information and selecting actions from a user interface presented at a workstation communicatively coupled to the process server computer system. The business process may promote enrolling healthcare providers into the healthcare system, for example enrolling physicians, therapists, hospitals, minor emergency medical centers, and other healthcare providers. The business process may promote enrolling healthcare recipients and/or patients. The business process may promote receiving and processing claims from enrolled healthcare providers for services provided to enrolled healthcare recipients.
- A function of the business process may be invoked by the user of the workstation, for example processing of a claim for healthcare services. The business process may process this claim by invoking a series of rules and/or a rule flow on the rules engine. The use of rules to provide automated processing may have the advantage of allowing for the rapid creation, modification, and deployment of rules with respect to processing based on specifically designed computer programs. The processing of the claim may involve the rules engine executing a sequence of rules that validate the enrollment of the healthcare provider, validate the enrollment of the healthcare recipient, determine a coverage status of the subject procedure, determines a maximum coverage amount, and other such processes. In an embodiment, the rules engine may return a rule result object to the business process that invoked the rules engine, wherein the rule result object may identify one or more exceptions if something went wrong, or no exceptions if the rule processed successfully.
- Turning now to
FIG. 1 , asystem 100 for rules testing is discussed. Thesystem 100 comprises acomputer system 102, aworkstation 108, anetwork 110, and adatabase 112. Thecomputer system 102 comprises abuilder component 104 and atest execution component 106. Theworkstation 108 may execute abuilder interface 126 and atest control interface 128. Thedatabase 112 may also be referred to as a test case repository, and may comprise a plurality oftest cases 114, a plurality oftest scenarios 116, a plurality oftest results 122, and a plurality of success/fail reports 124. It is understood that when thesystem 100 is first placed into service, thedatabase 112 may be substantially empty and thetest cases 114, thetest scenarios 116, thetest results 122, and the success/fail reports 124 may accumulate in number over time. While thecomputer system 102 inFIG. 1 is shown to be in direct communication with thedatabase 112, in some embodiments thecomputer system 102 may communicate with thedatabase 112 via thenetwork 110. Thecomputer system 102, theworkstation 108, and thedatabase 112, along with their contained components, applications, and data, may be referred to in some contexts as a rules engine test harness. Thenetwork 110 may or may not be considered as a part of the rules engine test harness. Thesystem 100 also comprises aserver 118 and arules engine 120. Thetest execution component 106 then invokes theserver 118 to execute the test cases. Thecomputer system 102, theworkstation 108, and theserver 118 may each be implemented as a general-purpose computer system, which is discussed in greater detail hereinafter. - The
builder component 104 may comprise one or more components. Thebuilder component 104 promotes defining a plurality of test cases and a plurality of test scenarios for testing rules, for example using thebuilder interface 126 on theworkstation 108. Thebuilder interface 126 may be a web interface that is provided by thebuilder component 104, for example as a hypertext mark-up language (HTML) page transmitted from thebuilder component 104 to theworkstation 108. Likewise, the test cases and test scenarios may be stored in thedatabase 112. As used herein, atest case 114 identifies a set of inputs, a rule, and a set of expected results from executing the rule based on the set of inputs. In an embodiment, a rule may comprise a condition portion and an action portion. When the condition evaluates TRUE when executed by therules engine 120, the action is performed. The condition may comprise a left hand variable associated with a right hand value by a generic template. Worded differently, the condition may comprise a left hand operand associated with a right hand operand by an operator. As an example, a rule may comprise a condition that evaluates the enrollment of an individual in a healthcare management information system and an action that indicates the rule succeeded if the condition evaluates to TRUE and sets an exception code if the condition evaluated to FALSE. As used herein, atest scenario 116 comprises an ordered sequence oftest cases 114. Eachtest scenario 116 may be used to determine the correct operability of one or more rules. - The set of inputs associated with a
test case 114 may comprise a number of different input parameters and the values associated with those different input parameters. Similarly, the set of expected results may comprise an output parameter and the values associated with the output parameter. For exemplary purposes, asimple test case 114 may comprise pairs of parameter names and input values and a result as follows (member identity=1234567, age=67, procedure code=2007, provider identity=7654321, expected action=SUCCESS). An alternativesimple test case 114 may comprise pairs of parameter names and input values in the form of XML and a result in XML as well (member identity=1234567, age=−2, procedure code=2007, provider identity=7654321, expected action=set exception code 1234). In some embodiments, as many as 6,000 or more rules may be deployed to be executed by a rules engine in an enterprise function, for example a portion of a healthcare management information system. A representative set oftest cases 114 adequate to provide an acceptable level of confidence in the rules may exceed 100,000test cases 114. - In an embodiment, the
builder component 104 provides drop down menus for composingtest cases 114, for example in a web interface provided to theworkstation 108. Thebuilder interface 126 may be a web interface that is provided by thebuilder component 104, for example as an HTML page transmitted from thebuilder component 104 to theworkstation 108. The drop down menus may comprise drop down menus for selecting invocation points and/or entry points, for defining inputs, and for defining results, for example rule return object contents and/or values. In an embodiment, thebuilder component 104 may provide a way to begin defining anew test case 114 by first cloning or copying a previously definedtest case 114. Thenew test case 114 is then changed in some manner, for example by specifying a different value for an input parameter and a different expected value of the rule return object. Thebuilder component 104 also provides means for composing atest scenario 116 as an ordered sequence oftest cases 114. Thebuilder component 104 writestest cases 114 to thetest cases 114 in thedatabase 112 and writestest scenarios 116 to thetest scenarios 116 in thedatabase 112. In some embodiments, thebuilder component 104 writestest cases 114 andtest scenarios 116 to thedatabase 112 by using a test case data access object (DAO). In some contexts, thebuilder component 104 may be referred to as a test case editor. - The
test execution component 106 simulates one or more service applications that therules engine 120 interacts with when executing rules. For example, thetest execution component 106 may simulate a portal server processor, a plurality of databases, a plurality of database management applications, a messaging application, and other applications and/or services. Thetest execution component 106 also invokes execution of rules on therules engine 120 according to thetest cases 114 and/or thetest scenarios 116 stored in thedatabase 112, for example under the control of a web interface on theworkstation 108. Thetest control interface 128 may be a web interface that is provided by thebuilder component 104, for example as a HTML page transmitted from thebuilder component 104 to theworkstation 108. Thetest execution component 106 may read thetest cases 114 and/ortest scenarios 116 from thedatabase 112 and, based on the content of thetest cases 114 and/ortest scenarios 116, invoke the execution of specific rules with appropriate input parameter values specified by thetest cases 114 and/ortest scenarios 116. - Additionally, the
test execution component 106 receives the results of the execution of the rules by therules engine 120, for example receiving a rules return object containing SUCCESS and/or exception code values. Thetest execution component 106 saves the results of executing thetest cases 114 and/or thetest scenarios 116 in thetest results 122 of thedatabase 112. In an embodiment, thetest execution component 106 compares the rule return object received from therules engine 120 to the expected rule return object defined in thetest cases 114 and generates a report based on this comparison. Thetest execution component 106 may compare the rule return object from the current execution of atest case 114 with the expected result that was optionally provided as input to the test case. If the expected result is not given as an input, the result will be marked as not available (N/A) since there is no data for the comparison. - The report may comprise a summary of numbers of
test cases 114 that passed, numbers oftest cases 114 that failed, numbers oftest scenarios 116 that passed, numbers oftest scenarios 116 that failed. The report may comprise trend analysis of pass/fail statistics over a plurality of testing sessions. Thetest execution component 106 stores the report in the success/fail report 124 in thedatabase 112. In an embodiment, thetest execution component 106 may communicate results information to thedatabase 112 at least in part using a reporting data access object. In some contexts, the rule return object may be referred to as results and/or test results. - In an embodiment, the
test execution component 106 may optionally perform timing of the execution of one or more of thetest cases 114, forexample test cases 114 that include an elapsed time of execution expected value. For example, atest case 114 may be directed to testing the timeliness of a response to an operator command input. Thetest execution component 106 may store the timing results along with other test results in thetest results 122 of thedatabase 112. In an embodiment, comparisons among timing results for executing the same orsimilar test cases 114 stored in thetest results 122 of thedatabase 112 may be performed to determine a trend of performance of therules engine 120. - The
network 110 may comprise any combination of private networks and public networks. Thenetwork 110 may comprise electrical or optical wired links as well as wireless links. In an embodiment, thenetwork 110 may be confined to a single facility or campus, but in another embodiment, thenetwork 110 may be distributed across a wide geographical area, including across the entire globe. - The
rules engine 120 may include some portions of a commercial off the shelf (COTS) rules engine tool. The COTS rules engine tool may be customized and/or encapsulated by a wrapper or a rules engine framework to provide functionality and convenience otherwise not provided by the COTS rules engine tool. In an embodiment, a Fair Isaac BLAZE ADVISOR COTS rules engine package may be encapsulated in therules engine 120. In an embodiment, therules engine 120 may extend the COTS rules engine tool, for example, by providing automated input value validation between a left hand variable and a right hand input. Therules engine 120 also may feature other extensions of the COTS rules engine tool. - Turning now to
FIG. 2 , amethod 200 is described. Atblock 204, a test builder interface is provided comprising a list of rules to be tested. The tested rules may be all of the rules or only some of the rules. Thetest builder interface 126 may be provided as a web interface that may execute on theworkstation 108. The list of rules and/or entry points may be provided as one or more drop down menus in thetest builder interface 126. Atblock 208, a plurality oftest cases 114 are defined using the test builder interface. Definingtest cases 114 may comprise identifying a specific rule, an entry point, one or more input parameter values associated with the rule, and one or more expected output parameter values associated with executing the rule based on the input parameter values. Atblock 212, therule test cases 114 andtest scenarios 116 defined atblock 208 above are stored, e.g., in thedatabase 112. In an embodiment, a plurality of test case designers may be concurrently definingtest cases 114 andtest scenarios 116 by executing the test builder interface onseparate workstations 108, and may be concurrently storing thetest cases 114 and thetest scenarios 116 in thedatabase 112. - At
block 216, the rules are invoked on therules engine 120 based on thetest cases 114 and based on thetest scenarios 116. A single invocation point may be invoked at one time, providing appropriate input parameter values based on thetest case 114. Alternatively, a plurality of invocation points may be serially invoked by atest scenario 116, thereby providing appropriate input parameter values based on thetest cases 114 composing thetest scenario 116. Therules engine 120 may access the rule and/or rules out of a rule repository database (not shown), or the rule definition may be provided along with the input parameter values. In an embodiment, the rules are precompiled and/or preprocessed into a binary file format that is stored on theserver 118, for example in Fair Isaac binary (ADB) file format. - At
block 220, one or more service applications are simulated by thetest execution component 106. Therules engine 120 may interact with the service applications during the course of executing the rule under test. The service applications may comprise a portal server processor, a database management system, a messaging service, and other services. Atblock 224, the elapsed time of execution of the rule processing by therules engine 120 is optionally timed by thetest execution component 106. Atblock 228, the results of executing the rule and/or rules by therules engine 120, for example output parameter values and optionally elapsed time information is collected. Atblock 232, the results of executing the rule and/or rules are stored in thedatabase 112. The results may also be displayed by an interface, for example a web interface that may execute on theworkstation 108. - At
block 236, a success/fail report is optionally generated based on the results of executing the rule and/or rules. The expected output parameter values defined in thetest cases 114 are compared to the results to determine whether thesubject test cases 114 succeeded or failed. Summary statistics, for example total numbers of successes and failures, may be aggregated and placed in the success/fail report 124. The success/fail report 124 may also include information comparing the current test session results with earlier test results to report a test success trend. The success/fail report 124 may be stored in thedatabase 112 and may be displayed by an interface, for example a web interface that may execute on theworkstation 108. Theprocess 200 then exits. - Some aspects of the system described above, for example the
computer system 102, theworkstation 108, and theserver 118, may be implemented on any general-purpose computer with sufficient processing power, memory resources, and network throughput capability to handle the necessary workload placed upon it.FIG. 3 illustrates a typical, general-purpose computer system suitable for implementing one or more embodiments disclosed herein. Thecomputer system 380 includes a processor 382 (which may be referred to as a central processor unit or CPU) that is in communication with memory devices includingsecondary storage 384, read only memory (ROM) 386, random access memory (RAM) 388, input/output (I/O)devices 390, andnetwork connectivity devices 392. Theprocessor 382 may be implemented as one or more CPU chips. - The
secondary storage 384 is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device ifRAM 388 is not large enough to hold all working data.Secondary storage 384 may be used to store programs that are loaded intoRAM 388 when such programs are selected for execution. TheROM 386 is used to store instructions and perhaps data that are read during program execution.ROM 386 is a non-volatile memory device, which typically has a small memory capacity relative to the larger memory capacity ofsecondary storage 384. TheRAM 388 is used to store volatile data and perhaps to store instructions. Access to bothROM 386 andRAM 388 is typically faster than tosecondary storage 384. - I/
O devices 390 may include printers, video monitors, liquid crystal displays (LCDs), touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices. - The
network connectivity devices 392 may take the form of modems, modem banks, Ethernet cards, universal serial bus (USB) interface cards, serial interfaces, token ring cards, fiber distributed data interface (FDDI) cards, wireless local area network (WLAN) cards, radio transceiver cards such as code division multiple access (CDMA), global system for mobile communications (GSM), and/or worldwide interoperability for microwave access (WiMAX) radio transceiver cards, and other well-known network devices. Thesenetwork connectivity devices 392 may enable theprocessor 382 to communicate with an Internet or one or more intranets. With such a network connection, it is contemplated that theprocessor 382 might receive information from the network, or might output information to the network in the course of performing the above-described method steps. Such information, which is often represented as a sequence of instructions to be executed usingprocessor 382, may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave. - Such information, which may include data or instructions to be executed using
processor 382 for example, may be received from and outputted to the network, for example, in the form of a computer data baseband signal or signal embodied in a carrier wave. The baseband signal or signal embodied in the carrier wave generated by thenetwork connectivity devices 392 may propagate in or on the surface of electrical conductors, in coaxial cables, in waveguides, in optical media, for example optical fiber, or in the air or free space. The information contained in the baseband signal or signal embedded in the carrier wave may be ordered according to different sequences, as may be desirable for either processing or generating the information or transmitting or receiving the information. The baseband signal or signal embedded in the carrier wave, or other types of signals currently used or hereafter developed, referred to herein as the transmission medium, may be generated according to several methods well known to one skilled in the art. - The
processor 382 executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage 384),ROM 386,RAM 388, or thenetwork connectivity devices 392. While only oneprocessor 382 is shown, multiple processors may be present. Thus, while instructions may be discussed as executed by aprocessor 382, the instructions may be executed simultaneously, serially, or otherwise executed by one ormultiple processors 382. - While several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted or not implemented.
- In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and could be made without departing from the spirit and scope disclosed herein.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/323,707 US20090271351A1 (en) | 2008-04-29 | 2008-11-26 | Rules engine test harness |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US4881008P | 2008-04-29 | 2008-04-29 | |
US12/323,707 US20090271351A1 (en) | 2008-04-29 | 2008-11-26 | Rules engine test harness |
Publications (1)
Publication Number | Publication Date |
---|---|
US20090271351A1 true US20090271351A1 (en) | 2009-10-29 |
Family
ID=41215889
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/257,782 Abandoned US20090271214A1 (en) | 2008-04-29 | 2008-10-24 | Rules engine framework |
US12/323,707 Abandoned US20090271351A1 (en) | 2008-04-29 | 2008-11-26 | Rules engine test harness |
US12/416,746 Active 2030-01-23 US8103683B2 (en) | 2008-04-29 | 2009-04-01 | Converting between software objects |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/257,782 Abandoned US20090271214A1 (en) | 2008-04-29 | 2008-10-24 | Rules engine framework |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/416,746 Active 2030-01-23 US8103683B2 (en) | 2008-04-29 | 2009-04-01 | Converting between software objects |
Country Status (1)
Country | Link |
---|---|
US (3) | US20090271214A1 (en) |
Cited By (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090271214A1 (en) * | 2008-04-29 | 2009-10-29 | Affiliated Computer Services, Inc. | Rules engine framework |
US7836346B1 (en) * | 2007-06-11 | 2010-11-16 | Oracle America, Inc. | Method and system for analyzing software test results |
US20110072027A1 (en) * | 2009-09-22 | 2011-03-24 | Siemens Product Lifecycle Management Software Inc. | System and method for customized file comparison |
US20130042222A1 (en) * | 2011-08-08 | 2013-02-14 | Computer Associates Think, Inc. | Automating functionality test cases |
US8429391B2 (en) | 2010-04-16 | 2013-04-23 | Micron Technology, Inc. | Boot partitions in memory devices and systems |
US8660861B2 (en) | 2012-01-24 | 2014-02-25 | Fmr Llc | Allocation of financial incentives for employee wellness programs |
US20140236662A1 (en) * | 2013-02-15 | 2014-08-21 | Mckesson Financial Holdings | Apparatuses, systems, and methods for providing a rules engine system |
US20150249823A1 (en) * | 2014-02-28 | 2015-09-03 | Airbus Helicopters | Method of testing an electronic system |
US20150278059A1 (en) * | 2008-06-04 | 2015-10-01 | Oracle International Corporation | System and method for supporting a sliding window for testing an event processing system |
US9208045B2 (en) | 2011-03-03 | 2015-12-08 | Hewlett-Packard Development Company, L.P. | Testing integrated business systems |
US10102091B2 (en) | 2008-06-04 | 2018-10-16 | Oracle International Corporation | System and method for supporting a testing framework for an event processing system using multiple input event streams |
US10146678B2 (en) | 2014-05-15 | 2018-12-04 | Oracle International Corporation | Test bundling and batching optimizations |
CN111552645A (en) * | 2020-04-29 | 2020-08-18 | 周赟 | Open type safety compliance permeability testing system |
CN112015636A (en) * | 2020-07-14 | 2020-12-01 | 北京淇瑀信息科技有限公司 | Decision engine testing method and device based on support vector machine and electronic equipment |
US11080178B2 (en) | 2018-12-28 | 2021-08-03 | Paypal, Inc. | Rules testing framework |
US20220138089A1 (en) * | 2020-10-30 | 2022-05-05 | EMC IP Holding Company LLC | Smart test executor |
US11645195B1 (en) * | 2020-02-18 | 2023-05-09 | Wells Fargo Bank, N.A. | Auto-decisioning test interface and test database for bypassing functionalities of decision engines and simulating return values |
CN116340187A (en) * | 2023-05-25 | 2023-06-27 | 建信金融科技有限责任公司 | Rule engine migration test method and device, electronic equipment and storage medium |
Families Citing this family (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110055107A1 (en) * | 2009-09-03 | 2011-03-03 | Von Unwerth Catherine D | Industry standards modeling systems and methods |
DE102010021382A1 (en) * | 2010-05-25 | 2011-12-01 | Abb Ag | Method and system for generating an integration model |
US8494997B2 (en) | 2010-07-20 | 2013-07-23 | Samuel W. Bellamy, III | System and method for validation of transaction data |
US8505047B2 (en) | 2010-11-20 | 2013-08-06 | Motorola Solutions, Inc. | Method and system for policy-based re-broadcast video on demand service |
US8621074B2 (en) | 2012-04-27 | 2013-12-31 | Xerox Business Services, Llc | Intelligent work load manager |
US10621523B1 (en) * | 2012-05-09 | 2020-04-14 | Parvinder CHADHA | Computer methods and computer systems for facilitating business rule generation and validation |
US20140122518A1 (en) * | 2012-10-29 | 2014-05-01 | Hewlett-Packard Development Company, L.P. | Codeless array validation |
US20150032907A1 (en) * | 2013-07-26 | 2015-01-29 | Alcatel-Lucent Canada, Inc. | Universal adapter with context-bound translation for application adaptation layer |
US20150160938A1 (en) * | 2013-12-10 | 2015-06-11 | Fmr Llc | Automation in rule engine in an integration development environment |
US20150331875A1 (en) * | 2014-05-16 | 2015-11-19 | Syntel, Inc. | System and method for validating integrated data recasting objects |
CN110209575A (en) * | 2019-05-20 | 2019-09-06 | 深圳壹账通智能科技有限公司 | Test document generation method, device, electronic equipment and storage medium |
CN110674047B (en) * | 2019-09-26 | 2023-06-27 | 北京字节跳动网络技术有限公司 | Software testing method and device and electronic equipment |
CN113176995B (en) * | 2021-04-28 | 2024-03-19 | 平安银行股份有限公司 | Interface testing method, device, equipment and medium based on testing scene |
US20230385544A1 (en) * | 2022-04-19 | 2023-11-30 | Tencent America LLC | Tools for conformance of network-based media processing (nbmp) documents and entities |
Citations (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5786816A (en) * | 1995-10-20 | 1998-07-28 | Araxsys, Inc. | Method and apparatus for graphical user interface-based and variable result healthcare plan |
US5881219A (en) * | 1996-12-26 | 1999-03-09 | International Business Machines Corporation | Random reliability engine for testing distributed environments |
US6039688A (en) * | 1996-11-01 | 2000-03-21 | Salus Media Inc. | Therapeutic behavior modification program, compliance monitoring and feedback system |
US6283761B1 (en) * | 1992-09-08 | 2001-09-04 | Raymond Anthony Joao | Apparatus and method for processing and/or for providing healthcare information and/or healthcare-related information |
US6343271B1 (en) * | 1998-07-17 | 2002-01-29 | P5 E.Health Services, Inc. | Electronic creation, submission, adjudication, and payment of health insurance claims |
US20020091968A1 (en) * | 2001-01-08 | 2002-07-11 | Donald Moreaux | Object-oriented data driven software GUI automated test harness |
US20030037314A1 (en) * | 2001-08-01 | 2003-02-20 | International Business Machines Corporation | Method and apparatus for testing and evaluating a software component using an abstraction matrix |
US20030069760A1 (en) * | 2001-10-04 | 2003-04-10 | Arthur Gelber | System and method for processing and pre-adjudicating patient benefit claims |
US20030233253A1 (en) * | 2002-04-30 | 2003-12-18 | Peth Thomas C. | Point-of-care clinical documentation software system and associated methods |
US20040078228A1 (en) * | 2002-05-31 | 2004-04-22 | Fitzgerald David | System for monitoring healthcare patient encounter related information |
US20050010436A1 (en) * | 2003-07-08 | 2005-01-13 | Richard Merkin | Health care administration method |
US20050144529A1 (en) * | 2003-10-01 | 2005-06-30 | Helmut Gotz | Method for defined derivation of software tests from use cases |
US6915265B1 (en) * | 1997-10-29 | 2005-07-05 | Janice Johnson | Method and system for consolidating and distributing information |
US20050166094A1 (en) * | 2003-11-04 | 2005-07-28 | Blackwell Barry M. | Testing tool comprising an automated multidimensional traceability matrix for implementing and validating complex software systems |
US6934934B1 (en) * | 1999-08-30 | 2005-08-23 | Empirix Inc. | Method and system for software object testing |
US20050209893A1 (en) * | 2004-03-18 | 2005-09-22 | Nahra John S | System and method for identifying and servicing medically uninsured persons |
US6993748B2 (en) * | 2001-10-26 | 2006-01-31 | Capital One Financial Corporation | Systems and methods for table driven automation testing of software programs |
US20060052965A1 (en) * | 2004-08-13 | 2006-03-09 | International Business Machines Corporation | Event driven testing method, system and program product |
US7039628B2 (en) * | 2004-04-21 | 2006-05-02 | Logan Jr Carmen | Portable health care history information system |
US20060129992A1 (en) * | 2004-11-10 | 2006-06-15 | Oberholtzer Brian K | Software test and performance monitoring system |
US20060143050A1 (en) * | 2004-12-27 | 2006-06-29 | The Trizetto Group, Inc. | Healthcare management system using patient profile data |
US20060224719A1 (en) * | 2005-03-30 | 2006-10-05 | Integrated Informatics, Inc. | Operator simulator and non-invasive interface engine |
US20060235280A1 (en) * | 2001-05-29 | 2006-10-19 | Glenn Vonk | Health care management system and method |
US20070006037A1 (en) * | 2005-06-29 | 2007-01-04 | Microsoft Corporation | Automated test case result analyzer |
US20080097789A1 (en) * | 2006-10-24 | 2008-04-24 | Huffer Robert L | Quality Management Of Patient Data For Health Care Providers |
US20080275729A1 (en) * | 2007-04-09 | 2008-11-06 | Nina Mithi Taggart | System and method for population health management |
US20080312951A1 (en) * | 2004-08-25 | 2008-12-18 | Berd Herpichboehm | Method for Optimizing Design Delivery and Implementation of Innovative Products in Healthcare |
US20090271214A1 (en) * | 2008-04-29 | 2009-10-29 | Affiliated Computer Services, Inc. | Rules engine framework |
US7656802B2 (en) * | 2006-11-14 | 2010-02-02 | International Business Machines Corporation | Simulating services on demand |
US7917895B2 (en) * | 2001-07-27 | 2011-03-29 | Smartesoft, Inc. | Automated software testing and validation system |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6047284A (en) * | 1997-05-14 | 2000-04-04 | Portal Software, Inc. | Method and apparatus for object oriented storage and retrieval of data from a relational database |
US6429882B1 (en) * | 1999-03-15 | 2002-08-06 | Sun Microsystems, Inc. | User interface component |
US7617240B2 (en) * | 1999-05-04 | 2009-11-10 | Accenture Llp | Component based task handling during claim processing |
US7013284B2 (en) * | 1999-05-04 | 2006-03-14 | Accenture Llp | Component based interface to handle tasks during claim processing |
AU1188902A (en) * | 2000-10-11 | 2002-04-22 | Ralph A Korpman | System for communication of health care data |
US6931429B2 (en) * | 2001-04-27 | 2005-08-16 | Left Gate Holdings, Inc. | Adaptable wireless proximity networking |
US6865573B1 (en) * | 2001-07-27 | 2005-03-08 | Oracle International Corporation | Data mining application programming interface |
US6917944B1 (en) * | 2001-08-30 | 2005-07-12 | Cisco Technology, Inc. | Method and apparatus for configuring access to a plurality of data repositories |
WO2004086198A2 (en) * | 2003-03-24 | 2004-10-07 | Siebel Systems, Inc. | Common common object |
US7480894B2 (en) * | 2004-02-20 | 2009-01-20 | International Business Machines Corporation | Method and system for retaining formal data model descriptions between server-side and browser-side javascript objects |
US7756882B2 (en) * | 2004-10-01 | 2010-07-13 | Microsoft Corporation | Method and apparatus for elegant mapping between data models |
WO2006050056A2 (en) * | 2004-10-29 | 2006-05-11 | Upstream Software, Inc. | Transaction network |
US7440955B2 (en) * | 2005-01-14 | 2008-10-21 | Im2, Inc | Method and system to compare data objects |
EP1708099A1 (en) * | 2005-03-29 | 2006-10-04 | BRITISH TELECOMMUNICATIONS public limited company | Schema matching |
US20070157155A1 (en) * | 2005-12-30 | 2007-07-05 | Peters Eric C | System and method for software generation and execution |
US7747569B2 (en) * | 2006-09-22 | 2010-06-29 | Raytheon Company | Systems, methods, and language for selection and retrieval of information from databases |
US7870412B2 (en) * | 2006-09-28 | 2011-01-11 | Oracle International Corporation | Passing client or server instructions via synchronized data objects |
-
2008
- 2008-10-24 US US12/257,782 patent/US20090271214A1/en not_active Abandoned
- 2008-11-26 US US12/323,707 patent/US20090271351A1/en not_active Abandoned
-
2009
- 2009-04-01 US US12/416,746 patent/US8103683B2/en active Active
Patent Citations (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6283761B1 (en) * | 1992-09-08 | 2001-09-04 | Raymond Anthony Joao | Apparatus and method for processing and/or for providing healthcare information and/or healthcare-related information |
US5786816A (en) * | 1995-10-20 | 1998-07-28 | Araxsys, Inc. | Method and apparatus for graphical user interface-based and variable result healthcare plan |
US6039688A (en) * | 1996-11-01 | 2000-03-21 | Salus Media Inc. | Therapeutic behavior modification program, compliance monitoring and feedback system |
US5881219A (en) * | 1996-12-26 | 1999-03-09 | International Business Machines Corporation | Random reliability engine for testing distributed environments |
US6915265B1 (en) * | 1997-10-29 | 2005-07-05 | Janice Johnson | Method and system for consolidating and distributing information |
US6343271B1 (en) * | 1998-07-17 | 2002-01-29 | P5 E.Health Services, Inc. | Electronic creation, submission, adjudication, and payment of health insurance claims |
US6934934B1 (en) * | 1999-08-30 | 2005-08-23 | Empirix Inc. | Method and system for software object testing |
US20020091968A1 (en) * | 2001-01-08 | 2002-07-11 | Donald Moreaux | Object-oriented data driven software GUI automated test harness |
US20060235280A1 (en) * | 2001-05-29 | 2006-10-19 | Glenn Vonk | Health care management system and method |
US7917895B2 (en) * | 2001-07-27 | 2011-03-29 | Smartesoft, Inc. | Automated software testing and validation system |
US20030037314A1 (en) * | 2001-08-01 | 2003-02-20 | International Business Machines Corporation | Method and apparatus for testing and evaluating a software component using an abstraction matrix |
US20030069760A1 (en) * | 2001-10-04 | 2003-04-10 | Arthur Gelber | System and method for processing and pre-adjudicating patient benefit claims |
US6993748B2 (en) * | 2001-10-26 | 2006-01-31 | Capital One Financial Corporation | Systems and methods for table driven automation testing of software programs |
US20030233253A1 (en) * | 2002-04-30 | 2003-12-18 | Peth Thomas C. | Point-of-care clinical documentation software system and associated methods |
US20040078228A1 (en) * | 2002-05-31 | 2004-04-22 | Fitzgerald David | System for monitoring healthcare patient encounter related information |
US20050010436A1 (en) * | 2003-07-08 | 2005-01-13 | Richard Merkin | Health care administration method |
US20050144529A1 (en) * | 2003-10-01 | 2005-06-30 | Helmut Gotz | Method for defined derivation of software tests from use cases |
US20050166094A1 (en) * | 2003-11-04 | 2005-07-28 | Blackwell Barry M. | Testing tool comprising an automated multidimensional traceability matrix for implementing and validating complex software systems |
US20050209893A1 (en) * | 2004-03-18 | 2005-09-22 | Nahra John S | System and method for identifying and servicing medically uninsured persons |
US7039628B2 (en) * | 2004-04-21 | 2006-05-02 | Logan Jr Carmen | Portable health care history information system |
US20060052965A1 (en) * | 2004-08-13 | 2006-03-09 | International Business Machines Corporation | Event driven testing method, system and program product |
US20080312951A1 (en) * | 2004-08-25 | 2008-12-18 | Berd Herpichboehm | Method for Optimizing Design Delivery and Implementation of Innovative Products in Healthcare |
US20060129992A1 (en) * | 2004-11-10 | 2006-06-15 | Oberholtzer Brian K | Software test and performance monitoring system |
US20060143050A1 (en) * | 2004-12-27 | 2006-06-29 | The Trizetto Group, Inc. | Healthcare management system using patient profile data |
US20060224719A1 (en) * | 2005-03-30 | 2006-10-05 | Integrated Informatics, Inc. | Operator simulator and non-invasive interface engine |
US20070006037A1 (en) * | 2005-06-29 | 2007-01-04 | Microsoft Corporation | Automated test case result analyzer |
US20080097789A1 (en) * | 2006-10-24 | 2008-04-24 | Huffer Robert L | Quality Management Of Patient Data For Health Care Providers |
US7656802B2 (en) * | 2006-11-14 | 2010-02-02 | International Business Machines Corporation | Simulating services on demand |
US20080275729A1 (en) * | 2007-04-09 | 2008-11-06 | Nina Mithi Taggart | System and method for population health management |
US20090271214A1 (en) * | 2008-04-29 | 2009-10-29 | Affiliated Computer Services, Inc. | Rules engine framework |
Cited By (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7836346B1 (en) * | 2007-06-11 | 2010-11-16 | Oracle America, Inc. | Method and system for analyzing software test results |
US20090271214A1 (en) * | 2008-04-29 | 2009-10-29 | Affiliated Computer Services, Inc. | Rules engine framework |
US20150278059A1 (en) * | 2008-06-04 | 2015-10-01 | Oracle International Corporation | System and method for supporting a sliding window for testing an event processing system |
US10140196B2 (en) | 2008-06-04 | 2018-11-27 | Oracle International Corporation | System and method for configuring a sliding window for testing an event processing system based on a system time |
US10102091B2 (en) | 2008-06-04 | 2018-10-16 | Oracle International Corporation | System and method for supporting a testing framework for an event processing system using multiple input event streams |
US9892009B2 (en) * | 2008-06-04 | 2018-02-13 | Oracle International Corporation | System and method for supporting a sliding window for testing an event processing system |
US9753825B2 (en) | 2008-06-04 | 2017-09-05 | Oracle International Corporation | System and method for using an event window for testing an event processing system |
US20110072027A1 (en) * | 2009-09-22 | 2011-03-24 | Siemens Product Lifecycle Management Software Inc. | System and method for customized file comparison |
US8458204B2 (en) * | 2009-09-22 | 2013-06-04 | Siemens Product Lifecycle Management Software Inc. | System and method for customized file comparison |
US8762703B2 (en) | 2010-04-16 | 2014-06-24 | Micron Technology, Inc. | Boot partitions in memory devices and systems |
US9342371B2 (en) | 2010-04-16 | 2016-05-17 | Micron Technology, Inc. | Boot partitions in memory devices and systems |
US8429391B2 (en) | 2010-04-16 | 2013-04-23 | Micron Technology, Inc. | Boot partitions in memory devices and systems |
US9208045B2 (en) | 2011-03-03 | 2015-12-08 | Hewlett-Packard Development Company, L.P. | Testing integrated business systems |
US9990274B2 (en) | 2011-03-03 | 2018-06-05 | Entit Software Llc | Testing integrated business systems |
US9477583B2 (en) | 2011-08-08 | 2016-10-25 | Ca, Inc. | Automating functionality test cases |
US8893087B2 (en) * | 2011-08-08 | 2014-11-18 | Ca, Inc. | Automating functionality test cases |
US20130042222A1 (en) * | 2011-08-08 | 2013-02-14 | Computer Associates Think, Inc. | Automating functionality test cases |
US8660861B2 (en) | 2012-01-24 | 2014-02-25 | Fmr Llc | Allocation of financial incentives for employee wellness programs |
US9990598B2 (en) * | 2013-02-15 | 2018-06-05 | Allscripts Software, Llc | Apparatuses, systems, and methods for providing a rules engine system |
US20140236662A1 (en) * | 2013-02-15 | 2014-08-21 | Mckesson Financial Holdings | Apparatuses, systems, and methods for providing a rules engine system |
US9288483B2 (en) * | 2014-02-28 | 2016-03-15 | Airbus Helicopters | Method of testing an electronic system |
US20150249823A1 (en) * | 2014-02-28 | 2015-09-03 | Airbus Helicopters | Method of testing an electronic system |
US10146678B2 (en) | 2014-05-15 | 2018-12-04 | Oracle International Corporation | Test bundling and batching optimizations |
US10802955B2 (en) | 2014-05-15 | 2020-10-13 | Oracle International Corporation | Test bundling and batching optimizations |
US11080178B2 (en) | 2018-12-28 | 2021-08-03 | Paypal, Inc. | Rules testing framework |
US11645195B1 (en) * | 2020-02-18 | 2023-05-09 | Wells Fargo Bank, N.A. | Auto-decisioning test interface and test database for bypassing functionalities of decision engines and simulating return values |
CN111552645A (en) * | 2020-04-29 | 2020-08-18 | 周赟 | Open type safety compliance permeability testing system |
CN112015636A (en) * | 2020-07-14 | 2020-12-01 | 北京淇瑀信息科技有限公司 | Decision engine testing method and device based on support vector machine and electronic equipment |
US20220138089A1 (en) * | 2020-10-30 | 2022-05-05 | EMC IP Holding Company LLC | Smart test executor |
CN116340187A (en) * | 2023-05-25 | 2023-06-27 | 建信金融科技有限责任公司 | Rule engine migration test method and device, electronic equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
US20090271214A1 (en) | 2009-10-29 |
US8103683B2 (en) | 2012-01-24 |
US20090271426A1 (en) | 2009-10-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20090271351A1 (en) | Rules engine test harness | |
Kumeno | Sofware engneering challenges for machine learning applications: A literature review | |
US11791021B2 (en) | System and method for building intuitive clinical trial applications | |
US6609128B1 (en) | Codes table framework design in an E-commerce architecture | |
US6601233B1 (en) | Business components framework | |
US6633878B1 (en) | Initializing an ecommerce database framework | |
US6718535B1 (en) | System, method and article of manufacture for an activity framework design in an e-commerce based environment | |
US7100195B1 (en) | Managing user information on an e-commerce system | |
US6704873B1 (en) | Secure gateway interconnection in an e-commerce based environment | |
CN109344170B (en) | Stream data processing method, system, electronic device and readable storage medium | |
US10079858B2 (en) | Managing access in one or more computing systems | |
US20220318004A1 (en) | System and method for predicting the impact of source code modification based on historical source code modifications | |
CN108647141A (en) | Automatic test approach, device, computer-readable medium and electronic equipment | |
US20210173762A1 (en) | Dynamic integration testing | |
CN106873957A (en) | The processing method and equipment of a kind of operation flow | |
CN109977012B (en) | Joint debugging test method, device, equipment and computer readable storage medium of system | |
US11544055B2 (en) | System and method for identifying source code defect introduction during source code modification | |
US20210117313A1 (en) | Language agnostic automation scripting tool | |
CN107526676A (en) | A kind of cross-system method of testing and device | |
US20070225943A1 (en) | Executable application operation monitoring system | |
Vargas et al. | Enabling real-time feedback in software engineering | |
Li et al. | Testing machine learning systems in industry: an empirical study | |
Lakshmanan et al. | A business centric end-to-end monitoring approach for service composites | |
Fagerström et al. | Verdict machinery: On the need to automatically make sense of test results | |
Kim et al. | Machine learning frameworks for automated software testing tools: a study |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: AFFILIATED COMPUTER SERVICES, INC., TEXAS Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KANDASAMY, UMA MAHESWARI;DATLA, KRISHNAM RAJU B.;REEL/FRAME:021895/0484 Effective date: 20081118 |
|
AS | Assignment |
Owner name: AFFILIATED COMPUTER SERVICES, LLC, TEXAS Free format text: CHANGE OF NAME;ASSIGNOR:AFFILIATED COMPUTER SERVICES, INC.;REEL/FRAME:028994/0470 Effective date: 20111216 |
|
AS | Assignment |
Owner name: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT, NEW YORK Free format text: SECURITY AGREEMENT;ASSIGNORS:XEROX COMMERCIAL SOLUTIONS, LLC (F/K/A ACS COMMERCIAL SOLUTIONS, INC.);XEROX STATE & LOCAL SOLUTIONS, INC. (F/K/A ACS STATE AND LOCAL SOLUTIONS, INC.);RSA MEDICAL LLC;AND OTHERS;REEL/FRAME:040905/0458 Effective date: 20161207 Owner name: JPMORGAN CHASE BANK, N.A., AS ADMINISTRATIVE AGENT Free format text: SECURITY AGREEMENT;ASSIGNORS:XEROX COMMERCIAL SOLUTIONS, LLC (F/K/A ACS COMMERCIAL SOLUTIONS, INC.);XEROX STATE & LOCAL SOLUTIONS, INC. (F/K/A ACS STATE AND LOCAL SOLUTIONS, INC.);RSA MEDICAL LLC;AND OTHERS;REEL/FRAME:040905/0458 Effective date: 20161207 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: CONDUENT HEALTH ASSESSMENTS, LLC, NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180 Effective date: 20211015 Owner name: CONDUENT CASUALTY CLAIMS SOLUTIONS, LLC, NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180 Effective date: 20211015 Owner name: CONDUENT BUSINESS SOLUTIONS, LLC, NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180 Effective date: 20211015 Owner name: CONDUENT COMMERCIAL SOLUTIONS, LLC, NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180 Effective date: 20211015 Owner name: ADVECTIS, INC., GEORGIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180 Effective date: 20211015 Owner name: CONDUENT TRANSPORT SOLUTIONS, INC., NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180 Effective date: 20211015 Owner name: CONDUENT STATE & LOCAL SOLUTIONS, INC., NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180 Effective date: 20211015 Owner name: CONDUENT BUSINESS SERVICES, LLC, NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:057969/0180 Effective date: 20211015 |