WO2005121964A1

WO2005121964A1 - Method for automatically testing computer programs

Info

Publication number: WO2005121964A1
Application number: PCT/EP2005/006084
Authority: WO
Inventors: Karsten Hannig
Original assignee: T-Mobile Deutschland Gmbh
Priority date: 2004-06-11
Filing date: 2005-06-07
Publication date: 2005-12-22
Also published as: DE102004028526A1

Abstract

The invention relates to a method for automatically testing computer programs, comprising the following steps: a test program is started on a computer system; user activities on the computer system are recorded, said user activities corresponding to a predefined test sequence; a computer program that is to be tested is started; the previously recorded user activities are automatically processed; and the actions of the computer system and the computer program that is to be tested are automatically recorded as a result of the user activities.

Description

Process for the automated execution of tests on computer programs

The invention relates to a method for the automated execution of tests on computer programs. The method, which is carried out in the form of a computer program, is used to test other computer programs whose user interface uses the widespread X11 protocol for display. Most of these are UNIX® systems. The so-called X protocol is a standardized protocol for data exchange between the X server and the X client over a network. The X protocol supports the application on an X client through the representation provided by the X server.

Computer programs and Internet applications used by companies must be checked for correct functioning and compatibility with other applications. However, "manual" test methods that have been carried out hitherto involve a high expenditure of personnel and time. As a result, tests are often not carried out to the necessary extent. It is therefore the object of the present invention to specify a method for the automated execution of tests on computer programs with which computer programs automatically and can be checked with little effort in terms of personnel and costs This object is achieved according to the invention by the method specified in claim 1.

Advantageous refinements and preferred developments of the invention are specified in the dependent claims.

According to the invention, the method itself is carried out as a computer program and is designed such that it does the user activities on the one hand and the system activities thereby triggered by the program to be tested on the other hand records. The recorded user activities can then be repeated fully automatically and as often as required without user intervention and applied to a computer program to be tested.

User activities associated with the use of the programs to be tested are in the form of keystrokes, mouse movements,

Mouse button operations, etc., recorded or formulated in the form of a sequence language. These user activities are then applied to a program to be tested. The behavior of the program to be tested in response to the automatically running, previously defined user activities is recorded. This can be expenditure or activities of the program to be tested on the X server.

The recorded activities of the tested program are automatically compared with the corresponding specifications in order to check the correct operation of the program. Of course, a "manual" evaluation of the result data is also possible, in particular if irregularities are found during the automatic test, which require a thorough review of the program.

Furthermore, the records can be used for documentation purposes by evaluating the actions and reactions of the computer program to be tested and / or the behavior of the computer system.

The possible uses of the invention extend to the testing and documentation of computer programs in general and in particular of X-protocol-based surfaces. The invention enables a recording, processing and synchronization of user activities and thereby caused actions and reactions of the program or computer system to be tested. In particular, the invention offers a

• Automation of computer processes, which are originally carried out manually by the user for - test purposes - mass data processing • recording of user actions for documentation purposes • obtaining program documentation (mask screenshots)

The test program is preferably based on a program code based on OpenSource.

Post-processing of the recording of both the user activities and the system reactions is part of the process.

When "playing" or processing the recorded user activities, the behavior of a user is modeled, ie the test program waits at the critical points for certain system reactions of the program to be tested, such as opening input masks, windows, texts, etc., before it automatically performs further entries or mouse clicks.

In the case of the X protocol, the system responses of the computer system to be tested essentially comprise image data, an analysis and detection of errors being carried out on the basis of a fuzzy comparison of the image data with comparison image data.

As a result, faulty program behavior and slow program execution can be determined immediately by the test program. By using this test method according to the invention, the personnel expenditure when testing computer programs can be greatly reduced. Tests are now possible that were previously impossible due to the high personnel costs. In a preferred embodiment of the invention, it is provided that the processes of several test procedures running simultaneously can be synchronized with one another.

An embodiment of the invention is described below. The following abbreviations and terms are used:

Record the test cases

Good preparation is required to record the test cases. The test cases must be fully planned! Each entry should be carried out "dry" beforehand, the tester / test user should be trained to operate the test object.

Use of test data

Each test section should be finished in exactly the state in which the test section started. This is the only way to run test sections in loops multiple times. If there are clear key fields here it is important to have a sufficiently large test data pool. A different pool record can then be used for each run.

Each test section should be broken down into small, manageable sequences that are recorded one after the other (but as separate files). This has the advantage that you can use the individual scripts to jump to different points in the test process if necessary, and not have to repeat the whole test case if something goes wrong.

Special care should be taken when recording the sequences, as typing and operating errors and their correction must be manually removed from the recording afterwards. The speed at which a sequence is recorded is completely irrelevant. So you can take your time.

The individual sequences should be linked together as modules. Ideally in a hierarchical form. The "Include" script command is intended for precisely this purpose. If more complex functions are required, such as variable substitution or a connection to a database, the "Exec" or "Pipeln" script commands can be used to pass the processing on to external components With the command "ReadString" data can be read from an ASC file. The process control can only be outsourced with great effort, since the called components cannot easily recognize the status of the processing.

A screen mask or a screen window does not know the state "finished". This means that the test program can never recognize whether what is displayed on the monitor is already complete or whether we (as a user) are still waiting for something To inform the test program that we feel the window is "complete", the so-called synchronization is required. At the beginning of the synchronization, a "sync button" is set with which this synchronization is triggered. During synchronization, part of the monitor display is saved in a file. When it is played, the current monitor display is regularly compared with this file. If it matches, the software "knows" that the output expected by the user has arrived and continues processing.

The test program can be operated from an ASC terminal while the test object is being executed on an X terminal. This enables a nice control of the results. The scripts generated by the test program are very long, not structured and confusing. Therefore, every script should be post-processed with a post-processing module. A post-processing module combines the recorded actions into meaningful blocks and pre-processes the synchronization points. A sequential control system must be installed manually.

Example of a recording

Carried out on a RedHat 6.2 Linux computer with KDE 2.0 as an interface as a test object and as a test computer. The test computer was accessed from another PC via a network connection

Testling We are logged on to the X console. The desktop is empty (except for our standard icons) We open a shell window and enter: xhost + <RETURN> exit <RETURN> Test computer We are in the directory $ HOME / xnitt- 1.00_install / xnitt_intro (except for the subdirectory "snapshot "empty). Test computer <Testling> is to be replaced by the network name. We enter: export DISPLAY = <Testling>: 0.0 xnrec -s> 1st test.rec

Test computer display XServer has version XXXXXXXX. xtest extension version XXXXX Press the stop button with which the recording is to be ended (this button must otherwise be unused)

Testling We press the WINDOWS key (or ALT-ünks)

Test computer display The stop key has the keycode 115 Press the sync key with which the screen synchronization should be marked (This key must otherwise be unused)

Testling We press the Home key (or SHIFT links)

Test computer display The sync key has the keycode 97 Start of the recording Info: Events were ignored (4) ... (the recording starts) Tip: We can transfer the displayed key codes to xnrec later with the parameters -k and -y, and save then these steps Testling We click on the shell symbol at the bottom of the screen Testling A shell window opens (important: top left on the desk) Testling We move the mouse to the LO corner of the shell window and press the test computer sync button 0: 0 as the first corner noted Testling We move the mouse approx. 2 cm down to the right and press the sync key again Test computer 57:58 noted as the second corner. Scheduled synchronization Testling We click on "File" in this window. Then "Linux Console" (the color of the window changes)

Testling input: Is -I <RETURN>

Testling input: exit <RETURN> (the shell window becomes bright again input: exit <RETURN> (the shell window disappears)

Testling We press the stop button

Test computer program is terminated ... execution terminated.

Here is an excerpt from the recorded script

MouseMove 106 739 Mouse + 1 Delay 70 Mouse- 1 Delay 4600 Delay 26010 Sync 3 0 0 ScreenshotOOLxwd Delay 43960 MouseMove 12 28 Mouse + 1 Mouse- 1 Delay 25960 MouseMove 45 50 Mouse + 1 Delay 140 Mouse- 1 Delay 16510 KeyName + I Delay 110 KeyName- 1 Delay 50 KeyName + s Delay 90 KeyName- s Delay 250 KeyName + space Play the recording

We make the entries on the test computer, and we observe the reactions on the test object. If we have done everything correctly, we can watch the process. Our activities are repeated.

The above Script is rather confusing. Finding our activities in this script requires some practice. In addition, the process is quite slow.

A log is displayed during execution:

Post-processing with the post-processing module

The recorded raw script is rather unsuitable for automated playback, but still contains all activities. In order to facilitate further processing, we have to rework the script. This can be done automatically by the post-processing module. After the post-processing program has run, the following new file results:

Echo Echo \ # 1ter_test.script

Echo MouseMove 110 749

Delay 60 MouseClick 1

Delay 5940 Sync 1 0 0 XTermJDffen.xwd OnSucc Adjust

OnErr AdjustScreen 0 0 XTerm_Offen.xwd

OnErr ScreenDump Error_XTerm_Offen_% Y% m% d% H% M% S.xwd

OnErr Goto EndOfThisScript

Delay 4290 MouseMove 18 26 MouseClick 1

Delay 1620 MouseMove 46 53

Delay 190 MouseClick 1

Delay 2750 String Is -l \ r

Delay 4830 String exitλr

Delay 2940 String exit \ r: EndOfThisScript

In this form, the script is much easier to understand and we can make further changes.

Insert-in-Cut

Videographers already know this feature: You play a film up to a position X and record new film scenes at this point. This is also possible with the test program. If you add the following line to a script at any point, the script will be processed to that point and then the recording will start. After the end of the recording the script continues:

PushAdjust

Echo Attention Recording!

EXEC xnrec -s> Neueaufnahme.rec

PopAdjust

If you have reworked the new script (in the example "NeueAufnahme.rec") you can easily integrate it into the existing test case: Add the following lines to the existing script:

PushAdjust ResetAdjust include NeueAufnahme.scr PopAdjust

Of course, you can also include the Adjust statements in the new script, which makes the "father script" easier to understand. In this case, the Adjust statements are used to "decouple" the scripts. As an alternative to this "decoupling", xnrec can be called with the parameter -adj. This procedure is particularly suitable for generating modular test cases.

Of course, the entire spectrum of possibilities of the method according to the invention cannot be illustrated using the examples shown here. With each test, you will have to consider whether the use of a tool or a capture / replay tool as described here justifies the additional effort that the use of the tool inevitably entails. The use of such a tool is always worthwhile if

• Test cases have a lot of steps that have to be carried out in a certain order • Test cases often have to be repeated (or modified) (troubleshooting !, verification!) • Many test cases have to run simultaneously (load / stress tests)

Claims

claims

1. A method for the automated execution of tests on computer programs, comprising the steps of: starting a test program on a computer system, recording user activities on the computer system which correspond to a predetermined test sequence, starting a computer program to be tested, automatic processing of the previously recorded user activities, and automatic recording the reactions of the computer program to be tested and the computer system as a result of the automated user activities.

2. The method according to claim 1, characterized in that the program to be tested is started and the system responses to the user activities are obtained and recorded.

3. The method according to claim 1 or 2, characterized in that the user activities include user input using the keyboard and / or computer mouse.

4. The method according to claim 3, characterized in that the time sequence and the time interval of the keyboard and / or computer mouse activities are recorded.

5. The method according to any one of the preceding claims, characterized in that the recorded user activities are editable.

6. The method according to any one of the preceding claims, characterized in that the sequence of automated user activities are formulated in the form of a computer language.

, Method according to one of the preceding claims, characterized in that the reproduction of the recorded user activities is based on the behavior of a user in relation to their chronological sequence.

8. The method according to any one of the preceding claims, characterized in that the processes of several test procedures running simultaneously can be synchronized with one another.

9. The method according to any one of the preceding claims, characterized in that the recorded actions of the computer program to be tested and the computer system are evaluated for documentation purposes.

10. The method according to any one of the preceding claims, characterized in that the recorded actions of the computer program to be tested and the computer system are evaluated for incorrect behavior.

11. The method according to any one of the preceding claims, characterized in that the system responses of the computer system to be tested comprise image data, an analysis and detection of errors being carried out on the basis of a fuzzy comparison of the image data.