WO2000077672A2

WO2000077672A2 - Method and system for skills assessment

Info

Publication number: WO2000077672A2
Application number: PCT/US2000/016547
Authority: WO
Inventors: Robert W. Hatton
Original assignee: Knowledgebase.Com
Priority date: 1999-06-16
Filing date: 2000-06-15
Publication date: 2000-12-21
Also published as: WO2000077672A3; AU5742000A

Abstract

Skill assessment exams are administered to prospective employment candidates (140) for clients, preferably over a network (110) such as the Internet. A server (120) is connected to a database of skill assessment exams so that candidates, using a browser and a client-supplied code, can access the server (120) over the network (110) in order to take a selected exam (130). The server (120) immediately grades the exams and notifies the client (100) when a candidate (140) has completed an assessment exam. Results can be viewed and managed by the client (100) over the network (110). Clients (100) can select standard exams or create custom exams for prospective candidates (140).

Description

TITLE: METHOD AND SYSTEM FOR SKILLS ASSESSMENT

FIELD OF THE INVENTION The present invention is drawn to a system to allow clients, such as recruiters and prospective employers, to screen the skills of candidates without havmg to tie up their own employees More particularly, it is drawn to usmg the Internet to administer and evaluate the results of a broad database of skills assessment exams

BACKGROUND OF THE INVENTION Part of the duties of the human resources department in any corporation is the pre- screemng and hiring of company employees For the most part, members of the human resources department have been trained in the art of asking candidates the πght type of questions to grade their abilities as they apply to the position The challenge that has aπsen in the last ten years, though, is the rapid development of the technology infrastructure mside almost every major corporation The typical human resources manager is not capable of handling the challenges of qualifying technical candidates properly Most managers are very aware of this problem and are m need of systems that would overcome such deficiencies Most human resources managers who receive a degree m human resources at a major college or university have little if any formal training in qualifying computer technology personnel, for example The same problem aπses in any other specialized or techmcal field This challenge has put almost all human resource managers at a disadvantage, which in turn has put a great deal of pressure on the human resources department when making hiring decisions To date, human resources departments, or their equivalents, do not have a measurement tool at their disposal that would help them gauge the proficiency of candidates Most large corporations in the last seven or so years have acknowledged their human resources deficiencies, but the steps they are taking to combat the problems have been futile Corporations, in some cases, only expect their human resources managers to act as a pre- screening buffer, whose job is to give the candidate a "look-over" and, if acceptable, send the candidate to the respective department for a techmcal interview From that point, usually a team of employees from the department will spend time technically qualifying the candidate in a face to face interview If the group collectively feels that the candidate technically qualifies, the department hiring manager will meet the candidate for a techmcal screemng In the majority of hiring interview cases, the human resources manager lacks the ability to properly pre-screen the candidate's techmcal proficiencies, or lack thereof The only barometer that is available to measure the candidate's techmcal proficiency is the candidate's resume, which is often self-serving and circumstantial. This process has proved to be an ineffective method Existmg employees are being forced to interview numerous candidates before they find the πght person for their specific requirements This process is also causing a great deal of pressure between the human resources department and the departmental hiring managers, due to the amount of people and resources that the department has to dedicate to the mterview process, before finally findmg a qualified candidate The corporation as a whole suffers from this process, due to the amount of time and resources that the employees are surrendering each day to interview candidates The interviewing process interferes with the daily duties for a significant duration of time An mterview usually lasts somewhere between 45 minutes to an hour and a half per candidate This is obviously a very inadequate solution to an already difficult problem There is no justification for having a department that is obviously already strained, due to its need for an additional employee, compound the situation by pulling away its staff to conduct a seπes of interviews. This is especially true, and a major concern, when it applies to team interviews. For example, if a department uses a three-employee team to mterview six candidates with an average interview duration of one hour, the time cost to the department is over 18 hours of productive man time. Accordmgly, there is a need for a system to allow corporations and other prospective employers to screen the skills of applicants without having to tie up their own employees. Independent contract recruiters were also facing the same problem as corporations' human resources departments. Recruiters were also seeking a solution to their techmcal evaluation needs. With the advancements made m computer technology in the last few years, the utilization of an Internet type solution was sought There were some available sites, but they were deficient in meeting the needs. Some of the available sites only tested single topics, while others charged per single test administration These solutions were not adequate because the testing needed to address a broad range of skills in a multitude of different subject matters. Additionally, a per subject test fee simply was not practical or cost-effective Accordmgly, there was a need to develop a techmcal evaluation program tailored to fit a wide range of needs BRIEF SUMMARY OF THE INVENTION The present system uses the Internet to administer and evaluate a broad database of skills assessment exams, allowing recruiters and prospective employers to screen the skills of applicants without havmg to tie up their own employees

It is an object of the mvention to provide a system to allow recruiters and prospective employers to screen the skills of applicants without havmg to tie up their own employees It is a further object of the mvention to provide a means for usmg the Internet to administer and evaluate a broad database of skills assessment exams It is another object of the invention to provide a means for usmg the Internet to create, administer, and evaluate customized skills assessment exams It is another object of the invention to provide a means for automatically notifying a test creator when a candidate has completed an assessment exam It is yet another object of the invention to provide an exam process that is secure and tamper-resistant, yet allows candidates to take an exam from anywhere in the world via the Internet

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 illustrates a first embodiment of the present mvention Figure 2 discloses some examples of the subjects to be covered by the present mvention Figure 3 shows a possible grouping of exams for a techmcal support candidate in accordance with the present invention Figure 4 shows a groupmg of tests exemplary of the present mvention Figure 5 shows some of the tests for office skills of the present invention Figure 6 shows one sample of the results report of the present invention Figure 7 shows another sample of the results report of the present invention Figure 8 illustrates a page for viewing a selected exam and the associated candidates for the administration of the present invention Figure 9 illustrates an exam taker management area page of the present invention Figure 10 illustrates an exam modification or deletion page of the present mvention DETAILED DESCRIPTION OF THE INVENTION The present system starts with one or more databases of assessment exams. Exams covering similar subject matter can be grouped together on separate databases if desired for access speed or other functional reasons. Figure 2 gives some examples of the subjects to be covered. Additional subjects are added as the need arises or as the technology develops. This listing is intended to provide examples and not as limiting the scope of the invention. Some of these tests are grouped together for certain job specifications. As an example, figure 3 shows the possible grouping of exams for a technical support candidate. Figure 4 shows a grouping of tests designed to evaluate the skills and knowledge of those candidates applying for entry level positions in a computer or information technology group. While the system is particularly helpful for computer technology employment, it is also utilized for testing for various other skills such as office worker, medical staff, legal assistants, litigation support staff, accounting personnel, customer service and the like. Figure 5 shows some of the tests for office skills. The tests are designed to provide the most information in the shortest test period. Feedback from the test results is utilized to modify and optimize the test. Additionally, a customer or client can supply their own private test to be added to the database. The present system provides recruiters with the flexibility of designing a customized technical exam in a very short period of time. In accordance with one embodiment, the customization can take place in less than three minutes. The system provides an extremely user-friendly interface that requires only a very few steps to create an exam. The following outline describes the five easy steps to create one of the exams in accordance with one embodiment of the invention. 1. After entry into the Internet web site, the user would click on the "Create An Exam" icon and type in the usemame and password. The program will then generate a page that would allow the user to view every subject and skill level. 2. The user chooses the subjects and skill levels that are desired for the exam. In accordance with one embodiment of the invention, there are three skill levels for each exam. The user may give exams in all three skill levels if desired. 3. At the bottom of the page, the user clicks on "Create This Exam". The program will then generate an outline of the customized exam that was just created. The program will then ask if the user wishes to make any changes to the exam. 4. If the exam outline is acceptable, the user is prompted to give the exam a name in a special field. If the exam outline is not acceptable, the user clicks on a "Back" button on the desktop computer to make changes. 5. After all the changes are made, an "Okay" button is clicked by the user. The program will then give the user a code number which is utilized by the user to give to the candidates whenever the user wishes one candidate to take that specific exam. The program will also email the user this information for their records. A user may design as many exams as are desired by clicking on the "Create An Exam" icon. The above five steps are repeated for each type of exam created. A candidate can take the exam, once they have the code number, at any time by accessing the database from any site. Since the test does not have to be administered at the prospective employers site, taking the test can be done 24 hours a day, all days of the year. Accessing the test site over the Internet is the preferred method. This allows access by the largest number of potential candidates. Those candidates that do not have access to the Internet can still utilize the system at the employer's site or other sites available to the public. The ability to access the system over the Internet also bridges geographic boundaries. The test is available to the candidate at any time from anywhere in the world that has Internet connection. Once the candidate takes the exam, the test creator is automatically notified via email. The process allows users to customize their own exams. This is advantageous, because a recruiter cannot be an expert in every release of every product in every environment. The system of the present invention provides an online screening tool to combat that very challenge. With the present invention, a recruiter can design an exam that is specific to the particular technical needs. The system can handle a significant number of different subjects available in various difficulty levels. In accordance with one embodiment, a recruiter may utilize exams covering over 100 different subjects available in three different difficulty levels. The recruiter can pick and choose the skill requirements to match the needs. This flexibility gives the recruiter great advantage when interviewing a candidate. It allows the recruiter to already know where the weak points are with the candidate as they apply to the specific requirements. Additionally, others in the organization may have reviewed the technical results of the technical exams. The exams are structured in such a way as to provide additional validation to candidates' statements on their resumes regarding their technical abilities before an organization invests additional time and effort in the interview process The results of the exams m accordance with the present invention would eliminate wasted time in the interview process and, at a minimum, reduce the amount of time spent in interviewing by allowing some external feedback about the candidates pπor to the personal mterview process Additionally, the ability to verify a candidate's claims on their resume eliminates future problems in case the candidate has exaggerated or misrepresented their qualifications In accordance with the present invention, the exams are structured to give vaπous exams directed to vaπous levels of expertise in any one particular subject For example, three separate skill levels are provided for each subject These levels are put m place so that recruiters can measure their candidate at the true level that is desired for each specific position For example, when a requirement calls for a basic understanding of a subject matter, a beginner level or basic level exam is structured for these purposes On the other hand, if the position requires three years of heavy expeπence. for example, the basic level test would not provide the proper feedback for such a requirement In accordance with the present invention, the recruiter can design the exam around the specific levels of skill sets that are needed and save the time and aggravation of trying to decide if a candidate has enough ability to perform the specific tasks required As an example of the vaπous levels, most subjects can be tested at three levels: a basic level, an intermediate level, and an advanced level These levels can be modified m accordance with the subject to be tested For example, a basic level question is utilized when the applicant has to demonstrate that they understand the functionality of the subject or have a comprehension of how to use the fundamentals of the subject A basic skill set would be a person who has used the subject between one to six months, or someone who has formerly studied the subject but has not yet had a professional job in the topic area An intermediate level question is utilized when the applicant needs to demonstrate that they understand how to operate the commonly used functions in the subject, and how to troubleshoot common issues of the subject An intermediate skill set would be a person who works with the subject on a daily basis and can navigate and troubleshoot the common issues within the subject This person would not need to have any hands on supervision when usmg the functions mside the subject The majoπty of techmcal people who work with any given subject on a daily basis would fit into this category An advanced level set of questions is utilized when the applicant needs to demonstrate that they have mastered how to operate the uncommon functions in the subject The applicant who needs to demonstrate that they can troubleshoot uncommon issues that are WO 00/77672 PCT/USOO/l 6547

related to the subject is given the advanced level tests An advanced skill set would be a person who has worked professionally in the subject for at least three years on a daily basis This person would be able to technically lead a group and would be able to mentor other technical people m regards to the subject The above are merely representative examples of the flexibility of the system in accordance with the present invention Other levels, or lesser levels, can be utilized dependmg on the subject For most subjects, these three levels would be adequate In accordance with the present invention, the exam process is secure and tamper- resistant Candidates can take an exam from anywhere in the world via the Internet Since the exam and the system would reside on a network that is available 24 hours a day, an advantage is provided as opposed to taking the exam at the potential employer's facility The Internet never closes The candidate can take the exam at any time, day or mght In accordance with one embodiment of the present invention, each exam question is set on a 80 second timer, to allow the candidate ample time to answer the question The set time for each question is mtended to make it difficult to cheat by utilizing manuals or other references This is not mtended as an absolute fool-proof method The time may be set m such a way that if the candidate can utilize a reference and answer the questions m such a time, the skill level is already given as an indication Once a candidate takes their exam, the database instantly grades the exam and notifies the recruiter that a test has been taken The recruiter may also go online to their pπvate section of the database or the system to see a graph of how the grade was broken down and measured To ensure accurate and detailed reporting to the recruiters, each exam module is graded by subjects, skill levels, and categoπes Each module is graded, then broken down into a results outline format that displays the subject, which shows the recruiter what the following questions consisted of The next area of the results outline consists of the subject matter that is covered in the exam A third category is the difficulty level of the questions An additional category shows whether or not the question was answered correctly A score is then given for the overall module which is generally used by the recruiter to understand where the inefficiencies, if any, exist in the candidate's techmcal abilities One embodiment of the present invention is illustrated in Figure 1 The client 100 obtains access to the server 120 through a network connection 110 (steps 1 and 2) The client is a potential employer or another entity that has an interest m screenmg applicants This includes employment agencies and the like The network 110 connection can be a pπvate connection or a public connection However, to maximize usage of the system, the Internet is utilized for maximum access Larger entities can utilize the system over both pπvate and public networks The server 120 is the location of the database of all exams Additionally the working or controlling software resides on the server 120 As mdicated earlier, the server is not limited to one computer system but can be a collection of systems networked together to achieve the desired function Access to the server includes an identification of the client user The identification is used to veπfy that the user is a subscπber and also to locate any customized exam sets previously utilized or requested Once the client 100 has obtained access to the server 120 through the network 110, the client can select a new exam or set of exams This is achieved by selecting from the exams existing on the server or by addmg a pπvate exam created by the client user The selected exam or set of exams 130 is stored on the server 120 (step 3) The client 100 is provided with an access code to be given to any candidate to access the selected exam 130 (steps 4. 5 and 6) The client can create as many selected exams as desired and each is given a different access code The client passes the access code to any candidate 140 that the client wishes to be tested (step 7) The candidate 140 utilizes the access code to connect to the server 120 and access the selected exam 130 (step 8) If the Internet is utilized as the network, access to the exam can be made at any time from any location, preferably usmg any standard browser software Additionally, the client may request that the exam be accessed only duπng a limited time peπod either to duration or to time of the day In this process, the candidate is asked to provide identification so that the employer can tie the results to the candidate Examination centers can be set up at \ aπous locations where the identity of the candidate taking the exam can be veπfied This is utilized where the candidate is suspected of havmg someone else take the exam for him or her or to prevent a group from assisting in taking the exam Depending on the situation, other secuπty means, including biometrics like voice signatures, can be employed However, it is contemplated that m the majoπty of situations, such added measures are not required The exam program includes means to protect copying or pπnting of the questions or answers Built in software is utilized for this purpose Additionally, software to log off the network, if prmtmg or copying is attempted, is utilized Other tamper proof systems can also be employed Each question is timed for a preset peπod For example 80 seconds per question has been found appropπate for most situations Other intervals may also be utilized Once all the questions have answered or cycled, the candidate is denied access to the questions The candidate is optionally given the opportunity to comment on the exam or the process This feedback is utilized to optimize the system to meet the vaπous needs of clients and to improve the exam process based on the target group of candidates Vaπous options for obtaining feedback from the candidate are contemplated for utilization with the present invention Once the exam is completed, the system reviews the answers and generates a report 150 of results or comments (step 9) to be provided to the client 100 (steps 10. 11 and 12). If desired, the client 100 can also access the actual selected exam taken by the candidate to review the specific answers Figure 6 shows one sample of the results report for a candidate taking vaπous skill level exams for the Microsoft® family of software products Figure 7 shows one sample of the results report for a candidate taking the basic skills level exam for HTML 4 0 knowledge In the "view exam results" area of the site, the recruiter sees all the exams they created, the number of candidates viewed, the number of candidates whose results have not been viewed and the exam passed number When an exam is selected, in order to view the exam result, as illustrated in figure 8, the recruiter can see the date when the candidate took the test, how long it took the candidate to complete the test and the date when the recruiter viewed the results Results may also be deleted from this page Additionally, the system can offer the customer the ability to reset candidate's results through the "Manage Exam Takers" area of the site, as illustrated in figure 9 Now if a candidate loses their connection while taking a test, the recruiter can reset the candidate so they can retake the test As customer requirements change, the human resource or recruiter can now modify exams by clicking on "Modify or Delete an exam," as illustrated m figure 10 Then they select the exam to be modified and follow the same process as the "create an exam" process Once the exam is modified, the HR recruiter will be sent an automatic e-mail confirmmg the exam change As can be seen from the above discussion of an embodiment of the present invention, the human resources personnel or any recruiter would obtain sigmficantly better mformation regarding the skills of any potential candidate This is particularly useful m the more techmcal areas such as computer science Advantages of the present invention include the following 1. The system includes means to reduce the likelihood of cheating. The product has timed questions and mcludes codes to prevent pnnting. Specifically, the "print screen" function is blocked. The system also mcludes random question selection so that duplicate tests will be infrequent 2. The present system provides for exams covering an unlimited number of subjects. For each subject, a large number of questions are generated. In one embodiment, each subject would include about 300 questions The actual exam given to any candidate would not include all of the questions. In one embodiment, each exam is 17 questions. 3. The present invention allows the client to tailor the exam set to the particular candidate. In today's environment, each candidate is umque m their qualifications and often would have expertise in more than one techmcal field or disciplme. 4. In addition, the client can load their own exam to the system for their own pπvate use. 5. Feedback from the candidates is sought in order to continually optimize and modify the exams to meet the need of both the target candidates and the client. 6. The exam preparation is a process involving both techmcal experts and promment authors m the subject field. The exams are subjected to several review stages pπor to blind testmg. The results of testing are utilized to enhance the product exam, where needed, pπor to placmg on the server system for use by clients. 7. The participation by the authors or experts m the subject field is encouraged by providing the experts with economic incentive tied to the net mcome of the system. The authors or experts are not usually employees of the orgamzation but are outside experts who have excelled m their respective fields This method results in tapping a larger pool of expertise than would be possible m a single orgamzation 8 Subscπber or clients are offered access to the system on a flat annual fee basis in addition to a one time one exam basis Although English is more often used than any other language, particularly m techmcal fields, the system is adapted to conduct the exams in vaπous languages This invention is applicable to any field or disciplme requiπng knowledge not readily available to recruitmg personnel. Examples include computer technology, programming, network applications, office skills, scientific disciplines, medical fields, accounting and the like The hardware components utilized m the present invention are readily identifiable by a person skilled m the art For example, a Pentium II 400 mhz with 30 gig of disk space is utilized with 3 Tl connections The web server is runnmg Wmdows NT and SQL 6.5. The exams application is implemented using ASP (Active Server Pages) Other systems can be utilized to improve speed and storage capabilities Additional Internet connections can also be provided dependmg on traffic requirements Although the present mvention has been descπbed above as employed on the Internet, the term Internet, as used herem. is also meant to include future developments, such as the Next Generation Internet (NGI) or Intemet2

Claims

I Claim. 1. A method for skills assessment of candidates by clients, compπsmg providing a server connected to a database of skill assessment exams; allowmg clients to access said server via a network and electromcally select at least one skill assessment exam; generating a code for each selected skill assessment exam, delivering a code for each selected skill assessment exam to said client, wherem said client supplies a candidate with a code correspondmg to an exam that the client wishes to admimster to said candidate: and allowmg candidates access to said server to electromcally complete the exam that the client wishes to admimster to said candidate upon the candidate mputtmg the code corresponding to the exam that the client wishes to admimster to said candidate.

2. The method for skills assessment of candidates of claim 1, further compπsmg' automatically grading completed exams; and notifying said client of the completed exam.

3. The method for skills assessment of candidates of claim 1 , wherem the step of electromcally selectmg at least one skill assessment exam mcludes selecting a topic and a skill level.

4 The method for skills assessment of candidates of claim 1 , wherem the step of electromcally selecting at least one skill assessment exam includes creating a customized exam by selectmg a plurality of topics and a skill level for each topic.

5. The method for skills assessment of candidates of claim 2, further compnsing: allowmg the client to access and view candidate^'s exam results

6. The method for skills assessment of candidates of claim 1, wherein administration of the exam is subject to time constraints.

7 The method for skills assessment of candidates of claim 7, wherem the time constraints are selected from the group consistmg of limiting the time peπod within which an exam may be taken, limiting the length of time to complete the exam, and limiting the length of time to answer each question on an exam

The method for skills assessment of candidates of claim 1, further compπsmg providing security to the exam administration by a method selected from the group consistmg of limiting the location of the test administration, limiting the prmtmg and copymg of the exam by the candidate, requiring personal venfication of candidate identity, and requiring biometπc venfication of candidate identity

The method for skills assessment of candidates of claim 1, wherem the client accesses the server via the Internet

The method for skills assessment of candidates of claim 1 , wherem the candidate accesses said server to electromcally complete the exam via the Internet

A system for skills assessment of candidates by clients, compπsmg a server connected to a database of skill assessment exams, means for allowmg clients to access said server via a network and electromcally select at least one skill assessment exam, means for generating a code for each selected skill assessment exam, means for delivering a code for each selected skill assessment exam to said client, wherem said client supplies a candidate with a code corresponding to an exam that the client wishes to admimster to said candidate, and means for allowing candidates access to said server to electronically complete the exam that the client wishes to admimster to said candidate upon the candidate mputtmg the code correspondmg to the exam that the client wishes to admimster to said candidate

The system for skills assessment of candidates of claim 1 1. further compnsing means for automatically grading completed exams, and means for notifying said client of the completed exam

13. The system for skills assessment of candidates of claim 1 1 , including means for creating a customized exam by selecting a plurality of topics and a skill level for each topic.

14. The system for skills assessment of candidates of claim 12, further comprising means for allowing the client to access and view candidate's exam results.

15. The system for skills assessment of candidates of claim 1 1 , further comprising means to provide security to the exam administration selected from the group consisting of: an examination center, software limiting the printing and copying of the exam by the candidate, personal verification of candidate identity, and biometric verification of candidate identity.

16. The system for skills assessment of candidates of claim 1 1, wherein the means for allowing clients to access said server and means for allowing candidates to access said server is the Internet.

17. A system for skills assessment of candidates by clients, comprising a server connected to a network; a database of skill assessment exams connected to said server; at least one client computer with access said server via the network to electronically select at least one skill assessment exam from said database; a code generated for each selected skill assessment exam; at least one candidate computer connected to said network to access said server, wherein candidate computers access said server so candidates can electronically complete the exam that the client wishes to administer to said candidate upon the candidate inputting the code corresponding to the exam that the client wishes to administer to said candidate.

18. The system for skills assessment of candidates of claim 17, wherein the network is the Internet.

19. The system for skills assessment of candidates of claim 17, further comprising: software on said server operable to automatically grade completed exams; and software on said server operable to notify said at least one chent computer upon completion of an exam.

20. The system for skills assessment of candidates of claim 17, further compπsmg: software on said server operable to allow the at least one client computer to access and view candidate exam results