WO2004045725A1 - Instruction method using virtual apparatus - Google Patents

Instruction method using virtual apparatus Download PDF

Info

Publication number
WO2004045725A1
WO2004045725A1 PCT/NZ2003/000256 NZ0300256W WO2004045725A1 WO 2004045725 A1 WO2004045725 A1 WO 2004045725A1 NZ 0300256 W NZ0300256 W NZ 0300256W WO 2004045725 A1 WO2004045725 A1 WO 2004045725A1
Authority
WO
WIPO (PCT)
Prior art keywords
trainee
sequence
muscles
representation
action
Prior art date
Application number
PCT/NZ2003/000256
Other languages
French (fr)
Inventor
John Hansen Ryall
Original Assignee
John Hansen Ryall
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by John Hansen Ryall filed Critical John Hansen Ryall
Priority to AU2003283887A priority Critical patent/AU2003283887A1/en
Publication of WO2004045725A1 publication Critical patent/WO2004045725A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0003Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B69/00Training appliances or apparatus for special sports
    • A63B69/36Training appliances or apparatus for special sports for golf
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0003Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
    • A63B24/0006Computerised comparison for qualitative assessment of motion sequences or the course of a movement
    • A63B2024/0012Comparing movements or motion sequences with a registered reference
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B69/00Training appliances or apparatus for special sports
    • A63B69/36Training appliances or apparatus for special sports for golf
    • A63B69/3667Golf stance aids, e.g. means for positioning a golfer's feet
    • A63B2069/367Detection of balance between both feet, i.e. weight distribution
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00Measuring of physical parameters relating to sporting activity
    • A63B2220/50Force related parameters
    • A63B2220/51Force
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00Measuring of physical parameters relating to sporting activity
    • A63B2220/80Special sensors, transducers or devices therefor
    • A63B2220/806Video cameras
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00Measuring of physical parameters relating to sporting activity
    • A63B2220/80Special sensors, transducers or devices therefor
    • A63B2220/807Photo cameras
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2225/00Miscellaneous features of sport apparatus, devices or equipment
    • A63B2225/50Wireless data transmission, e.g. by radio transmitters or telemetry
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2230/00Measuring physiological parameters of the user
    • A63B2230/60Measuring physiological parameters of the user muscle strain, i.e. measured on the user
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2243/00Specific ball sports not provided for in A63B2102/00 - A63B2102/38
    • A63B2243/0066Rugby; American football
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2244/00Sports without balls
    • A63B2244/08Jumping, vaulting
    • A63B2244/082Long jumping
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2244/00Sports without balls
    • A63B2244/08Jumping, vaulting
    • A63B2244/085Pole vaulting
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2244/00Sports without balls
    • A63B2244/15Discus throwing
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2244/00Sports without balls
    • A63B2244/16Spear or javelin throwing
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0075Means for generating exercise programs or schemes, e.g. computerized virtual trainer, e.g. using expert databases

Definitions

  • This invention relates to apparatus and a method for teaching a trainee to better perform a coordinated sequence of movements or actions, including but not limited to procedures used in sports, or when working, or when undergoing rehabilitation, and particularly relates to use of kinesthetic concepts.
  • Teaching a human to perform a coordinated sequence of movements or actions often involve teaching by example, and/or having an instructor supply guidance and corrections while watching the student or trainee practice. These methods are suitable only up to a point because the trainee may achieve an end result with a sub-optimal coordinated sequence.
  • a procedure such as pole vault or a golf swing.
  • Saville et al in US 6261189 provide an example method for teaching golf.
  • the method relies upon causing the trainee who is learning to hit the ball to appear correct when swinging, as monitored by means of a set of optical sensors.
  • the inventor considers this method typical of many that teach "by appearance or result” rather then “by process”.
  • Saville et al do not lead the trainee to the desired result by kinesthetic methods that identify the particular body parts involved in the complex sequence. They use "positive and negative reinforcement” with bells or status lights, and means to take the ball out of the way, to signal a visibly incorrect swing.
  • This teaching method like that of Goodwin et al (US 5277428) and McCardle Jr (US 5269528) with on-club position sensors is indirect.
  • Marley uses the term "kinesthesia" in relation to learning the golf swing
  • the patents employ on-club position sensors and surrounding structures only.
  • Sanchez et al US 5672155 uses a cord (instead of image sensors) to monitor hip position.
  • Whelan (US 6612937) uses lights to control head position.
  • McTeigue et al (US 5221088) use force transducers - inside shoes and on the grip of a bat or club.
  • O'Leary et al provide mirrors to overlay a video-taped image of an instructor on the trainee's view of himself taking the swing (US 5249967); again, an outside view.
  • Other patents simulate the actual course - such as Williams US 5976022.
  • trainee means the person being instructed by means of the instructional sequence.
  • electromyogram refers to a record of myoelectric activity (muscle fibre action potential) from an identified place on or in a body. This roughly indicates the instantaneous force that a muscle is generating.
  • "krn(a)esthesia” is "That quality of sensations whereby we become aware of our position in space, our movements, and that gives us our impression of weight and resistance” — from Gould's Medical Dictionary, 5th edition.
  • the concept also embraces "feedback" as derived from the widely distributed group of proprioceptive receptors including those in muscles, tendons and joints, also those in (for example) the inner ear (the vestibular receptors) and as supplied to the brain, although such information is usually not perceived by the average person unless trained in order to become aware of such inputs.
  • muscle system refers to the muscles, tendons, fascia, bones and ligaments that make up the motor apparatus used by a human to perform physical actions.
  • OBJECT It is an object of this invention to provide an improved instruction method and demonstration apparatus in relation to performing actions, or at least to provide the public with a useful choice.
  • the invention provides an audio-visual teaching sequence for instructing a trainee how to carry out at least one action or procedure in an improved manner; wherein the teaching sequence provides instruction leading to kinesthetic appreciation by the trainee of those portions of the trainee's musculoskeletal system used when carrying out the action or procedure, and the teaching sequence includes a visual portrayal of substantially anatomically correct selected portions of the musculoskeletal system upon a first representation of a human body or mannikin shown carrying out the action or procedure, so that use of kinesthetic discrimination and/or perception principles by the trainee enhances his learning of the at least one action or procedure.
  • the representation of a human body is made capable of appearing to speak to the trainee by means of replay of recorded speech together with appropriate synchronised motions, and is provided with the capacity to move and demonstrate the at least one action or procedure; so that the first representation of a human body serves as a virtual instructor.
  • the teaching sequence is capable of including a portrayal of muscle activity displayed with reference to the first representation of a human body, over time; the portrayal of muscle activity being based on data previously collected and recorded from at least one skilled person while performing the action or procedure; so that the teaching sequence includes a usable display of information concerning the related muscle activity.
  • the method involves the use of a display of the muscles, tendons, joints, and posture by means of the human anatomical model and the resulting kinesthetic sensations while the action is being carried out.
  • the teaching sequence further includes a portrayal of forces and movements generated as a result of activity of the musculoskeletal system; data describing the generated forces and movements having been previously collected and recorded from at least one skilled person while performing the action or procedure.
  • the teaching sequence provides, when in use, a visual demonstration of a display of the muscles of a human shown upon the representation of a human body, and displays the individual and collective activities of the muscles in relation to the kinesthetic sensations that the trainee would perceive while while performing the action or procedure.
  • the teaching sequence includes at least one exercise directed at reinforcing awareness by the trainee of activity of a relevant portion of the musculoskeletal 100 system, in relation to the resulting change in body position.
  • the teaching sequence includes means allowing the trainee to compare stored muscle activity with muscle activity data collected from the trainee by means of activity collecting apparatus during the instructional sequence while performing the action or procedure; the trainee activity collecting apparatus including at least one of: a video camera 105 focused upon the trainee; an array of electromyogram electrodes placed in relation to the trainee's muscles, and means (such as a pair of load cells) to detect instantaneous forces applied by the feet of the trainee to the ground. ,
  • the data collected from the trainee during the instructional sequence is portrayed upon a representation of the trainee - a second representation - while carrying out the action.
  • the data collected from the trainee is used as input to a computational means capable of generating corrective commands which are based on kinesthetic principles (rather than simply copying postures).
  • the invention provides apparatus for use when making an instructional or teaching sequence as previously described in this section, wherein the apparatus
  • 115 includes: (a) the moveable representation of a' human body, capable of being shown carrying out the action or procedure, (b) means for recording speech and images, comprising at least a part of the instructional or teaching sequence, within a storage medium, (c) activity collecting apparatus including at least one of: a video camera focused upon a demonstrator; an array of electromyogram electrodes placed in relation to the demonstrator's muscles, and means to
  • each model included in the apparatus includes means to display selected muscles without covering, so that the relevant muscles are made visible in the instructional or teaching sequence.
  • the model is a virtual model, generated within a computer under control of software.
  • the invention provides a method for using a teaching sequence as 130 previously described in this section, wherein the method is based on developing kinesthetic appreciation by the trainee, wherein the method relies in part on demonstrating the role of muscles involved in the action to the trainee by means of a moveable representation of a human body so that the trainee can visualise the interplay of muscles involved in the action and consciously repeat the sequence in his own body.
  • the teaching sequence is displayed together with annotations , accompanying information, and exercises.
  • the teaching sequence includes software capable of providing for the inclusion into the sequence of data collected during use from the trainee.
  • Preferred data includes images captured during the action, optionally with electromyogram data and data describing other 140 parameters including force.
  • One preferred form of position data comprises at least one videotape record of the subject during the recording process. More preferably the positional data is capable of being geometrically corrected and is taken by a number of cameras.
  • the sequence comprises a visual demonstration capable of making use of kinesthetic signals, including those related to visual understanding of the muscles and postures involved in 145 carrying out the action.
  • the visual demonstration includes other procedures capable of enhancing the trainee's awareness of relevant kinesthetic signals.
  • An example other procedure comprises an exercise for reinforcing awareness of a relevant muscle, tendon, joint, or posture.
  • Another example comprises generating a second mannikin representing the trainee, together 150 with muscle activity and resulting movements, according to data collected from the trainee in real time, so that the trainee can make his mannikin resemble the mannikin representing the skilled person, as seen on a display, by applying kinesthetically based "drive" to his muscles.
  • teaching sequence is embodied in a repeatedly playable recorded medium
  • example forms of display include: three-dimensional holograms, projected two-dimensional 155 images, images viewed upon screens including computer and video (television) screens, and screens showing images from projected movies.
  • the invention provides a method for teaching and learning at least some of the activities involved in the game of golf, including the swing.
  • the invention provides a method for teaching and learning at least some of the
  • Fig 1 is a diagram of apparatus, showing how muscular activity information may be recorded for inclusion in a teaching sequence.
  • Fig 2a is a perspective view; a line drawing (taken from Fig 2b) showing an example human 175 anatomical model or mannikin, with visible muscles, being used in a teaching sequence relating to a golf swing. Inputs and optional inputs are shown.
  • Fig 2b is a perspective view; a shaded image taken from a 3D, lit, shaded computer-generated image showing the mannikin being used in a teaching sequence relating to a golf swing. (This display is preferably in colour).
  • Fig 3 is like Fig 2 but from a rear perspective.
  • Fig 4 is like Fig 2 but from an elevated perspective.
  • Fig 5 is like Fig 2 but is a closer view from yet another perspective.
  • Fig 6 shows a representation of a skeleton (a usually hidden layer of the mannikin) and a track of the path followed by the head of a golf club during an entire process, overlaid 185 on X-Y co-ordinates (the Z co-ordinate is repressed here).
  • the invention maximises the use of kinesthetic principles within a method for teaching a musculoskeletal sequence of movements (for an action or procedure), to a person.
  • the invention assumes that a "teaching sequence" will be developed for each subject.
  • the teaching sequences are intended to be displayed in colour on (for example) a video (television)
  • DVD "Quicktime ⁇ M" " computer video, holograms, or the like. They may be distributed on videotape, DND or CD, interactive DND or CD, or a non-tangible distribution medium including cable, broad-band web, and the like. Sequences may be shown in a non-interactive manner on an appropriate player. For some applications, 16 mm film, 35 mm slides, or data 195 projector images are suitable. An interactive teaching sequence is preferred and may involve use of a computer programme (perhaps holding supporting data on a CD or DND, from which the programme is loaded for use) developed for the above purpose. Some applications, such as golf techniques, may merit or may economically justify more complex interactive teaching environments than others. One trainee at a time is assumed in this section.
  • the teaching sequence is based upon a virtual person; a human anatomical model or marmi n which serves as the central character for the teaching sequence and upon which are displayed cues for the kinesthetic aspects of the teaching programme.
  • this invention includes a physical basis for kinesthetic principles. Note that the mannikin to be described below is first generated during construction of the teaching sequence, and may later be either modified or re-
  • the trainee will be visually comparing his own musculoskeletal activity, muscle by muscle, with that of a demonstrator, rather than comparing the end
  • Each mannikin (200 in Figs 2-6) is a representation of the human body upon which selected muscles involved in movements are, from time to time, displayed.
  • avatar is used by some to refer to a virtual or computer-generated person. Speech and behaviour attributes are intended to be friendly and realistic. Because the human anatomical model is shown naked, yet 215 being sufficiently stylised (by virtue of being stripped down to the actual muscles) that most viewers are not distracted or embarrassed, the trainee is able to concentrate on fundamental actions involved. It is psychologically and generally useful to give the mannikin a personality, a voice, normal human movements, and have it become the instructor for the teaching sequence(s). Preferably the animated mannikin itself serves as the presenter of the programme.
  • the mannikin might be based upon a film or video record of a real person who might be a selected model with good muscle development and no fat.
  • a low-cost way to create a programme creation starts with images of a real person carrying out the action at normal speed and reasonably perfectly.
  • the person might be filmed wearing a skin-tight coverall or simply grey-painted skin (on at least the relevant parts of the body) so that muscle activity can be
  • a movable mannikin may be created through use of (for example) drawings or the use of frame by frame shots of clay models as by the film producer Nick Park for his characters "Wallace & Grommit"). Initial teaching sequences using the mannikin were made on a frame-by-frame basis.
  • the inventor prefers to create the mannikin with an animation or 3D rendition computer 230 programme which will accept transducer inputs (as Fig 2a, a,b, and c.) according to Fig 1, and generate a mannikin in real time showing either the skilled person's or the trainee's activity.
  • this allows the trainee to learn the musculoskeletal sequence of movements in an interactive way.
  • Current developments in simulation software will allow complete sequences of movements to be displayed based on (for example) numerical modelling of specific muscles, 235 taking into account the specific properties of each and their placement upon the skeleton, using instantaneous power generated over time for each of these. (That is a computer analogue of kinesthetics).
  • Fig 2a, 2b where 200 indicates the human anatomical model, 201 is a golf club and 202 is a golf ball upon a tee.
  • This image is normally presented as a three-dimensional image in shaded, lit coloured form, and as part of a connected, moving sequence, not as the black line drawing of Fig 2a favoured in patent specifications).
  • Muscles eg 203 - m. vastus lateralis
  • tendons eg 205 - patellar tendon
  • bones eg 204 - left femur
  • each muscle will be shown in any view. In most display modes each muscle is shown in a substantially correct configuration and position, with a red striped appearance, and preferably an emphasis (see later) if the muscle is active. Tendons, ligaments and other anatomical features are shown if relevant. It may be preferable to hide those muscles not involved in a particular movement beneath a simulated track suit, overall, or the like although in many actions, such as
  • the model can be shown from a variety of viewpoints; the rear 206 as in Fig 3, from a raised viewpoint (Fig 4) or in close-up (fig 5) while carrying out the action.
  • the software used for the flexible model includes a number of "layers” provided by the preferred computer package "Maya” ® (Alias/ Wavefront, Toronto, Canada) used for 3D modelling.
  • the first layer is a straight-line version of the skeleton, with lines joining various anatomically distinct points.
  • Another layer is a complete skeleton (see Fig 6).
  • the visible layer comprises a covering of muscles, ligaments, and bones as if skin and subcuta- 265 neous fat had been removed from a human body.
  • the resulting model can be viewed from any direction and illuminated in order to show off rounded surface shapes, and is based on accepted anatomical texts so that the human anatomical model can be shown while carrying out movements involved in the action being imparted to the trainee.
  • the trainee learns how to carry out the action by seeing and feeling the muscular 270 sequence of activities displayed by the representation of a human, with muscles, including posture, movement, and specific muscular activities, the trainee can understand the action more easily and is better able to relate what is being shown to what he is doing himself.
  • Full awareness of the status of a relevant muscle, tendon, joint, or of one's overall movement and posture (including sensors in the inner ear, the vestibular system) is relatively uncommon in 275 people.
  • the invention makes use of this status information which may be deliberately brought out to reach a level of consciousness in a training session.
  • This kinesthetic process of learning a complex musculoskeletal sequence of movements is believed to involve at least (1) the cerebral cortex, as well as (2) the basal ganglia and (3) the cerebellum in co-operative function, with the final sequence being held in the cerebellum.
  • Hikosaka, O in Neurobiol Learn Mem 67(1) 29-33 280 (1997) offers a hypothesis suggesting that at least several sets of the above brain circuits work in parallel when a complex action is being acquired.
  • specific knowledge of the or any actual site of learning within the central nervous system is not known to be very usef l for the purpose of this invention, given the difficulty of carrying out detailed brain research of such complexity. The invention works even without having that knowledge.
  • one kinesthetic approach to teaching the golf swing might have the mannikin ask the trainee to visualise a completely unrelated situation in order to grasp, or become aware of, a feeling generated inside the trainee's body. For example, when at the "address position", the trainee might be asked to imagine he is in a dark room. "Someone might push you over. Ensure that your legs have tension, but your upper body can give way to the push. Think
  • teaching sequence is presented by means of a suitably programmed computer able to take input (Fig 2a, a 1 , b' and c' ... at right) from the trainee, using image sensors, in-shoe or on-fioor force transducers such as load cells, and EMG sensors and interpreting the results in terms of muscle activity, rather than as a simple non-interactive videotape (or the like).
  • a suitably programmed computer able to take input (Fig 2a, a 1 , b' and c' ... at right) from the trainee, using image sensors, in-shoe or on-fioor force transducers such as load cells, and EMG sensors and interpreting the results in terms of muscle activity, rather than as a simple non-interactive videotape (or the like).
  • 295 Computer-based versions of teaching sequence medium, optionally copied onto a computer hard disk before use, are more suited to flexible user control. Many computer games can show realistic movements - and because of related demand, suitable programmes, computers and graphics
  • the trainee is optionally provided with a kinesthetic form of biofeedback teaching, facilitated through the programme which makes the 300 trainee's actions appear on the mannikin.
  • the software displays a first mannikin, representing the instructor. If transducer inputs are available during a teaching session, a second (possibly simplified) trainee mannikin is displayed for comparison purposes.
  • muscle tone and/or muscle activity is portrayed to the trainee preferably as a colour change or alternatively by some other intuitively recognisable indicator of
  • Resulting forces may be shown, or may be used internally by the computer running the teaching sequence as an indication of muscle activity.
  • a pressure transducer may be placed in each of the trainee's shoes in order to indicate balance and then movement, and another pressure transducer may be placed in the grip of the club in order to sense the extent of the grip.
  • load cells may be preferred for use in a golf clinic.
  • Pressure transducers may be placed in each of the the instructor's shoes (or in load cells) in order to acquire baseline data.
  • Video camera data is likely to be required in order to support the load cell results, and the instructor may have electromyogram electrodes apphed to him; all of course having been connected by means of appropriate interface electronics to the computer.
  • More complex apparatus may be more suited to professional teaching rather than to a home- use product.
  • the trainee data to be displayed on the human anatomical model can be compared with "proper" data by the computer (which should be programmed to offer corrective messages in terms of kinesthetic terminology), or by the trainee (and possibly by an instructor) in order to generate knowledge of any differences or shortcomings to be overcome
  • Kinesthesiology relies on showing exactly what muscles are activated, by how much, when, and in what sequence.
  • the 330 trainee is instructed to carry out the sequence as shown and to think about his own body and how his muscles feel as he does it — according to the principles of kinesthesiology.
  • Learning processes may proceed at least in part by a process of error correction, so repeated trials, with feedback to the trainee indicating any errors, is useful.
  • We may be creating a new task for the brain of the trainee - to understand a complex musculoskeletal sequence of movements by
  • 340 displays, even .displays in an "exploded" view so that the trainee can discriminate activity of individual muscles more clearly, or the like.
  • Figs 2-5 the mannikin can be viewed from many aspects, and close-ups are possible.
  • the trainee's swing could be examined at say 6 points, by means of one or more video cameras that captured a sequence of steps. Rather than simply
  • the computer program nning within processor 111 running the training sequence is programmed to issue corrections in kinesthetic terms, detailing what the trainee's musculoskeletal system should be doing at or by each point.
  • the program would interpret information derived from one or more video cameras showing body and club position, and
  • EMGs are a physiological, recordable modality that is related to neuromuscular activity and which may be used by the demonstrator, or in an interactive way during learning.
  • the inventor has found electromyography as used upon the trainee technically difficult to apply because of wires.
  • a wireless, self-contained stick-on electrodes and preamplifier unit would avoid that problem.
  • EMGs are more likely to be brought into use during a teaching session if the session
  • the processor 111 as used by a "standard" trainee is preferably provided with software capable of interpreting combinations of image data and/or load cell data as events occurring within the body of the human subject. This software is likely to be particularly useful when applying trainee data onto the mannikin in real time 370 during a teaching sequence, helping the trainee to visualise and understand his own body in terms of kinesthetic principles.
  • Sports medicine facilities (kindly provided by the Tamaki Campus, University of Auckland) placed a set of video cameras arranged in a precisely known range of positions (as indicated by 120, 120A in Fig 1) in relation to the subject for studying motion by recording images of the frame-by-frame position of bright dots placed on dark clothing during the execution of
  • Electromyogram (EMG) data was also collected from specified muscles during the execution of an action (a golf swing) by the subject (see 101-107 in Fig 1).
  • Fig 6 shows the path (601) before, and 602 (during and after) the swing, taken by the head of the golf club 201 when
  • This particular illustration lacks perspective and may be more use to the course developer than to a trainee who is normally shown a perspective view; a three-dimensional illustration.
  • display screen 112 shows a replay view of part of the arm 122 bearing areas such as 115 which are changed in appearance during activity of the underlying muscle(s), as previously recorded (such as by electrodes 101A, 102A). Muscle activity and forces appleid through feet and hands may also be stored on a videotape sound track, in computer files, or in some other synchronised recording medium for use in comparative demonstrator/trainee inter- 405 active sessions.
  • the sequence is repeated a number of times or from a variety of angles, in order to show other parts of the body (e.g. the back) and it may also be repeated in slow motion.
  • the kinesthetic teaching sequence assists the trainee to become aware of his own muscles and other body parts 415 in action.
  • our training programme may include exercises intended to over-use specific muscles so that the trainee is acutely aware of them during a learning session.
  • muscle activity can be quantified in a multi-channel data collection device 100 by detecting the EMG; typically a signal of a few millivolts which can be picked up by contact skin electrodes (101 and 102, paired) or with unipolar needle electrodes from within the
  • a person's arm 122 has a first pair, 101 A, 102 A of electrodes placed on the skin over a biceps muscle and a second pair, 101B, 102B placed over m. extensor digitomm.
  • a number of muscles are sampled simultaneously by n separate channels, and a multi-channel amplifier/recorder would be appropriate.
  • Each signal is amplified (differential amplifier 103) and recorded for later analysis on a preferably synchronised storage device
  • Fig 1 also shows a video recorder 110 and video cameras 120, 120A connected by lead 121 to the
  • processor/mixer 111 which is typically a digital computer having one or more analogue input boards and one or more video editing boards inserted (as is well known to those skilled in computer applications), together with appropriate software, and is passed to a display device 112 (with control means 113). More than one video camera (as 120A) may be used (according to well-known techniques) in order to gain three-dimensional data collection. 435 In addition to visual imagery, the device 111 may conveniently be supplied with pressure information from a pair of force transducers (commonly known as load cells) 116, 118; one under each foot of the human subject. These devices may be included within the trainee's shoes. All this information helps describe the demonstrator's or the trainee's resting position and actual movements. Sensors (such as accelerometers) on the club itself, or pressure sensors within the
  • grip of the club or within gloves may also assist in describing the resultant actions.
  • One way to superimpose electromyogram data onto the marinikin uses an editor to create a colour box superimposed in real time on each image, in which the colour shown over each muscle (115) in each frame is proportional to muscle activity (as derived in 105 and 106).
  • a person can emulate a "paint by numbers” process to colour the image of a specific muscle, 445 frame by frame, during editing.
  • Muscle force can be quantified by use of a force transducer (typically a strain gauge or sometimes a load cell) placed appropriately in a shoe or in a
  • EMG processing apphed subsequently to the storage stage (104) allows the user to adjust the parameters if required.
  • EMG data requires interpretation, because of factors including basic muscle tone, effects of opposing sets of muscles, body inertia, action/reaction, and consequences of preceding and current activity by remote muscles.
  • EMG inputs 107, 108, 109
  • processor/mixer 111 Use of simultaneously collected EMG inputs (107, 108, 109) to the processor/mixer 111 provides inter-relationship information. Although only one muscle group may be edit or enhanced on the human anatomical model at a time, knowledge of other activities that are also occurring at the same time is useful. It is possible that actual displays of real data derived 465 from physiological procedures such as those described above should be modified and simplified in order to lead a trainee towards making the right sequence of actions.
  • the machine itself may overlay the colours over each of the muscles in question, either onto a human anatomical model or onto an image of a person.
  • Sometimes the use of an actual person with muscle activity superimposed onto the skin is an inadequate solution to the problem of 470 providing an effective teaching machine.
  • a first approximation to a complete program may be obtained by use of the actual person, who is then edited out and replaced by a computer-generated marmikin.
  • the mannikin may be created or shaped during construction of a teaching sequence on a frame-by-frame basis to replace/ overlay the image of either the person who acts as a demonstrator, The same process may be used to replicate the trainee's
  • teaching sessions are designed for a solitary trainee, multiple trainees (especially in the case of non-interactive versions), and/or actual instructors or demonstrators (optionally with their marmikins) may be present, especially at training clinics.

Abstract

A method for instructing a trainee to perform a complex sequence of movements as in golf employs a 'virtual human'(200), or anatomical model (200) displaying relevant muscles (203, 204, 205) and associated body parts, using principles of kinesthsiology. Useful sensations reflecting posture, movement, and the like arise from muscles, joints, tendons, vestibular and other proprioreceptors. The trainee learns by seeing and feeling the actual sequence of activities as displayed with the model on a display screen. Advanced training employs a programmed computer using data taken from the trainee's own actions to create interactive comparison sequences.

Description

TITLE Instruction method using virtual apparatus.
FIELD
This invention relates to apparatus and a method for teaching a trainee to better perform a coordinated sequence of movements or actions, including but not limited to procedures used in sports, or when working, or when undergoing rehabilitation, and particularly relates to use of kinesthetic concepts.
BACKGROUND
Teaching a human to perform a coordinated sequence of movements or actions, including but not limited to procedures used in sports often involve teaching by example, and/or having an instructor supply guidance and corrections while watching the student or trainee practice. These methods are suitable only up to a point because the trainee may achieve an end result with a sub-optimal coordinated sequence. To consider the sports example, there is much benefit to be gained, especially in top-level competition, from perfection when carrying out a procedure, such as pole vault or a golf swing. For example, there are over a hundred patents relating to teaching the art of swinging a golf club.
Saville et al in US 6261189 provide an example method for teaching golf. The method relies upon causing the trainee who is learning to hit the ball to appear correct when swinging, as monitored by means of a set of optical sensors. The inventor considers this method typical of many that teach "by appearance or result" rather then "by process". Saville et al do not lead the trainee to the desired result by kinesthetic methods that identify the particular body parts involved in the complex sequence. They use "positive and negative reinforcement" with bells or status lights, and means to take the ball out of the way, to signal a visibly incorrect swing. This teaching method, like that of Goodwin et al (US 5277428) and McCardle Jr (US 5269528) with on-club position sensors is indirect. Even though Marley (US5860871 and US 6059668) uses the term "kinesthesia" in relation to learning the golf swing, the patents employ on-club position sensors and surrounding structures only. Sanchez et al US 5672155 uses a cord (instead of image sensors) to monitor hip position. Whelan (US 6612937) uses lights to control head position. McTeigue et al (US 5221088) use force transducers - inside shoes and on the grip of a bat or club. O'Leary et al provide mirrors to overlay a video-taped image of an instructor on the trainee's view of himself taking the swing (US 5249967); again, an outside view. Other patents simulate the actual course - such as Williams US 5976022. None of the existing ways to instruct the players are particularly successful. Almost all of the existing patents involve apparatus to capture the action of the golfer under training and to help the instructor to point out any problems - for example on a video screen. These procedures show the golfer under training what he or she looks like to an outsider. Enhancing the learning process remains a considerable challenge. It is said that "the game of golf is an activity requiring co-ordinated neuromuscular responses in order to attain proper positioning of the body and its extremities. Integral to a good golf swing is adherence to well-defined club head movement and consistency throughout the golf swing". Apart from the golf example, there are many instances where sports training and job training requires an effective teaching method that lets the trainee mentally "get inside" the body of a demonstrator. For example, learning the correct way to lift a heavy object in an industry or employment context can avoid a great deal of back injury, saving the expense of rehabilitation, loss of employment, and suffering. There is a need for a better instruction method for instructing a person in a mainly neurally controlled musculoskeletal sequence of movements required to carry out a function, a method involving kinesthetic principles.
DEFINITIONS
Note: reference to the male gender herein is to be taken to also include the female gender. For clarity I confirm that I use the word "including" to cover a wide range of items specified as well as others not specified, while the word "comprising" refers to only the items specified.
"trainee" means the person being instructed by means of the instructional sequence.
"electromyogram" refers to a record of myoelectric activity (muscle fibre action potential) from an identified place on or in a body. This roughly indicates the instantaneous force that a muscle is generating. "krn(a)esthesia" is "That quality of sensations whereby we become aware of our position in space, our movements, and that gives us our impression of weight and resistance" — from Gould's Medical Dictionary, 5th edition. In this document the concept also embraces "feedback" as derived from the widely distributed group of proprioceptive receptors including those in muscles, tendons and joints, also those in (for example) the inner ear (the vestibular receptors) and as supplied to the brain, although such information is usually not perceived by the average person unless trained in order to become aware of such inputs.
"musculoskeletal system" refers to the muscles, tendons, fascia, bones and ligaments that make up the motor apparatus used by a human to perform physical actions. OBJECT It is an object of this invention to provide an improved instruction method and demonstration apparatus in relation to performing actions, or at least to provide the public with a useful choice.
STATEMENT OF INVENTION ,
In a first broad aspect, the invention provides an audio-visual teaching sequence for instructing a trainee how to carry out at least one action or procedure in an improved manner; wherein the teaching sequence provides instruction leading to kinesthetic appreciation by the trainee of those portions of the trainee's musculoskeletal system used when carrying out the action or procedure, and the teaching sequence includes a visual portrayal of substantially anatomically correct selected portions of the musculoskeletal system upon a first representation of a human body or mannikin shown carrying out the action or procedure, so that use of kinesthetic discrimination and/or perception principles by the trainee enhances his learning of the at least one action or procedure. '
In a related aspect, the representation of a human body is made capable of appearing to speak to the trainee by means of replay of recorded speech together with appropriate synchronised motions, and is provided with the capacity to move and demonstrate the at least one action or procedure; so that the first representation of a human body serves as a virtual instructor.
Preferably the teaching sequence is capable of including a portrayal of muscle activity displayed with reference to the first representation of a human body, over time; the portrayal of muscle activity being based on data previously collected and recorded from at least one skilled person while performing the action or procedure; so that the teaching sequence includes a usable display of information concerning the related muscle activity.
More specifically the method involves the use of a display of the muscles, tendons, joints, and posture by means of the human anatomical model and the resulting kinesthetic sensations while the action is being carried out. Preferably the teaching sequence further includes a portrayal of forces and movements generated as a result of activity of the musculoskeletal system; data describing the generated forces and movements having been previously collected and recorded from at least one skilled person while performing the action or procedure.
Preferably the teaching sequence provides, when in use, a visual demonstration of a display of the muscles of a human shown upon the representation of a human body, and displays the individual and collective activities of the muscles in relation to the kinesthetic sensations that the trainee would perceive while while performing the action or procedure.
In a further related aspect the teaching sequence includes at least one exercise directed at reinforcing awareness by the trainee of activity of a relevant portion of the musculoskeletal 100 system, in relation to the resulting change in body position.
In a yet further related aspect, the teaching sequence includes means allowing the trainee to compare stored muscle activity with muscle activity data collected from the trainee by means of activity collecting apparatus during the instructional sequence while performing the action or procedure; the trainee activity collecting apparatus including at least one of: a video camera 105 focused upon the trainee; an array of electromyogram electrodes placed in relation to the trainee's muscles, and means (such as a pair of load cells) to detect instantaneous forces applied by the feet of the trainee to the ground. ,
Preferably the data collected from the trainee during the instructional sequence is portrayed upon a representation of the trainee - a second representation - while carrying out the action.
110 Optionally the data collected from the trainee is used as input to a computational means capable of generating corrective commands which are based on kinesthetic principles (rather than simply copying postures).
In a second broad aspect the invention provides apparatus for use when making an instructional or teaching sequence as previously described in this section, wherein the apparatus
115 includes: (a) the moveable representation of a' human body, capable of being shown carrying out the action or procedure, (b) means for recording speech and images, comprising at least a part of the instructional or teaching sequence, within a storage medium, (c) activity collecting apparatus including at least one of: a video camera focused upon a demonstrator; an array of electromyogram electrodes placed in relation to the demonstrator's muscles, and means to
120 detect the instantaneous forces applied by the feet of the demonstrator to the ground during a performance of the the action or procedure by the demonstrator, (d) means for applying said data onto the representation of a human body, and (e) means for integrating data collected by the activity collecting apparatus with the portrayal of the moveable representation of a human body into a teaching sequence.
125 Preferably each model included in the apparatus includes means to display selected muscles without covering, so that the relevant muscles are made visible in the instructional or teaching sequence.
Preferably the model is a virtual model, generated within a computer under control of software. In a third broad aspect, the invention provides a method for using a teaching sequence as 130 previously described in this section, wherein the method is based on developing kinesthetic appreciation by the trainee, wherein the method relies in part on demonstrating the role of muscles involved in the action to the trainee by means of a moveable representation of a human body so that the trainee can visualise the interplay of muscles involved in the action and consciously repeat the sequence in his own body.
135 Preferably the teaching sequence is displayed together with annotations , accompanying information, and exercises.
More preferably the teaching sequence includes software capable of providing for the inclusion into the sequence of data collected during use from the trainee. Preferred data includes images captured during the action, optionally with electromyogram data and data describing other 140 parameters including force. One preferred form of position data comprises at least one videotape record of the subject during the recording process. More preferably the positional data is capable of being geometrically corrected and is taken by a number of cameras.
Preferably the sequence comprises a visual demonstration capable of making use of kinesthetic signals, including those related to visual understanding of the muscles and postures involved in 145 carrying out the action.
In an associated aspect, the visual demonstration includes other procedures capable of enhancing the trainee's awareness of relevant kinesthetic signals. An example other procedure comprises an exercise for reinforcing awareness of a relevant muscle, tendon, joint, or posture.
Another example comprises generating a second mannikin representing the trainee, together 150 with muscle activity and resulting movements, according to data collected from the trainee in real time, so that the trainee can make his mannikin resemble the mannikin representing the skilled person, as seen on a display, by applying kinesthetically based "drive" to his muscles.
Preferably the teaching sequence is embodied in a repeatedly playable recorded medium, and example forms of display include: three-dimensional holograms, projected two-dimensional 155 images, images viewed upon screens including computer and video (television) screens, and screens showing images from projected movies.
In a fourth broad aspect, the invention provides a method for teaching and learning at least some of the activities involved in the game of golf, including the swing.
Alternatively the invention provides a method for teaching and learning at least some of the
160 activities involved in other sports, including rugby, pole vaulting, long jumps, javelin and discus throwing, weight lifting, and the like, and/or a method for teaching and learning at least some of the activities involved in non-sporting procedures, including the lifting of heavy weights, various rehabilitation activities such as alleviation of back pain or repetitive strain injury, walking and other sequences of movements for disabled people, and the like.
PREFERRED EMBODIMENT
165 The description of the invention to be provided herein is given purely by way of example and is not to be taken in any way as limiting the scope or extent of the invention. While we describe the golf swing as an example of an action to be imparted by means of kinesthetic training, a reader will appreciate that this invention is not limited in any way to golf, or indeed to any sport. Work-related actions, or rehabilitation after injury or stroke are other example
170 applications.
DRAWINGS
Fig 1: is a diagram of apparatus, showing how muscular activity information may be recorded for inclusion in a teaching sequence.
Fig 2a: is a perspective view; a line drawing (taken from Fig 2b) showing an example human 175 anatomical model or mannikin, with visible muscles, being used in a teaching sequence relating to a golf swing. Inputs and optional inputs are shown.
Fig 2b: is a perspective view; a shaded image taken from a 3D, lit, shaded computer-generated image showing the mannikin being used in a teaching sequence relating to a golf swing. (This display is preferably in colour).
180 Fig 3: is like Fig 2 but from a rear perspective.
Fig 4: is like Fig 2 but from an elevated perspective.
Fig 5: is like Fig 2 but is a closer view from yet another perspective.
Fig 6: shows a representation of a skeleton (a usually hidden layer of the mannikin) and a track of the path followed by the head of a golf club during an entire process, overlaid 185 on X-Y co-ordinates (the Z co-ordinate is repressed here).
The invention maximises the use of kinesthetic principles within a method for teaching a musculoskeletal sequence of movements (for an action or procedure), to a person.
The invention assumes that a "teaching sequence" will be developed for each subject. The teaching sequences are intended to be displayed in colour on (for example) a video (television)
190 screen, as a display generated by a programmed computer, or in other media including film,
DVD, "QuicktimeσM)" computer video, holograms, or the like. They may be distributed on videotape, DND or CD, interactive DND or CD, or a non-tangible distribution medium including cable, broad-band web, and the like. Sequences may be shown in a non-interactive manner on an appropriate player. For some applications, 16 mm film, 35 mm slides, or data 195 projector images are suitable. An interactive teaching sequence is preferred and may involve use of a computer programme (perhaps holding supporting data on a CD or DND, from which the programme is loaded for use) developed for the above purpose. Some applications, such as golf techniques, may merit or may economically justify more complex interactive teaching environments than others. One trainee at a time is assumed in this section.
200 The teaching sequence is based upon a virtual person; a human anatomical model or marmi n which serves as the central character for the teaching sequence and upon which are displayed cues for the kinesthetic aspects of the teaching programme. Hence this invention includes a physical basis for kinesthetic principles. Note that the mannikin to be described below is first generated during construction of the teaching sequence, and may later be either modified or re-
205 created as a second mannikin during an interactive sequence during which the trainee's own musculoskeletal activity is imported into software by means of transducers and displayed upon (or by) the mannikin such as for purposes of comparison or emphasising things to learn. In this situation, according to the invention, the trainee will be visually comparing his own musculoskeletal activity, muscle by muscle, with that of a demonstrator, rather than comparing the end
210 result, for example the path of the end of a club as was the case in the prior art.
Each mannikin (200 in Figs 2-6) is a representation of the human body upon which selected muscles involved in movements are, from time to time, displayed. The term "avatar" is used by some to refer to a virtual or computer-generated person. Speech and behaviour attributes are intended to be friendly and realistic. Because the human anatomical model is shown naked, yet 215 being sufficiently stylised (by virtue of being stripped down to the actual muscles) that most viewers are not distracted or embarrassed, the trainee is able to concentrate on fundamental actions involved. It is psychologically and generally useful to give the mannikin a personality, a voice, normal human movements, and have it become the instructor for the teaching sequence(s). Preferably the animated mannikin itself serves as the presenter of the programme.
220 The mannikin might be based upon a film or video record of a real person who might be a selected model with good muscle development and no fat. A low-cost way to create a programme creation starts with images of a real person carrying out the action at normal speed and reasonably perfectly. The person might be filmed wearing a skin-tight coverall or simply grey-painted skin (on at least the relevant parts of the body) so that muscle activity can be
225 added manually during subsequent editing. A movable mannikin may be created through use of (for example) drawings or the use of frame by frame shots of clay models as by the film producer Nick Park for his characters "Wallace & Grommit"). Initial teaching sequences using the mannikin were made on a frame-by-frame basis.
The inventor prefers to create the mannikin with an animation or 3D rendition computer 230 programme which will accept transducer inputs (as Fig 2a, a,b, and c.) according to Fig 1, and generate a mannikin in real time showing either the skilled person's or the trainee's activity. During teaching, this allows the trainee to learn the musculoskeletal sequence of movements in an interactive way. Current developments in simulation software will allow complete sequences of movements to be displayed based on (for example) numerical modelling of specific muscles, 235 taking into account the specific properties of each and their placement upon the skeleton, using instantaneous power generated over time for each of these. (That is a computer analogue of kinesthetics). Use of the mannikin has the intention that the trainee (such as a golfer under training) can be led to understand the musculoskeletal sequence of movements (as seen on the screen) in terms of events and actions which he can be made to perceive, in a conscious 240 manner, within his own body. Drills to raise kinesthetic consciousness of specific parts of the body may be used. The inventor believes that the principles of kinesthesia used in the invention provides an opportunity for the trainee to "feel" the important muscles (and possibly the tendons and joints as well). He can "see the feel". A trainee may not previously have been aware of these body parts, at a conscious level, while carrying out the action.
245 See Fig 2a, 2b where 200 indicates the human anatomical model, 201 is a golf club and 202 is a golf ball upon a tee. (This image is normally presented as a three-dimensional image in shaded, lit coloured form, and as part of a connected, moving sequence, not as the black line drawing of Fig 2a favoured in patent specifications). Muscles (eg 203 - m. vastus lateralis), tendons (eg 205 - patellar tendon), and bones (eg 204 - left femur) are shown here. Usually, all the relevant
250 muscles will be shown in any view. In most display modes each muscle is shown in a substantially correct configuration and position, with a red striped appearance, and preferably an emphasis (see later) if the muscle is active. Tendons, ligaments and other anatomical features are shown if relevant. It may be preferable to hide those muscles not involved in a particular movement beneath a simulated track suit, overall, or the like although in many actions, such as
255 a golf swing, most of the muscles of the body are likely to play a part. In order to display these, the model can be shown from a variety of viewpoints; the rear 206 as in Fig 3, from a raised viewpoint (Fig 4) or in close-up (fig 5) while carrying out the action.
The inventor bases all details on the textbook "Atlas of Human Anatomy" by Frank Netter (et al), particularly in relation to muscle disposition (including origins, positions, insertions, related
260 ligaments and actions). The software used for the flexible model includes a number of "layers" provided by the preferred computer package "Maya"® (Alias/ Wavefront, Toronto, Canada) used for 3D modelling. The first layer is a straight-line version of the skeleton, with lines joining various anatomically distinct points. Another layer is a complete skeleton (see Fig 6). The visible layer comprises a covering of muscles, ligaments, and bones as if skin and subcuta- 265 neous fat had been removed from a human body. The resulting model can be viewed from any direction and illuminated in order to show off rounded surface shapes, and is based on accepted anatomical texts so that the human anatomical model can be shown while carrying out movements involved in the action being imparted to the trainee.
Because the trainee learns how to carry out the action by seeing and feeling the muscular 270 sequence of activities displayed by the representation of a human, with muscles, including posture, movement, and specific muscular activities, the trainee can understand the action more easily and is better able to relate what is being shown to what he is doing himself. Full awareness of the status of a relevant muscle, tendon, joint, or of one's overall movement and posture (including sensors in the inner ear, the vestibular system) is relatively uncommon in 275 people. The invention makes use of this status information which may be deliberately brought out to reach a level of consciousness in a training session. This kinesthetic process of learning a complex musculoskeletal sequence of movements is believed to involve at least (1) the cerebral cortex, as well as (2) the basal ganglia and (3) the cerebellum in co-operative function, with the final sequence being held in the cerebellum. Hikosaka, O in Neurobiol Learn Mem 67(1) 29-33 280 (1997) offers a hypothesis suggesting that at least several sets of the above brain circuits work in parallel when a complex action is being acquired. In any case, specific knowledge of the or any actual site of learning within the central nervous system is not known to be very usef l for the purpose of this invention, given the difficulty of carrying out detailed brain research of such complexity. The invention works even without having that knowledge.
285 In practice, one kinesthetic approach to teaching the golf swing might have the mannikin ask the trainee to visualise a completely unrelated situation in order to grasp, or become aware of, a feeling generated inside the trainee's body. For example, when at the "address position", the trainee might be asked to imagine he is in a dark room. "Someone might push you over. Ensure that your legs have tension, but your upper body can give way to the push. Think
290 about your front-of-thigh muscles".
Optionally the teaching sequence is presented by means of a suitably programmed computer able to take input (Fig 2a, a1, b' and c' ... at right) from the trainee, using image sensors, in-shoe or on-fioor force transducers such as load cells, and EMG sensors and interpreting the results in terms of muscle activity, rather than as a simple non-interactive videotape (or the like). 295 Computer-based versions of teaching sequence medium, optionally copied onto a computer hard disk before use, are more suited to flexible user control. Many computer games can show realistic movements - and because of related demand, suitable programmes, computers and graphics cards are widely available. Accordingly, the trainee is optionally provided with a kinesthetic form of biofeedback teaching, facilitated through the programme which makes the 300 trainee's actions appear on the mannikin. The software displays a first mannikin, representing the instructor. If transducer inputs are available during a teaching session, a second (possibly simplified) trainee mannikin is displayed for comparison purposes.
In an interactive display mode, muscle tone and/or muscle activity is portrayed to the trainee preferably as a colour change or alternatively by some other intuitively recognisable indicator of
305 activity, such as colour, shape, flashing, or brightness; even sound. There is of course a difference between creating a force using muscles and actually causing movement; for example opposing muscles share some level of muscle tone even if no resulting movement occurs. It may be convenient to use an integrated sum of action potential activity as one way to determine what should be shown on the monitor during program creation, given the relative
310 impracticability of measuring tensions apphed at each muscle insertion. Resulting forces may be shown, or may be used internally by the computer running the teaching sequence as an indication of muscle activity. For example, a pressure transducer may be placed in each of the trainee's shoes in order to indicate balance and then movement, and another pressure transducer may be placed in the grip of the club in order to sense the extent of the grip. External
315 load cells may be preferred for use in a golf clinic. Pressure transducers may be placed in each of the the instructor's shoes (or in load cells) in order to acquire baseline data. Video camera data is likely to be required in order to support the load cell results, and the instructor may have electromyogram electrodes apphed to him; all of course having been connected by means of appropriate interface electronics to the computer.
320 More complex apparatus may be more suited to professional teaching rather than to a home- use product. The trainee data to be displayed on the human anatomical model can be compared with "proper" data by the computer (which should be programmed to offer corrective messages in terms of kinesthetic terminology), or by the trainee (and possibly by an instructor) in order to generate knowledge of any differences or shortcomings to be overcome
325 by further training. One must take care with interactive "see and do" sessions not to slip into the prior-art "copyist" mode of learning. Therefore we prefer to involve kinesthetic principles.
The design of an actual teaching sequence may be modelled on procedures used by effective instructors - in terms of the milestones set and how they are reached. Kinesthesiology relies on showing exactly what muscles are activated, by how much, when, and in what sequence. The 330 trainee is instructed to carry out the sequence as shown and to think about his own body and how his muscles feel as he does it — according to the principles of kinesthesiology. Learning processes may proceed at least in part by a process of error correction, so repeated trials, with feedback to the trainee indicating any errors, is useful. We may be creating a new task for the brain of the trainee - to understand a complex musculoskeletal sequence of movements by
335 watching a representation of muscle activity in another "person" and then replicate that action through the usual motor pathways. The text (spoken or otherwise) accompanying the visual demonstration will rely on the principles of kinesthesiology to a considerable extent, even though the words may relate to posture and balance while addressing the golf ball. Facilitation of this kind of learning may involve (for example) repeated replays of a sequence, slow-motion
340 displays, even .displays in an "exploded" view so that the trainee can discriminate activity of individual muscles more clearly, or the like. As shown in Figs 2-5, the mannikin can be viewed from many aspects, and close-ups are possible.
As part of an interactive procedure, the trainee's swing could be examined at say 6 points, by means of one or more video cameras that captured a sequence of steps. Rather than simply
345 providing corrections ("you were too high" for example) with the intention of copying an instructor's position, the computer program nning within processor 111 running the training sequence is programmed to issue corrections in kinesthetic terms, detailing what the trainee's musculoskeletal system should be doing at or by each point. The program would interpret information derived from one or more video cameras showing body and club position, and
350 optionally also from load cells under the trainee's feet, club tip position sensors, or electromy- ograms. This would be done in real time. (Note that interaction, and sensors upon the trainee are options that enhance the basic principles). We prefer that a second mannikin be brought onto the screen, transforming what the trainee is doing into the same format as the instructor. The trainee can visually compare his marmikin with the instructor mannikin and adjust his own
355 muscle activity or sequence, as well as resulting movements, posture and position, so as to match that of the pre-recorded instructor mannikin. If the trainee was achieving a similar posture with the wrong muscles this should be clearly evident. (Note: Here, both the trainee and the instructor marinikins may be generated "five") and no pre-planned teaching sequence is in use).
360 EMGs are a physiological, recordable modality that is related to neuromuscular activity and which may be used by the demonstrator, or in an interactive way during learning. The inventor has found electromyography as used upon the trainee technically difficult to apply because of wires. A wireless, self-contained stick-on electrodes and preamplifier unit would avoid that problem. EMGs are more likely to be brought into use during a teaching session if the session
365 is under the supervision of a golf professional in a golf academy (or corresponding venues for other sports or other aspects of personal training). The processor 111 as used by a "standard" trainee is preferably provided with software capable of interpreting combinations of image data and/or load cell data as events occurring within the body of the human subject. This software is likely to be particularly useful when applying trainee data onto the mannikin in real time 370 during a teaching sequence, helping the trainee to visualise and understand his own body in terms of kinesthetic principles.
EXAMPLE 1
In order to create a teaching sequence for teaching the golf swing, we used a procedure for collecting data from real persons and integrated that data to make the frame-by-frame sequence 375 of the human anatomical model in action.
1. Sports medicine facilities (kindly provided by the Tamaki Campus, University of Auckland) placed a set of video cameras arranged in a precisely known range of positions (as indicated by 120, 120A in Fig 1) in relation to the subject for studying motion by recording images of the frame-by-frame position of bright dots placed on dark clothing during the execution of
380 an action (a golf swing) by the subject.
2. Electromyogram (EMG) data was also collected from specified muscles during the execution of an action (a golf swing) by the subject (see 101-107 in Fig 1).
3. Conversion of the position information onto the simplest level in our human anatomical model (a "stick" simplification of the skeleton,) made it compatible with the simplified
385 skeleton used in motion study techniques. We mapped the moving skeleton display from the video cameras onto our stick skeleton, frame by frame, so that the representation of a human could be certified as carrying out the correct movements. For some aspects of a movement, it is useful to track the course of an object over time. Fig 6 shows the path (601) before, and 602 (during and after) the swing, taken by the head of the golf club 201 when
390 the skeletal human 200 performs the action. This particular illustration lacks perspective and may be more use to the course developer than to a trainee who is normally shown a perspective view; a three-dimensional illustration.
4. We then "fleshed out" our stick skeleton with a real skeleton, and added anatomically correct muscles and other relevant body parts to fill all gaps, (see Figs 2-5) so that the
395 human anatomical model appeared complete and in colour.
5. We superimposed electromyogram (EMG) data from the demonstrator (or an average of many of them) as described below, so that the person learning can see (by a visually distinctive sign, such as a glowing or swelling appearance superimposed on the model) when underlying muscles are active. This display is shown diagrammaticalty in Fig 1, where the
400 display screen 112 shows a replay view of part of the arm 122 bearing areas such as 115 which are changed in appearance during activity of the underlying muscle(s), as previously recorded (such as by electrodes 101A, 102A). Muscle activity and forces appleid through feet and hands may also be stored on a videotape sound track, in computer files, or in some other synchronised recording medium for use in comparative demonstrator/trainee inter- 405 active sessions.
6. According to the nature of what is being taught, the sequence is repeated a number of times or from a variety of angles, in order to show other parts of the body (e.g. the back) and it may also be repeated in slow motion.
7. We then supply annotations, for instance appropriate words or text as audio channels or 410 visual overlays or subtitles, in order to support the kinesthetic principles of the sequence.
Versions in different languages (English or Japanese) are saved in separate channels.
Interactive sections are included if the distribution medium is suitable. Otherwise an accompanying manual, or an actually present trained demonstrator might be used. The kinesthetic teaching sequence assists the trainee to become aware of his own muscles and other body parts 415 in action. Optionally our training programme may include exercises intended to over-use specific muscles so that the trainee is acutely aware of them during a learning session.
As shown in Fig 1, muscle activity can be quantified in a multi-channel data collection device 100 by detecting the EMG; typically a signal of a few millivolts which can be picked up by contact skin electrodes (101 and 102, paired) or with unipolar needle electrodes from within the
420 muscle under study. (At left, a person's arm 122 has a first pair, 101 A, 102 A of electrodes placed on the skin over a biceps muscle and a second pair, 101B, 102B placed over m. extensor digitomm.) Preferably a number of muscles are sampled simultaneously by n separate channels, and a multi-channel amplifier/recorder would be appropriate. Each signal is amplified (differential amplifier 103) and recorded for later analysis on a preferably synchronised storage device
425 104, such as an audio channel of a tape recorder, or the video recorder 110, or digitally stored within a computer 111. On replay the amplitude and/or the frequency of the signals is detected in box 105 then integrated with a short time constant of perhaps 0.1 second within box 106, so becoming converted into an activity-related voltage available at output 107 (or 108, 109). Fig 1 also shows a video recorder 110 and video cameras 120, 120A connected by lead 121 to the
430 processor/mixer 111 which is typically a digital computer having one or more analogue input boards and one or more video editing boards inserted (as is well known to those skilled in computer applications), together with appropriate software, and is passed to a display device 112 (with control means 113). More than one video camera (as 120A) may be used (according to well-known techniques) in order to gain three-dimensional data collection. 435 In addition to visual imagery, the device 111 may conveniently be supplied with pressure information from a pair of force transducers (commonly known as load cells) 116, 118; one under each foot of the human subject. These devices may be included within the trainee's shoes. All this information helps describe the demonstrator's or the trainee's resting position and actual movements. Sensors (such as accelerometers) on the club itself, or pressure sensors within the
440 grip of the club or within gloves may also assist in describing the resultant actions.
One way to superimpose electromyogram data onto the marinikin uses an editor to create a colour box superimposed in real time on each image, in which the colour shown over each muscle (115) in each frame is proportional to muscle activity (as derived in 105 and 106). A person can emulate a "paint by numbers" process to colour the image of a specific muscle, 445 frame by frame, during editing. (We have already referred to use of a "trainee mannikin" generated by software in real time with muscle activity and posture made visible, alongside an "instructor mannikin", making it easy for the trainee to match the colours and closely emulate the instructor - to "get inside his body")- The software can also replay sequences slowly, time after time, and issue corrective messages preferably couched in kinesthetic terminology.
450 It may be preferable to play an audio analogue of muscle activity. A trainee could select a particular muscle or group by mouse action, then the programme would cause a sound to be made for that muscle — whether it is an instructor's or the trainee's muscle. Sound is suitable for comprehending rapid sequences. Muscle force can be quantified by use of a force transducer (typically a strain gauge or sometimes a load cell) placed appropriately in a shoe or in a
455 glove, and recorded in a further channel, in order to describe an overall result of action. The option of having the EMG processing apphed subsequently to the storage stage (104) allows the user to adjust the parameters if required. EMG data requires interpretation, because of factors including basic muscle tone, effects of opposing sets of muscles, body inertia, action/reaction, and consequences of preceding and current activity by remote muscles. Using transducers
460 apphed to something grasped (pole of pole vaulter, handle of golf club, etc) is helpful.
Use of simultaneously collected EMG inputs (107, 108, 109) to the processor/mixer 111 provides inter-relationship information. Although only one muscle group may be edit or enhanced on the human anatomical model at a time, knowledge of other activities that are also occurring at the same time is useful. It is possible that actual displays of real data derived 465 from physiological procedures such as those described above should be modified and simplified in order to lead a trainee towards making the right sequence of actions.
The machine itself may overlay the colours over each of the muscles in question, either onto a human anatomical model or onto an image of a person. Sometimes the use of an actual person with muscle activity superimposed onto the skin is an inadequate solution to the problem of 470 providing an effective teaching machine. In that case, a first approximation to a complete program may be obtained by use of the actual person, who is then edited out and replaced by a computer-generated marmikin. The mannikin may be created or shaped during construction of a teaching sequence on a frame-by-frame basis to replace/ overlay the image of either the person who acts as a demonstrator, The same process may be used to replicate the trainee's
475 muscle activity upon the, or another, mannikin during replay.
VARIATIONS
It will be clear to a person of skill that the methods behind this illustrative example (the golf swing) may be used when teaching other complex motor activities with little modification of the basic principles. This invention is not limited to golf or to any sports. Work-related actions,
480 such as Hfting weights, minimising computer-related repetitive strain injury or postural problems, or rehabilitation after injury or stroke are deserving of attention. The idea of an animated muscle model may be transferred to animal versions, and used to enhance the teaching of animal anatomy, locomotion, and diagnosis of lameness to veterinary students. Use of slow motion and stop-frame facilities is always likely to be helpful, given that many sports-
485 related actions are quite quick and complex, and include many near-simultaneous steps.
Although teaching sessions are designed for a solitary trainee, multiple trainees (especially in the case of non-interactive versions), and/or actual instructors or demonstrators (optionally with their marmikins) may be present, especially at training clinics.
COMMERCIAL BENEFITS or ADVANTAGES
490 Because the trainee leams how to carry out an action with a kinesthesiological approach, the trainee can understand the action more easily "from the inside" or "by process, not by result" and so relate what is being shown to him to what he is doing himself. Prior-art instruction has focused on result, not mechanisms by which the result is achieved. A person learning golf (for example) without access to the present invention may have had many opportunities to watch
495 other people, and may have learnt, mis-learnt (or been taught) all that he can learn in that way.
Finally, it will be understood that the scope of this invention as described and/or illustrated herein is not limited to the specified embodiments. Those of skill will appreciate that various modifications, additions, applications, known equivalents, and substitutions are possible without departing from the scope of the invention as set forth in the following claims.

Claims

500 I claim:
1. An instruction method or teaching sequence using virtual apparatus for instmcting a trainee how to carry out at least one action or procedure in an improved manner; characterised in that the teaching sequence provides material leading to kinesthetic appreciation by the trainee of those portions of the trainee's musculoskeletal system involved in the action or 505 procedure, and the teaching sequence includes a visual portrayal of selected portions of the musculoskeletal system upon a first representation of a human body shown carrying out the action or procedure, thereby leading the trainee to use kinesthetic discrimination and/or perception principles in order to enhance performance of the at least one action or procedure.
510 2. A teaching sequence as claimed in claim 1; characterised in that the visual portrayal of selected portions of the musculoskeletal system, including selected muscles, upon the first representation of a human body is substantially anatomically correct.
3. A teaching sequence as claimed in claim 1; characterised in that the first representation of a human body is made capable of appearing to speak to the trainee by means of replay of
515 recorded speech together with appropriate motions, and is provided with the capacity to move and thereby demonstrate the at least one action or procedure; so that the first representation of a human body serves as a virtual instructor.
4. A teaching sequence as claimed in claim 1, characterised in that the teaching sequence is capable of including a portrayal of muscle activity displayed with reference to the first repre-
520 sentation of a human body, over time; the portrayal of muscle activity being based on data collected and recorded from at least one skilled person while performing the action • or procedure; so that the teaching sequence includes a visual display of information representing the related muscle activity.
5. A teaching sequence as claimed in claim 4, characterised in that the teaching sequence 525 further includes a portrayal of movements generated as a result of activity of the musculoskeletal system; data describing the movements having been collected and recorded from at least one skilled person while performing the action or procedure.
6. A teaching sequence as claimed in any one of claims 1 to 5, characterised in that the sequence provides, when in use, a visual display of at least some of the muscles of a human
530 shown upon the first representation of a human body, and presents the individual and collective activities of the muscles and movements in relation to the kinesthetic sensations that the trainee would perceive while while performing the action or procedure.
7. A teaching sequence as claimed in claim 6, characterised in that the sequence provides, when in use, a visual demonstration of a display of the muscles of the trainee shown upon a
535 second representation of a human body, and displays the individual and collective activities of the muscles with resulting movements thereof so that the trainee may perceive the actions of his body in kinesthetic terms while performing the action or procedure.
8. A teaching sequence as claimed in claim 7, characterised in that the sequence provides, when in use, a visual demonstration of the display of the muscles of the at least one skilled
540 person shown upon a first representation of a human body, and the second display of the muscles of the trainee shown upon the second representation of a human body, so that the trainee may compare actions of his body in kinesthetic terms with those of the at least one skilled person while performing the action or procedure, and may adjust activity within his body in order to match those of the at least one skilled person.
545 9. A teaching sequence as claimed in claim 5, in claim 6, or in claim 7, characterised in that the visual demonstration includes at least one exercise directed at reinforcing awareness by the trainee of activity of a relevant portion of the musculoskeletal system, in relation to the resulting change in body position.
10. A teaching sequence as claimed in claim 6, characterised in that the teaching sequence 550 makes use of computational means allowing the trainee to compare stored muscle activity with muscle activity data collected from the trainee by means of activity collecting apparatus and passed to the computational means during the instructional sequence while performing the action or procedure; the trainee activity collecting apparatus including at least one of: a video camera focused upon the trainee; an array of electromyogram electrodes placed in 555 relation to the trainee's muscles, and means to detect instantaneous forces applied by the feet of the trainee to the ground; said trainee activity collecting apparatus being connected through interface means to the computational means.
11. A teaching sequence as claimed in claim 10, characterised in that the data collected from ' the trainee during the instructional sequence is portrayed upon a second representation of a
560 human body, representing the trainee carrying out the action.
12. A teaching sequence as claimed in claim 10, characterised in that the data collected from the trainee during the instructional sequence is used by the computational means, in conjunction with stored data, in order to issue kinesthetically based corrective information portrayed upon a representation of the trainee while carrying out the action.
565 13. Apparatus for use when making an instractional or teaching sequence as claimed in claim 1; characterised in that the apparatus includes: (a) at least one representation of a human body, capable of being shown carrying out the action or procedure, fb) means for recording speech and images, comprising at least a part of the instructional or teaching sequence, within a storage medium, (c) activity collecting apparatus including at least one of: a video
570 camera focused upon the at least one skilled person; an array of electromyogram electrodes placed in relation to the skilled person's muscles, and means to detect the instantaneous forces apphed by the feet of the skilled person to the ground during a performance of the the action or procedure by the skilled person, and (d) means for integrating data collected by the activity collecting apparatus with the portrayal of the at least one representation of a
575 human body into a teaching sequence.
14. Apparatus for making an instractional or teaching sequence as claimed in claim 13; characterised in that the at least one representation of a human body includes means to display selected muscles without covering, so that the relevant muscles,and activity of said muscles are made visible in the instructional or teaching sequence.
580 15. Apparatus for making an instractional or teaching sequence as claimed in claim 14; characterised in that the at least one representation of a human body is a virtual representation, generated within a computer under control of software.
16. Apparatus for making an instractional or teaching sequence as claimed in claim 15; characterised in that the second representation of a human body is generated within a computer
585 under control of software responsive to data collected by a second set of the activity collecting apparatus, applied to the trainee.
17. A method for using a teaching sequence as claimed in claim 1; characterised in that the method is based on developing kinesthetic appreciation by the trainee, wherein the method relies in part on demonstrating the role of muscles involved in the action to the trainee by
590 means of a representation of a human body so that the trainee can visualise the interplay of muscles involved in the action and consciously repeat the sequence in his own body.
18. A method for using a teaching sequence as claimed in claim 17; characterised in that the method is further based on providing the trainee with means to compare his second representation of a human body, including muscles of the trainee and activity within said
595 muscles, with the first representation of a human body including muscles of the skilled person and activity within said muscles, so that the trainee can perform kinesthetic adjustments to his muscles in order to match the other representation of a human body.
PCT/NZ2003/000256 2002-11-20 2003-11-20 Instruction method using virtual apparatus WO2004045725A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2003283887A AU2003283887A1 (en) 2002-11-20 2003-11-20 Instruction method using virtual apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
NZ522682 2002-11-20
NZ52268202 2002-11-20

Publications (1)

Publication Number Publication Date
WO2004045725A1 true WO2004045725A1 (en) 2004-06-03

Family

ID=32322583

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/NZ2003/000256 WO2004045725A1 (en) 2002-11-20 2003-11-20 Instruction method using virtual apparatus

Country Status (2)

Country Link
AU (1) AU2003283887A1 (en)
WO (1) WO2004045725A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010017700A1 (en) * 2010-07-01 2012-01-05 Mirjam Cordt Virtual interactive live remote coaching- and training system i.e. sports remote coaching- and training system, for coaching-and training e.g. yoga for student in fitness studio, has reproducing unit reproducing audio signal
US20120128201A1 (en) * 2010-11-19 2012-05-24 Microsoft Corporation Bi-modal depth-image analysis
JP6440335B1 (en) * 2017-12-22 2018-12-19 一成 白崎 Golf foam training equipment

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5554033A (en) * 1994-07-01 1996-09-10 Massachusetts Institute Of Technology System for human trajectory learning in virtual environments
US5577981A (en) * 1994-01-19 1996-11-26 Jarvik; Robert Virtual reality exercise machine and computer controlled video system
US5904484A (en) * 1996-12-23 1999-05-18 Burns; Dave Interactive motion training device and method
US20020031753A1 (en) * 2000-06-28 2002-03-14 Shigeo Yanai Instructional method for sports, hobbies, amusements and for activities in the daily environment, etc.
US6503086B1 (en) * 2000-04-25 2003-01-07 Michael M. Golubov Body motion teaching system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5577981A (en) * 1994-01-19 1996-11-26 Jarvik; Robert Virtual reality exercise machine and computer controlled video system
US5554033A (en) * 1994-07-01 1996-09-10 Massachusetts Institute Of Technology System for human trajectory learning in virtual environments
US5904484A (en) * 1996-12-23 1999-05-18 Burns; Dave Interactive motion training device and method
US6503086B1 (en) * 2000-04-25 2003-01-07 Michael M. Golubov Body motion teaching system
US20020031753A1 (en) * 2000-06-28 2002-03-14 Shigeo Yanai Instructional method for sports, hobbies, amusements and for activities in the daily environment, etc.

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102010017700A1 (en) * 2010-07-01 2012-01-05 Mirjam Cordt Virtual interactive live remote coaching- and training system i.e. sports remote coaching- and training system, for coaching-and training e.g. yoga for student in fitness studio, has reproducing unit reproducing audio signal
US20120128201A1 (en) * 2010-11-19 2012-05-24 Microsoft Corporation Bi-modal depth-image analysis
US9349040B2 (en) * 2010-11-19 2016-05-24 Microsoft Technology Licensing, Llc Bi-modal depth-image analysis
JP6440335B1 (en) * 2017-12-22 2018-12-19 一成 白崎 Golf foam training equipment
JP2019111046A (en) * 2017-12-22 2019-07-11 一成 白崎 Golf form training tool

Also Published As

Publication number Publication date
AU2003283887A1 (en) 2004-06-15

Similar Documents

Publication Publication Date Title
White et al. Use of different imagery perspectives on the learning and performance of different motor skills
US5823786A (en) System for instruction of a pupil
US6183259B1 (en) Simulated training method using processing system images, idiosyncratically controlled in a simulated environment
Spelmezan An investigation into the use of tactile instructions in snowboarding
Hodges et al. Modelling coaching practice: the role of instruction and demonstration
Van Der Linden et al. Musicjacket—combining motion capture and vibrotactile feedback to teach violin bowing
US6164973A (en) Processing system method to provide users with user controllable image for use in interactive simulated physical movements
Fernandez-Cervantes et al. VirtualGym: A kinect-based system for seniors exercising at home
Scully et al. Observational learning in motor skill acquisition: A look at demonstrations
US7095388B2 (en) Method and system for developing consistency of motion
Melero et al. Upbeat: augmented reality-guided dancing for prosthetic rehabilitation of upper limb amputees
JPS6379684A (en) Physical athletic proficiency teaching system
Li et al. Development of exergame-based virtual trainer for physical therapy using kinect
CN1064854C (en) System for instruction of papil
Ruttkay et al. Elbows higher! Performing, observing and correcting exercises by a virtual trainer
US20080003554A1 (en) Interactive system and method whereby users direct idio-syncratically controllable images to reverse simulated movements of actual physical movements, thereby learning usual sequences of actual physical movements
Camarillo-Abad et al. An environment for motor skill transfer based on wearable haptic communication
WO2004045725A1 (en) Instruction method using virtual apparatus
Schoonover et al. The use of videotape programs to teach interpersonal skills
IJsselsteijn et al. Virtual Cycling: Effects of immersion and a virtual coach on motivation and presence in a home fitness application
Burns On the relevance of using virtual humans for motor skills teaching: A case study on karate gestures
Hapsari et al. Dental health education game based-on IRVO model in augmented reality technology
CZ9702180A3 (en) Device for instructing hockey players
AU682528B2 (en) A system for instruction of a pupil
CN112534491A (en) Medical simulator, medical training system and method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP