SYSTEM, APPARATUS, AND METHOD FOR TELEMETRY AND MONITORING OF
DESIRED TARGETS
CROSS-REFERENCE TO RELATED APPLICATIONS
The present application claims priority from provisionally-filed U.S. Patent Application No. 60/145,621, filed July 26, 1999, entitled "System, Apparatus, And Method For Telemetry And Monitoring of Desired Targets."
BACKGROUND OF THE INVENTION
The invention generally relates to an apparatus and method for monitoring desired targets. More specifically, the invention involves collecting, transmitting, and or displaying data and/or images of one or more targets.
Various methods for displaying and monitoring desired subjects, such as athletes involved in competition, are known. For example, television broadcasts provide images and other information about athletes and competitions. Visual images of the subjects alone do not, however, always provide all the information a viewer or other user may desire or require. For example, a team coach or physician may not easily be able to determine the physical condition of an athlete during the competition merely by viewing the athlete from a distance. Additionally, a viewer of the athletic competition may desire more detailed information, such as acceleration data, that may not be readily apparent from visual images of the athletes.
What has been needed and heretofore unavailable is an improved method and apparatus for monitoring and displaying characteristics of various targets. The present invention satisfies this need.
SUMMARY OF THE INVENTION:
The invention is a system, apparatus, and method for monitoring characteristics of a desired target. The desired target can be a person, animal, plant, or inanimate object. More specifically, the invention is a system, apparatus, and method for collecting, transmitting, and/or displaying data and/or images of one or more desired targets.
Note that the term "target" is used to reference the person, animal, plant, or
inanimate object that is being monitored by the invention. The invention may be used to monitor individuals involved in various activities, or who require medical monitoring. The invention may also be used to monitor devices, including athletic equipment such as baseball bats and footballs, that are subject to movements.
The remote unit may be a telemetry transceiver device capable of sensing, acquiring, processing, transmitting, receiving, recording, and/or displaying real-time physiological and other performance data from the target to one or more receiving sites capable of receiving the transmitted signal. The receiving site may include a receiver for receiving the data and a processor for processing the data. A display may also be provided, which may be part of the receiving site.
The telemetry signals can be sent from multiple targets to one or more receivers wirelessly, and from the receiver can be forwarded in real-time to a personal computer or other processor, and then on to other media such as the Internet and/or television broadcasts. The transceivers themselves may have embedded TCP/IP code, thus allowing each target to be its own web page or URL. Operationally, all data from a target can be accessed in real-time over the Internet by the unique URL. the individual web page of each target and its associated data can be accessed by a web server via the Internet and viewed by multiple users at the same time.
The remote unit on the target may consist of an ASIC, FPGA, CMOS IC, and/or can be implemented in several integrated circuit, chip-size modular sections. Sensors, A/D convertor, processor, transceiver, and/or antenna may be located in a small (such as pager-size or smaller) package that transmits to one or more receivers. The Data Acquisition (DAQ) section would monitor and measure functions of the target, for example, if the target were an athlete, the DAQ section would monitor and measure functions such as ECG, heart rate, breathing rate, blood pressure, positive and negative acceleration, positive and negative G's, and vibration. Various sensors could be used, including analog and/or digital, as well as MEMS type. The sensors may all be non-invasive, depending on the particular application.
The receiving site recovers data from one or more remote units in the field. The receiving site can send the data to a computer where the data is sent to the Internet of to some other media where it can be broadcast to viewers, or accessed upon demand for viewers. In addition, multiple receivers can be used to provide wireless, headset viewer access to the live video, audio, and/or data information that is being transmitted from the target.
The invention may include a real-time data acquisition system that is capable of sensing, acquiring, processing, transmitting, receiving, recording, and/or displaying remote analog, digital, and/or serial data in real-time wirelessly to the Internet. The design allows dat from a sensor to be immediately transmitted wirelessly to the Internet where data can be displayed and/or accessed by multiple users. The system may operate on the Ethernet standard and incorporate two distinct sections: The Data acquisition and processing section and the transceiver section (such as an RF transmitter/receiver). Both sections may have embedded TCP/IP network capable protocols. The system can thus allow for command, control, and communication in real-time, wirelessly, to and from remote units and to and from the network and/or the Internet.
One embodiment of the invention comprises a remote unit to be worn on a target. The remote unit is capable of sensing, acquiring, processing, transmitting, receiving, recording, and/or displaying real-time data, such as physiological and other performance data, from the target to a receiver capable of receiving the transmitted signal. The remote unit may consist of a data acquisition circuit, a communication circuit (such as an RF circuit), and one or more sensors. The elements of the remote unit may be integrally formed on a self- contained application-specific integrated circuit (ASIC) chip and/or can be implemented in several CMOS integrated circuits, chip-size modular sections. Sensors, A/D converter, processor, transceiver, memory, and/or antenna can all be located in a small (such as pager- size) package that transmits to a receiver. One or more, or even all, of the elements of the remote unit, such as the sensor(s) and/or antenna, may be embedded in a substrate, such as the substrate of a chip that forms the remote unit. The antenna may be a planar antenna or traced onto the substrate with copper or graphite such that it is integral with the chip set.
Where the target is an athlete, the remote unit can be worn by the athlete, either directly on the athlete's body, or on or inside of a piece of apparel, such as a helmet, wristband, shoe, or other padding. The remote unit can include one or more sensors for gathering biometric data about the athlete's physiological condition, such as ECG data, blood pressure, heart rate, perspiration rate, and body temperature. Other biometric data may include chemical analysis of perspiration and/or blood, such as monitoring of blood sugar levels and other blood chemistry monitoring.
The remote unit may also include sensors for gathering data regarding the acceleration, speed, vibration, movement, and/or location of the target. In one embodiment, the remote unit includes an accelerometer that determines accelerations of the target in one or
more directions. In one embodiment, the accelerometer can determine accelerations in 3 axes. The acceleration data can be in the form of "g-forces" or other units as desired for a particular application.
In one embodiment, the accelerometer is a tri-axial accelerometer, such as a solid-state angular rate sensor sold by Systron Dormer Parts under the name Gyro-Chip(TM). Another device that might be applied to the invention is a quartz flexure accelerometer sold by the same company. One or more of the sensors may be MEMS (micro-electromechanical systems) sensors, which may be configured to be integrated into a single chip that forms the remote unit. Such integration could be in the form of building the MEMS sensor or sensors, such as a MEM accelerometer, into the substrate of the actual chip.
The sensors could be of various types, including analog, digital, MEMS, and/or other sensors. The sensors may all be non-invasive, particularly where the target is a human.
The remote unit may include a transmitter that transmits the biometric and/or other data from the remote unit to a receiving site, which may include as a processor that receives the data and provides the data in a desired format to a display. The remote unit may also include a receiver for receiving transmissions from various sources. The remote unit may include a combined transmitter and receiver, known as a transceiver, that receives and transmits data.
In one embodiment, the biometric and/or other data is transmitted from the remote unit in an Internet-compatible format, such as so-called TCP/IP format, so that the data is immediately accessible through internet protocols. The remote unit may be equipped with a transceiver (receiver and transmitter) so that it can transmit and receive data in Internet-compatible format. Each remote unit may act as a stand alone network device, with an individual network address for each remote unit that can be accessed by Internet users.
In another embodiment, the sensors themselves are configured to provide the biometric and or other data in Internet compatible format. In an embodiment, the remote unit includes a memory for storing the sensor data. The memory may be in various forms, and may provide short-term, long-term, and/or permanent storage of data.
In one embodiment, the remote unit includes a GPS receiver for receiving GPS position data from GPS satellites. The remote unit can store the GPS data, and/or transmit the GPS data to another location, such as a network address or a desired display.
In one embodiment, the remote unit is part of a system that includes one or
more remote units on one or more targets, one or more receivers for receiving data from the remote units, one or more processors for processing the data, and one or more displays for selectively displaying images of the targets on the screen, simultaneously with representations of the biometric and/or other data collected from the remote units attached to the targets. The representations of the biometric and/or other data may be in various forms, such as graphical and/or numerical representations of the data.
In applications such as athletic competitions, multiple remote units and personal display units could be used in sports stadiums to provide viewer access to the live video, audio, and/or data information that is being transmitted from the athletes. The link to the personal display units could be wireless, via a local computer server, and/or via a direct connection, or the display units could be located in the seat-backs and data could be accessed by the stadium fan. The personal display units could be rented out to spectators for a fee at a game. The personal display units could be adapted to provide a televised or similar sportscast of the game for an up-close experience. In an embodiment, dedicated cameras situated around the arena can be used to collect video/audio to a processor linked to the personal display units. Each personal display unit may be thin client-type computing device with a video monitor or other visual display. The system could provide real-time telemetry data from the athletes to the personal display unit, with the wearer selecting the particular telemetry data and athletes desired for viewing. The system could also provide non-real time data about the athletic competitions and/or athletes, such as athlete statistics that may be provided to a viewer, with the viewer selecting the type of data desired for display. A spectator could select the video/audio from any of the available cameras for viewing along with real-time telemetry data from several of the athletes on the field, rink, or other playing area.
The remote unit may include a power source. The power source may include a battery, solar array, capacitor, and/or other source for providing power to the remote unit. Other power sources that might be used include sources that derive power from the target, such as an athlete. For example, heat and/or perspiration by the athlete could be used as a power source, as well as movement of the athlete (which can be harnessed through means such as the winding mechanism for a so-called "self-winding" watch). Such alternative power sources may have particular applicability for remote units that use relatively small amounts of power.
The system of the invention may include a system memory for storing the data
transmitted by and/or received from the remote units. Such a system memory can be in lieu of, or in addition to, the previously-discussed memory positioned on or in the remote sensor. The system may also include a system processor, apart from or in place of any processor on the remote units, that processes the data from the remote units.
The system of the invention can include one or more remote units providing data from one or more targets. If multiple targets are involved, the remote units each preferably provide the data with a format or frequency or other differentiating characteristic that permits data from a particular target to be recognized as coming from the remote unit on that target.
The system of the invention can include one or more cameras to provide images of the target and/or of the target's environment. The camera may be positioned on the target and may form a part of the remote unit. The camera may be positioned at a distance from the target, in order to provide a desired image of the target in the particular environment. For example, in a football game, remote units can be attached to various players, and one or more cameras may be positioned so as to provide a view of the football teams' members on the playing field during and between various football plays. The camera(s) can provide images to the display for simultaneous presentation with the biometric and or other data from the remote unit. The data and image from the camera(s) are preferably correlated, so that the data displayed correlates to the action of the target. For example, g- forces experienced by a football player could be displayed in real-time in a windows (picture- in-picture) format positioned in a corner of a larger view showing the movements of the player.
A camera could also be positioned on or in an inanimate object, such as a camera positioned inside and looking out of an end of a football. Note that such cameras may provide unsteady images, particularly where the target is moving. The system may provide for image stabilization that compensates for such movements. For example, a camera looking out of a football would, during a spiral pass, provide a rotating and probably wobbling image. The system could, through various techniques such as the use of software, provide a stablized image to the display that compensates for the rotation and other wobbling movement.
The system of the invention may also include one or more microphones that provide audio data. The microphones may be positioned on or near the target, and may be part of the remote unit.
The system may include one or more displays, with different data provided to
different displays. For example, biometric data may be provided to one screen for private viewing by an athlete's physician or coach, while data regarding speed, position, and/or acceleration can be provided to a second set of displays intended for public viewing.
The displays of the invention can, in one embodiment, provide viewing of an athletic event with simultaneous viewing of biometric and/or other data about one or more athletes in the event. In one embodiment, the biometric and/or other data is provided on the same screen as the actual viewing of the athletic event. The biometric and/or other data may be provided in a format and manner that enhances, preferably without detracting from, the viewing of the athletic event itself.
The displays may be part of a personal computer, television set, or other display. The data can be sent to the display through various methods, including the Internet, World Wide Web, the Internet via an Application Service Provider software program residing either on a client web-site or our own enabled with a browser specific health related "my health" box located on the browser menu bar, on cable television systems, broadcast systems, satellite transmission, and/or phone lines.
The invention can be applied to various applications where data, such as location, movement, or biomedical information, is desired from a target. For example, the invention could be applied to a wide variety of sports and recreational activities, including professional and amateur sports. The invention could also be used in military and civil applications to train and monitor personnel. For example, soldiers, pilots, firefighters, police, and hazardous materials specialists could use the invention to enhance their effectiveness, in both training and real-life situations. The invention could also be applied to entertainment venues. Television game shows could make use of the invention, where, for example, a contestant was equipped with remote unit providing heart rate data that can be shown to a viewing audience to indicate the contestant's heart rate during the contest. The invention may also be used to monitor individuals who require medical monitoring, such as individuals in medical care facilities, nursing homes, or who require home care and monitoring, and passengers in automobiles, airplanes, and boats or ships. The invention could additionally be used to monitor persons whose movements are desired to be restricted, such as individuals who imprisoned or under house arrest.
The invention could also be applied to non-human targets, such as for use in monitoring livestock, commercial packages, etc. For example, remote units could be used to monitor the condition and/or movements of various farm and ranch animals, including cows,
pigs, and chickens. A rancher could use the invention to monitor the condition and/or movements of many individual animals, with the data maintained in a detailed database on a central computer that correlates condition and or movements with time. The invention could have particular application in monitoring animals, such as cows and horses, that may range over a large field or other area. The invention could also be applied to free or wild animals, such as deer or fish. The invention could also be applied to monitoring the condition and/or movements of plants and inanimate objects.
The invention thus allows for simultaneous access to target information, in real-time and/or archival formats, that cen be easily integrated and customized according to the needs of the particular broadcast venue.
The data from a particular target can be stored to archived files as a portfolio for the particular target. For example, for athletic competitions, the data for a particular athlete can be archived as player statistics. The data for such statistics could cover a single game, a season, or a career of the particular athlete. Data from various targets, such as various players, could be cross-referenced and compared with such archived files.
BRIEF DESCRIPTION OF THE DRAWINGS
FIGURE 1 shows an overview of a block diagram showing a system according to an embodiment of the current invention;
FIG. 2 depicts a block diagram of a system in accordance with a further embodiment of the invention;
FIG. 3a and 3b depict block diagrams of a camera control system in accordance with an embodiment of the invention.
FIG. 4 depicts an overview of a system according to an embodiment of the invention;
FIG. 5 depicts a block diagram of a remote unit in accordance with an embodiment of the invention;
FIG. 6 depicts a block diagram of a power supply in accordance with an embodiment of the invention;
FIG. 7 is a flowchart depicting the operation of aspects of a remote unit according to an embodiment of the invention;
FIG. 8 is a flowchart depicting the operation of aspects of a processor according
to an embodiment of the invention;
FIG. 9 depicts a display in accordance with an embodiment of the invention; FIG. 10 depicts a display in accordance with a further embodiment of the invention;
FIG. 1 1 depicts a display in accordance with another embodiment of the invention; FIG. 12 depicts a display in accordance with an embodiment of the invention;
DETAILED DESCRIPTION OF THE INVENTION
Figure 1 depicts a box diagram of a system according to an embodiment of the invention. The system 10 depicted includes a plurality of remote units 12a-12d associated with individual targets 15a-15d. The remote units 12 transmit data to a receiver 14 that provides the data to a processor 16, such as a PC server, that in turn provides the data to displays 18a, 18b, 18c. The system may also include one or more cameras 20a, 20b, such as television cameras, that provide real-time images of the particular event. The event may be, for example, an athletic competition wherein remote units are secured to the athletes and/or the athletic equipment (e.g., a ball). The cameras 20a, 20b may provide the real-time images directly to the displays 18a, 18b, 18c, and or may provide the real-time images to the processor 16 so that the processor can merge the real-time images with the desired biometric/other data for presentation on the displays 18a, 18b, 18c.
The processor 16 may be locally controlled, such as by a person in actual attendance at a sporting event providing input to the processor 16 through a keyboard 17 or other input device. The central processor 16 may actually consist of several different processors, with each processor accessing different data from the remote units 12.
Note that display 18a is representative of a set of one or more monitors that depict a specific set of data and images. In fact there could be multiple displays 18a showing the same data and images, such as is the case where a television network provides coverage of a sporting event to multiple home viewers. The same is true for displays 18b and 18c, with displays 18a showing data and images different from that of displays 18b or 18c.
The system may be configured to selectively provide different data to different
displays. For example, the biometric data can be provided to one display, with the acceleration data provided to a different display. Various approaches can be used to accomplish this. A system processor can be used that receives all the data from the remote units and selectively provides the different data to different displays. In another approach, the different data may be transmitted from the remote unit at different frequencies and/or in different formats (such as coded formats), so that a particular display is only provided with data from a select set of frequencies and/or formats. Using such approaches, the system can provide a first set of data, such as acceleration and/or speed data, to a first set of displays (such as a television network) that can be viewed by a first group of people (such as the viewing audience of an athletic competition), while the biometric data may be provided to a second set of one or more displays that can only be viewed by a second group of people, such as the coach, team physician, and/or other team personnel. The second group of people may be separate from, or be a subpart of, the first group.
The processor 16, which may actually be several processors, may be configured to analyze and/or modify the data received from the remote units 12. For example, the processor 16 may be configured to, using the acceleration data, calculate the resulting speeds and/or positions of the target, such as an athlete, in real time.
Fig. 2 depicts a system 10 according in accordance with an embodiment of the invention configured for use in an athletic competition, in this embodiment a football game, for broadcasting of real-time metrics from football players. The metrics can include biometric data and or movement-related information, such as accelerations, etc. The system 10 includes remote units 12 which can be worn by one or more players 15. The sensors of each remote unit 12 provide data on an associated football player 15.
Athletic equipment, such as a football, may also be equipped with a special remote unit positioned inside to provide movement data, and possibly video images, from the equipment. In the football example, the football's remote unit would preferably be configured with the football so as to ensure that the football has the same weight, balance, and other physical characteristics as a regular football, so that the football performs identically to a regular football.
The remote unit 12 worn by an athlete 15 may include an accelerometer capable of monitoring accelerations in 3 dimensions. The remote unit 12 may be positioned on the athlete's helmet, on a shoulder pad, directly on the athlete's body, etc. The remote unit is preferably small so that it can be worn without hampering the athlete's performance. The
remote unit can further include speed and or position sensors that provide data regarding the athlete's speed and/or position.
The remote unit 12 can further include biometric sensors, such as sensors monitoring heart rate, blood pressure, blood oxygen content, etc. The data from the remote unit is preferably provided in real time to a display. In various embodiments of the invention, the remote unit can be configured to transmit directly to the sideline, the fans, the internet, and/or a control room.
The raw data from the sensors may be processed by the remote unit 12 to form useful metrics, which are then transmitted to one or more system receivers 14. The transmitted metric data may be transmitted wirelessly and/or in the form of streams of digital packets. The system receiver 14 is preferably located within transmission range to receive and transmit data to and from all of the remote units. The system receiver 14 provides the received metric data to a processor 16, such as a PC server.
Cameras 20a-20e are positioned about the football field 22 to provide realtime video images. In the embodiment depicted, multiple cameras are provided to provide video from multiple perspectives. The video from the cameras is sent to a control facility 24, such as a mobile truck-mounted studio, from which a program director can view multiple video feeds from the various cameras 20a-20e on one or more video feed display screens 26a- 26c. The director can select the video feed desired for broadcast at a particular time. The selected video feed is virtually instantly broadcast as a video signal to television viewers via a broadcast system 28, such as a satellite system, a cable system, and/or direct broadcast.
The video signal may include overlaid data, such as scoring, statistics, advertisements, etc. Such overlaid data may be performed prior to transmission from the control facility, using known methods. With the current invention, the processor 16 can provide the director with additional overlaid information in the form of aesthetically useful and meaningful graphic displays and other visuals of selected metric data from the athletes and/or the athletic equipment, such as data on location, g-forces, pulse rates, etc of individual players. Such data may be presented to the director on one or more dedicated metric data displays 30a-30b. The director can thus select the desired metric data and display, and have that data converged in an overlay fashion with any of the video feeds received from the cameras 20a-20e. The resulting converged video is then transmitted to the viewing audience for viewing on their home televisions 32.
In the embodiment of Fig. 2, the receiver 14 and processor 16 are co-located at
the control facility 24. The receiver 14 and/or processor 16 could also function separately as third party equipment providing new enhanced content, including real-time metrics, to a broadcaster, re-broadcaster, or subscriber.
The selection of which metric data to overlay on the video seen by the end viewer may be made by the viewer. Such viewer selection of metric data could be part of an interactive broadcast.
One or more of the cameras could be controlled by an operator physically pointing each camera. One or more cameras could be controlled by remote control, such as by a person controlling the cameras from a control booth. Another method could involve using the metric data received from the remote units to control, completely or partially, the pointing and/or focus of one or more cameras. As depicted in the block diagram of Fig. 3a, the pointing and/or focus of the camera 20 could be controlled by the processor 16 based on metric data received from one or more of the remote units 12. For example, if it was desired to follow the moves of a particular player, the processor 16 could use the movement-related data from that player to anticipate and track the movements of the player 15. If a player 15 made a sudden tum, the metric data from that player's remote unit 12, which would received by the receiver 14 and provided to the processor 16, might indicate the acceleration or deceleration involved in the turn more rapidly than a human camera operator could detect the move. Moreover, even when an operator visually detected the movement, there is typically a delay in the pointing of the camera to follow the movement of the player. In the embodiment depicted in Fig. 3a, the processor 16 could be configured to use the movement-related data, such as acceleration, from the remote unit 12 to predict the movements of the player 15, and then control the camera 20 to follow the predicted path of the player 15. The processor's control of the camera 20 could be partial, with another operator, such as a human operator, having override capability.
The metric data could also be used to select which target, such as a particular player or piece of equipment, one or more particular cameras would track. For example, as depicted in the block diagram of Fig. 3b, if the metric data received from the remote units 12a-12c indicated that a particular player 15a had been subjected to a substantial g-force, such as may be received during a blocking maneuver [Joe — I need the proper football term here], the processor 16 might control one or more of the cameras 20 to point at and possibly zoom in on that particular player 15a. Similarly, if metric data received from a remote unit on a football (where the football was so equipped) indicated a sudden unexpected
acceleration deceleration of the football, such as may be associated with a fumble, the processor could control one or more cameras to point toward and possibly zoom in on the football.
Although the system depicted includes a single processor, multiple processors could in fact be used. Each processor might handle different tasks from the others. For example, one processor could be dedicated to camera control, while another could be dedicated to presentation of metric data, etc.
Fig. 4 depicts a block diagram of a sub-system 34 of system 10 for facilitating internet broadcast of real-time metrics from remote units 12 in accordance with a further embodiment of the invention. Subsystem 34 makes all or a portion of the metrics from each of the remote units 12 available for immediate access over the internet 36 by a home viewer sitting at a computer 38, a Internet-enhanced television 40, or other device capable of receiving and/or displaying data over the Internet. The displays of such devices may be configured to converge data in order to simultaneously display real-time metric data from one or more of the remote units, as well as real-time video from the cameras.
In the embodiment of Fig. 4, metrics are transmitted by each remote unit 12 in an Internet-compatible format, such as TCP/IP format, so that such data is immediately accessible to and from processor 16 to the Internet 36 using conventional routing equipment such as gateway 42 and routers 44a-c.
Where each remote unit acts as a stand-alone network device with an individual network address or URL (universal registration locator), each remote unit will function as its own web site with real-time data available over the internet. In such an embodiment, the processor 16 may serve as a web server, providing selected metric data from one or more of the remote units to home viewers over the Internet as web-page accessible content.
The processor 16 may itself function as its own web site, with its own individual network address or URL, providing compiled metrics on, for example, a dedicated web site. The processor 16 may also forward compiled metric data to the broadcast network's internet server (e.g., ABC, CBS, NBC, CNN). Each of the servers could then make the metrics available on its dedicated web site.
Enhanced content can also be made available to team members and staff, for example, the coach and/or team physician can view enhanced content made available on either on a dedicated system, such as a dedicated broadcast or internet-enable monitor, in
real-time or in archival format for later playback and review. A dedicated display could be provided that, based on various metrics such as player blood pressure and/or blood sugar levels, could be used in determining rotation of players, game plays, and other game planning.
Figure 5 depicts a block diagram of a remote unit 12 in accordance with an embodiment of the invention. The remote unit includes a power supply 46 (such as a battery), a biometric sensor 48, an accelerometer 50, a transceiver 52, a memory 54, and a data acquisition unit 56.
The remote unit 12 can be positioned on or in an individual target. For example, in a game such as football or hockey, individual players may each wear a remote unit. Such a remote unit may be positioned on or in the player's shoulder pad, helmet, knee pad, shoe, or anywhere on the athlete's body and/or apparel.
The biometric sensor 48 provides a biometric signal 49 on one or more biometric functions, and may actually consist of several different sensors collecting the various data. The biometric data collected by such sensors may include blood pressure, heart rate, perspiration rate, body temperature, and/or blood chemistry such as blood sugar levels.
The accelerometer 50 provides an acceleration signal 51 on accelerations in three dimensions. The accelerometer 50 may be a tri-axial accelerometer, such as a solid- state angular rate sensor sold by Systron Conner Parts under the name Gyro-Chip. Other accelerometers that may used include quartz flexure accelerometers and MEM accelerometers.
Data from the accelerometer can be used to determine acceleration, speed, location, and impact of a target, such as of an athlete wearing the remote unit. For example, where a remote unit is worn by a football player, the accelerometer on the remote unit can provide data that can be used to determine the impact experienced by the player in a collision with another player or object.
Note that other sensors are also within the scope of the invention. For example, a GPS receiver might be used in place of, or in addition to, the accelerometer 50, depending on the particular application.
The data acquisition unit 56 may include a microprocessor, such as a 8051- type core microprocessor. The processor may run a specific instruction set program engine that is stored therein or in the memory 54. The data acquisition unit 56 collects the raw data from the biometric sensor(s) and accelerometer. This raw data is internally processed by the
data acquisition unit 56 to generate appropriate metrics, which can be stored in the memory 54 and/or transmitted via the transceiver 52 to a remote location, such as the receiver 14 of Fig. 1. The memory 54 may be in various forms, and may provide short-term, long-term, and/or permanent storage of data.
The instruction set, memory requirements, processing speed, firmware, communications, power levels, and other hardware and software design parameters are typically all inter-related and are selected on the basis of a particular application.
If the remote unit of Fig. 5 were incorporated into a system such as that depicted in Fig. 1, the transceiver 52 of the remote unit 12 transmits the biometric data and acceleration data to a receiver 14. The data can be transmitted through various methods, including use of RF, infrared, laser, and other transmission methods. The data may, for example, be transmitted as associated streams of digital packets derived from the respective raw sensor data. The transmission link may be a wireless link employing conventional radio frequency (RF) wireless protocols, such as IEEE 802.11, at data rates that can exceed 9MB/sec. Depending on the requirements for a particular application, including any governing regulations, any number of different transmission protocols and modulation methods might be suitable to transmit data from the remote unit 12 to the receiver 14.
The transceiver 52 itself may have embedded TCP/IP code, which is the established network protocol, thus allowing each remote unit 12 to be its own web page or URL. Operationally, all data from the remote unit 12, which is the data acquired from the target, can be accessed in real-time over the Internet using the unique URL of the remote unit. Such a remote unit could communicate directly with a network server, so that the wearer of the remote unit, such as an athlete, is in effect an individual web site. The data can be archived in the memory 54 of the remote unit or in a memory located elsewhere, for later retrieval and/or processing. The sensors themselves may be configured to provide biometric and/or other data in Internet compatible format.
If the remote unit 12 of Fig. 5 is used as a part of a system 10 such as that depicted in Fig. 1 or Fig. 2, the receiver 14 can provide the data to the central processor 16, which may provide all the data to the display 18a. Alternatively, the central processor 16 may only provide limited data to each display. The central processor may provide different types of data to different displays. For example, the central processor 16 of Fig. 1 may provide a first set of data to a first display 18a, with a second set of data provided to a second display 18b, a third set of data provided to a third display 18c, etc. For example, in a football
game, the display 18a may correspond to a specific group of displays, namely the television sets of home viewers, which may depict accelerations and other data along with camera views of the game. The second display 18b may correspond to a monitor that is viewed only by the coach and/or team physician of one team, while the third display may be viewed only by the coach and or team physician of the other team. The second and third displays 18b, 18c may depict biometric data that can be of use in determining rotation of players, game plays, and other game planning.
One or more of the displays 18a, 18b, 18c can show television images of the athletic event as provided by the cameras 20a, 20b. The displays can also provide simultaneous displays of biometric and/or other data received from the remote units.
The remote unit may be equipped with one or more cameras that provide video images of the event from the athlete's perspective. For example, a remote unit mounted in or on a football player's helmet could provide continuous video images showing the football player's view of the event. The remote unit may also include a microphone or similar sensor for providing audio signals.
Due to size and weight constraints, especially where the remote unit 12 is to be worn by a person, it may be desirable to keep the remote unit as small as possible while still maintaining proper performance. The remote unit on the target may consist of an ASIC and or CMOS construction, and can be implemented in several integrated circuit, chip-size modular sections. Sensors, A/D convertor, processor, transceiver, and or antenna may all be located in a small (such as pager-size or smaller) package that transmits to one or more receivers. Anywhere from one to all of the elements of the remote unit may be integrally formed as part of a single integrated circuit chip. For example, an 8051 microprocessor could be built into an ASIC chip, with MEM sensors, a planar antenna, and a power source built into the substrate of the chip.
One of the more sizeable elements, depending on the power requirements, can be the power supply. In one embodiment of the invention, depicted in Fig. 6, the power supply 46 consists of a small battery 58, such as a lithium watch-type battery, in combination with a capacitor 60 and a solar cell 62. The capacitor 60 may be an ultracapacitor of the type sold by Maxwell Technologies, capable of collecting, storing, and releasing electrical energy for peak power requirements, such as may be required to power the transmitter of the remote unit. Thus, the capacitor 60 stores energy for rapid discharge of "cache" power for peak power delivery requirements, whereas the battery 58 provides general (i.e., lower) power
delivery. A solar cell 62 can charge both the battery 58 and the capacitor 60. The solar cell 62 may charge the capacitor 60 first, with overflow charging provided to the battery 58 after the capacitor 60 is charged. Use of the capacitor 60 for brief peak power requirements, in combination with a battery 58 for general power requirements, can significantly reduce the size of the power source 46 while still meeting peak power requirements.
In order to save power, the system may include a power-down sequence, for example, when the remote unit is not required to transmit information, or where it has lost contact with a system receiver/transmitter/controller, the remote unit may power down its own transmitter. The remote unit can continue to collect data, storing the data in the remote unit's memory. Use of such a "store and forward" power conserving method can substantially reduce the power requirements of the remote unit. The remote unit can also continue to receive data from the system transmitter. Because the sensor(s), memory, and receiver often use substantially less energy than a transmitter, the "store and forward" mode will typically draw much less energy than when the transmitter is operated. The system may power up the transmitter at regular intervals to transmit its data. The system may also power up the transmitter in response to a signal from the system transmitter, such as where a the signal from the system transmitter is received after a period of "dead-time," such as may occur when the remote unit is beyond the range of the system transmitter.
FIG. 7 is a flow chart depicting the operation of an embodiment of a remote unit. At step 64, sensors 48 and 50 collect raw data that is passed, at step 66, to the data acquisition unit 56. At step 68, the data acquisition unit digitizes the data and applies known heuristic and correlational algorithms to generate relevant biometric and/or movement related data in the form of streams of digital packets. These packets are appropriately formatted using selected protocols at step 70.
In one embodiment, the formatted data from the remote unit is tagged (step 72) with a unique Internet protocol address or DNS Domain Name Server that serves to specifically identify the remote unit 12 to the receiver 14 and processor 16, as well as to all the nodes coupled to the Internet. At step 74, the transceiver encodes outbound associated data packets and transmits them to the receiver 14 using known methods.
Fig. 8 is a flow chart describing various alternate operation of processor 16 for enabling real-time presentation of metrics in the form of one or more appropriate visuals and graphics superimposed over a live television broadcast as depicted by Figs. 9 and 10. A related real-time presentation of similar visuals and graphics in the form of a webcast
sportscast is depicted in Fig. 11 and accomplished substantially independently from the control facility 24.
At step 80, processor 16 decodes each set of inbound data packets from receiver 14 associated with the various remote units 12. At step 82, processor 16 manipulates the metrics to generate predetermined visuals/graphics as shown in Figs. 9-11, described further below in the particular embodiment of a football game sportscast. At step 84, the metric visuals/graphics are selectively superimposed over incoming live video feeds from one or more broadcast cameras 20.
At step 88, processor 16 transmits a first set of selected data to a first set of displays, such as a monitor that is seen by the coach or team physician. This first set of selected data may include biometric data the physical condition of players that the coach and/or physician may find useful, but that may not be desirable to publicly broadcast.
At step 90, the processor transmits converged video and metric visuals to the control facility 24 for selection by a program director responsible for the sportscast. In some applications, the video and metrics are not shown to the program director in converged form, but are instead presented separately. This allows the program director to pick and choose the metric visuals to incorporate into the various video feeds that may be available.
At step 90, the processor transmits and/or records selected metrics, alone or converged with video, for archival purposes. Such archived metrics and/or videos can be used for game replay and or for later playback.
Steps 92 to 98 describe possible alternate embodiments for using real-time metrics to further enhance the experience of a live sportscast by employing the Internet (i.e., webcasting of the game). At step 92, the processor digitizes a live video signal in Internet- compatible format. This permits the video to be broadcast over the Internet and viewed by a computer user using a computer-driven monitor or other display, which may involve the use of a streaming video software program such as Quicktime(TM). Appropriate icons or user- selectable pull-down menus can be superimposed over the digitally encoded sportcast to make it possible for the computer user to individually and remotely select among various real-time visuals/graphics of actual player/team metrics and to have these displayed in a non- obtrusive manner over the real-time webcast of the game.
Figure 9 depicts an embodiment of a display of the invention, including a monitor screen 100. The monitor 100 provides a television image 102 of the football game, which may be similar to standard television coverage of similar football games. In the
embodiment depicted, the monitor 100 includes in its lower right corner a graphical representation 104 of acceleration data from a remote unit attached to one of the players, namely "PLAYER 34" as indicated by reference 106. In the lower left corner is a graphical representation 108 of the player's heart rate. In such an embodiment, a viewer can see football plays on the monitor, with simultaneous viewing of the player's heart rate graphics 108 and experienced g-forces 104 during the football play. Although Fig. 9 depicts a graphical representation of the heart rate and acceleration data, other data, including combinations of data, could be presented. Moreover, presentations of data may be made in non-graphical form, such as in numeric form, or with a combination of numeric and graphical displays.
Although Fig. 9 depicts data from only one player, other embodiments could involve data from more players. Fig. 10 depicts a monitor screen 100 providing a television image 102 of the football game, except that Fig. 10 depicts the acceleration (i.e., shock) graphs for all players on each team. The data is provided on bar graphs 110 corresponding to each player's number. In the embodiment depicted, the graphic data is overlaid onto the television image 102 along either side of the screen, in such a way as to minimize any potential for distraction from visual images of the game.
A remote unit may be provided on selected non-players, such as the team coach. For example, the heart rate of the team coach might be shown to home viewers at particularly exciting parts of the game.
In a further embodiment of the invention, data from multiple players can be combined to provide to group data. For example, the accelerations of all players on a team could be combined into a single value over the course of the game as an indication of "total energy" of the team. The total energy could be determined as the energy expended at particular instant in time, or over a longer period, such as over a particular play or over the entire game. Fig. 11 shows a display 100 according to an embodiment of the invention, as may be seen by a home viewer during a live televised football sportscast. In the image shown, tachometer-like graphics 1 10, 112 of team energy, bar charts 114 of g-forces, sliding indicators 116 of player temperature, and other visuals and graphics 118 are overlaid on the live image 102 of the game to enhance the home viewer experience in a non-obtrusive, visually stimulating manner. Separate, converged, or overlaid video/data/audio graphics, such as item 120, may also be used to promote a variety of products for generating advertising revenues for the networks in conjunction with the various displayed visuals.
Fig. 12 shows a display similar to Fig. 11 but in the context of a live webcast of a football game. The monitor display 100 is a computer monitor. Selection buttons/controls 122, 124, which may be overlaid on the monitor display 100 or similar display as indicated, provide for customization of the webcast by enabling the viewer to select specific visuals to be displayed, if any, as well as enabling/disabling of available audio features.
Displays such as those depicted in Figs. 9-12 might be modified so that, during play that is broadcast live, only the visual images of the game are provided. In such an embodiment, the biometric and/or other data might be reserved for use only during instant replays.
Although the above discussion with respect to Figs. 9-12 was directed to use in a football game, such a system could be applied to a variety of other applications, including sporting events such as hockey, baseball, soccer, boxing, cross-country races, etc.
In boxing, the remote unit can include biometric sensors. Additionally, the remote unit can include one or more accelerometers and/or other movement sensors (e.g., speed, position, etc.). In one embodiment, the remote unit includes an accelerometer mounted in the boxer's glove. This accelerometer can provide real-time data regarding the acceleration/deceleration of the boxer's fists, so that the "g's" of a punch can be measured. The remote unit may be mounted in the wrist area of the glove. In another embodiment, the accelerometer can be mounted on the boxer's head to determine head accelerations and/or movements during a boxing match. The accelerometer may be sized to fit into a boxer's mouthpiece. In a further embodiment applicable in fights where the boxer's are wearing headgear (such as a padded helmet), the accelerometer can be mounted in the headgear.
As with football, the data in boxing can be selectively provided to different displays. A first display for public use (such as provided to a TV network) can provide simultaneous images of the fight along with representations of the acceleration data of the boxer's fists and/or heads. A second display may provide a second set of data, which may include the acceleration data as well as biometric data for selective display to the boxer's coach and/or physician. This second display may also include simultaneous images of the fight, so that the boxer's biometric and acceleration can be associated with particular events during the fight.
The invention could be used for cross-country activities, such as marathons, triathlons, biathlons, or cross-country bicycle races. In one embodiment for such activities, a
remote unit can be provided that provides data on biometric functions, position, acceleration, and/or velocity. For example, on a cross-country bicycle race, the remote unit might monitor heart rate, blood pressure, and blood sugar levels of an athlete, while also monitoring the speed and distances involved. This information could be provided to the athlete and/or to a athlete monitoring the competitor's progress.
In an embodiment of the invention, the remote unit includes a GPS receiver that determines the location of the athlete using GPS satellite data. The remote unit can transmit and/or store the location data, thus providing a record of the competitor's movements throughout the activity.
The invention can be used as a tracking device to confirm that a competitor has actually completed the event without deviating from the set course. For example, by positioning receivers at selected locations along a marathon's course, the receivers can receive transmissions from a remote unit as the runner wearing the particular remote unit passes that location. If the remote unit and/or system is equipped to determine locations of the remote units, such as be inclusion of a GPS receiver in the remote unit, the position data can be used to confirm that the competitor followed the set course.
The invention has application not just to athletes but also for use by any individuals, such as patient under a doctor's care, who require monitoring of various conditions and/or movements. For example, a person can wear a remote unit configured to monitor biometric performance data, such as blood pressure, body temperature, and ECG. The remote unit may store the biometric data in a memory for downloading to a system receiver at a later time, such as at the end of a day. The remote unit may alternatively provide a continuous stream of data in real-time to the system receiver. The system receiver can provide the data to a personal computer, television, or other system for displaying and/or recording of the data. Thus, the user or another individual can retrieve the data acquired by the remote unit over a period of time. The data could be used, possibly along with data from other individuals, for trend analysis and chrono-biological studies. The data can be made available on a web site. Such an application could also be useful for prenatal and infant monitoring.