WO2002058029A2 - Optical position determination on any surface - Google Patents

Optical position determination on any surface Download PDF

Info

Publication number
WO2002058029A2
WO2002058029A2 PCT/US2001/044854 US0144854W WO02058029A2 WO 2002058029 A2 WO2002058029 A2 WO 2002058029A2 US 0144854 W US0144854 W US 0144854W WO 02058029 A2 WO02058029 A2 WO 02058029A2
Authority
WO
WIPO (PCT)
Prior art keywords
related data
writing
points
position related
movable element
Prior art date
Application number
PCT/US2001/044854
Other languages
French (fr)
Other versions
WO2002058029A3 (en
Inventor
Oral F. Sekendur
Original Assignee
Sekendur Oral F
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US09/725,301 external-priority patent/US20020118181A1/en
Application filed by Sekendur Oral F filed Critical Sekendur Oral F
Priority to AU2001298118A priority Critical patent/AU2001298118A1/en
Publication of WO2002058029A2 publication Critical patent/WO2002058029A2/en
Publication of WO2002058029A3 publication Critical patent/WO2002058029A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G06F3/0325—Detection arrangements using opto-electronic means using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G06F3/0317—Detection arrangements using opto-electronic means in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03545—Pens or stylus

Definitions

  • This invention relates to apparatus and method for generating position related computer data by obtaining and outputting the instantaneous position and/or movement of a moveable element on a surface, such as might be used for determining the position and/or movement of a pen/pencil on paper.
  • this invention will allow the user to input graphical information (e.g., drawings or handwriting) and simultaneously provide an original hard copy of the information.
  • the use of a writing tablet and a stylus is common for inputting hand written data. Most two- dimensional devices require contact between the writing tablet and stylus. Three-dimensional devices usually do not require contact. They normally use a form of wave energy such as light, electromagnetic, or sonic energy. Generally, two relationships exist between the stylus and the writing tablet.
  • the passive stylus/active tablet utilizes a passive stylus interfacing with an active receiving surface (e.g., resistive and capacitive methods), while the active stylus/passive tablet utilizes an active stylus interfacing with a passive surface (e.g., optical, acoustic, tactile, or electromagnetic).
  • a third method using a mechanical linkage such as a pantograph is rarely used.
  • the passive stylus/active surface method has some significant shortcomings. The most significant is the active surface or tablet itself. Besides being complex, large, heavy, cumbersome, and difficult to transport, the tablet is expensive. Further, the tablet cannot usually distinguish between the stylus and another object pressing on it. Still further, active tablets are difficult to manufacture, subject to interference from outside factors, and have complex mechanical and electrical parts that are subject to malfunction.
  • the active stylus/passive surface method also has major drawbacks. Most significantly, this method generally requires an awkward tablet besides a separate transmitter and receiver (usually in different locations). Further, the transmitted signal can become obscured before reaching the receiver.
  • Another class of active stylus/passive surface devices provides relative position information.
  • An example is the computer mouse that includes the mechanical mouse comprising a ball rolling on a surface, and the optical mouse comprising a surface with grid lines and an optical input means within the mouse.
  • active stylus/passive surface methods comprise a form of transducer, gyroscope and/or accelerometer located in the stylus itself.
  • Both the passive stylus/active surface and active stylus/passive surface methods have the feeling of being unnatural and require a significant interface adjustment for the user from the conventional pen/pencil and paper.
  • the amount and accuracy of information provided by these methods are limited.
  • some of these methods require a physical connection between the stylus and the tablet. All the methods provide two-dimensional information. Some provide three-dimensional information. Further, they may provide one or more, but not all the following information: displacement, rotation, angle to tablet, and velocity. None provide all of this information.
  • the aforementioned methods can provide a printed hard copy, but they do not provide an original hard copy. Since the present invention teaches obtaining coordinate information by scanning a surface and simultaneously placing information on the surface by writing on the surface with a stylus, an original hard copy is produced by writing or drawing on the surface.
  • a significant advantage of the present invention is its interface. Overall, no matter how good a computer interface is, less of it would be better.
  • the present invention allows for an interface that is almost identical to that of a pen/pencil and paper.
  • the present invention is used in the same manner as a pen/pencil and paper and all of the computing is done in the background unnoticed by the user.
  • the pen/pencil and paper are familiar and comfortable interfaces to the user.
  • OCR optical character recognition
  • Fig. 1 is a perspective view of an embodiment using a pen, paper, CCD camera, and a computer.
  • microcomputer 14 display 15 function keys
  • the present invention aims to overcome the aforementioned disadvantages and to provide a system that most closely emulates the use of pen/pencil and paper.
  • the present invention proposes the use of a writing surface such as paper and a stylus.
  • the stylus comprises an input means such as a charge-coupled device (CCD), a microcomputer, memory, power supply, and a communications device, whereby the CCD camera scans the writing surface.
  • An output signal from the CCD or array of light sensitive elements is sent to a computer or processor and finally output to the user.
  • the output can be in various forms including an image on a computer display or a computer printout.
  • the first is triangulation and the second is pattern matching.
  • triangulation may include specific methods such as optical techniques of grid and moire triangulation.
  • other mathematical techniques such as interpolation, extrapolation, smoothing, and other compensating techniques may be appropriately used.
  • Pattern matching may include sliding window correlation, windowing, pattern recognition, and partial imaging. For the purposes of this description, these various methods are incorporated.
  • Pattern matching is used to determine the instantaneous position and movement of the writing element relative to a pattern on a surface. While writing on a surface, the marks formed are instantaneously input by acquiring successive images of the writing surface as the writing element moves across the surface. Images are compiled into a picture of the writing surface scanned by the CCD camera. As writing is being placed on the surface, successive images are acquired and compiled. Earlier images are matched to successive images until the entire text on the writing surface is compiled into a larger picture.
  • the images acquired can be existing images on the surface or can be the actual writing as it is being formed.
  • acquired images should overlap to the degree that consecutive images can be referenced back to previous images to build a larger picture of the writing surface.
  • the instantaneous position and movement of the stylus relative to two fixed reference points is determined.
  • the CCD camera automatically detects at least two fixed reference points, such as the corners of the writing surface, and triangulates them with a writing element on the stylus to determine the instantaneous position of the writing element.
  • the user using the writing element on the stylus can apply to the writing surface the reference points visible to the CCD camera.
  • at least two fixed reference points can be pre-applied to the writing surface.
  • the reference points are dynamic, in that, the CCD camera will select two fixed reference points based on a set of pre-determined criteria.
  • the two fixed reference points need only to be fixed instantaneously since the CCD camera is able to dynamically and instantaneously select a different set of two fixed reference points.
  • a first embodiment comprises a writing surface 1 in Fig. 1, a pen 2, a CCD camera 3, and a wireless interface 4.
  • the CCD camera detects two fixed reference points based on a set of pre-determined criteria such as corners 5 a and 5b of the writing surface and triangulates the instantaneous position of the writing element 6 of the pen.
  • the two fixed reference points are dynamic, in that, the CCD camera will detect any two points which may include the corners of the writing surface or any other points in its field of vision.
  • the other points may include pre-applied marks on the surface or marks instantly applied using the writing element.
  • the CCD camera may dynamically change the two fixed reference points it selects based on a set of pre-determined criteria.
  • the CCD camera is mounted on the pen using an adapter 7 that fits securely on the upper end of the pen. This position allows the CCD camera to view the writing surface, the writing element, and the two fixed reference points. Should the reference points become obscured from the field of view, the CCD camera will select other points of reference.
  • a universal joint 8 allows maximum mobility for the CCD camera.
  • position related information is sent by a wireless interface to a computer 9 for processing.
  • a clip board 10 can be provided comprising the two fixed reference points, such as the left clip board reference point 11a and the right clip board reference point 1 lb. Still further, the two fixed reference points can be provided external to the writing surface or the clip board, such as the left external reference point 12a and the right external reference point 12b.
  • a microprocessor 13 and a display 14 can be provided. Function keys 15 and an illumination means 16 can also be provided.
  • At least two points of coordinate-related information can be made to selectively reflect at least one selected frequency of light, and the detector means can be made to selectively detect at least one selected frequency of light.
  • the two fixed reference points are detected relative to the writing element, input, and analyzed. Thus, the instantaneous position and movement of the writing element is determined.
  • a second embodiment uses both macro triangulation and micro triangulation techniques to determine the instantaneous position and movement of the stylus relative to the writing.
  • Macro triangulation detects larger areas to triangulate such as the corners of the writing surface and the writing element.
  • Micro triangulation detects smaller areas to triangulate such as points of the written text on the writing surface. The general position is determined by macro triangulation and the exact position is determined by micro triangulation.
  • a third embodiment uses both triangulation and pattern matching techniques to determine the instantaneous position and movement of the stylus relative to the writing.
  • Triangulation determines the instantaneous position and movement of the writing element.
  • Pattern matching inputs the written text by acquiring successive images of the written surface as the writing element moves across the surface and compiles the images into a picture of the writing surface scanned by the writing element. The two techniques in conjunction give extremely accurate and high-resolution data regarding the instantaneous position and movement.
  • the instantaneous written text may not be immediately visible to the camera because the writer's fingers may be blocking the camera's view.
  • triangulation determines the instantaneous position and movement of the stylus. As the stylus moves across the writing surface, the formerly blocked text will eventually come into view. At this point, this image is matched with previous images to form the overall picture of the writing surface.
  • the above embodiments use a single camera mounted on the upper end of the stylus for a greater perspective of the writing surface.
  • Other embodiments are possible comprising a second camera mounted on the lower end of the stylus.
  • a forth embodiment in Fig. 2 utilizes a second camera mounted on the lower end to the stylus facing the same direction as the upper camera.
  • the upper camera provides macro triangulation and the lower camera provides micro triangulation.
  • the viewing area of the cameras in represented by 17 in Fig. 2.
  • a fifth embodiment in Fig. 3 comprises the upper and lower cameras mounted in opposite directions, whereby the upper camera uses macro triangulation and pattern matching and the lower camera uses micro triangulation and pattern matching.
  • the viewing area of the cameras in represented by 17 in Fig. 3. The two techniques combined provide high-resolution data regarding the movement of the stylus over the writing surface.
  • a sixth embodiment uses an upper camera and a lower sensor facing the opposite direction.
  • the lower sensor can be in the form of a track ball, pressure sensor, transducer, gyroscope, and/or accelerometer located in the stylus itself, whereby the upper camera uses macro triangulation, micro triangulation, and pattern matching, while the sensor provides movement information.
  • the stylus is incorporated into a cell phone 18 in Fig. 7 to form a PencellTM in the shape of a stylus comprising a mouth piece 19, ear piece 20, display 21, function keys 22 and 23, pen 24, pen retractor switch 25, key pad 26, CCD camera 3, micro computer 27, memory 28, power supply 29, writing element 30, and communications device/cell phone 18.
  • the display is on one side of the PencellTM as in Fig. 7 and the keypad is on the other side as in Fig. 8.
  • the pen is retractable inside the cell phone to hide the writing element when not in use.
  • the pen is extended only slightly when the writing element is closer to the user as in Fig. 7. When the writing element is further away from the user, the pen is extended further as in Fig. 8.
  • the retractor switch is used to move the pen.
  • the pen When used as a cell phone as in Fig. 9, the pen doubles as an antenna.
  • the PencellTM is flipped so that the pen is facing up, and the pen extended to act as an antenna. This puts the earpiece and mouthpiece in position for use.
  • the positions of earpiece and mouthpiece can be reversed so that the pen is extended downward to act as an antenna. In this case, the PencellTM does not have to be flipped.
  • the pen can be completely removed to write on the display as in Fig. 10.
  • the pen will have two writing ends, one comprising the writing element 30 and the other comprising the writing tip 31.
  • the writing tip is designed to write on the display while the writing element is intended to write on a writing surface.
  • the writing tip fits into the PencellTM first to allow the writing element to be extended for writing on a writing surface.
  • pattern matching is used to determine the instantaneous position and movement of the writing element relative to a pattern on a surface.
  • the marks 32 formed are instantaneously input by acquiring successive images 33 of the writing surface as the writing element moves across the surface. Images are compiled into a picture of the writing surface scanned by the CCD camera. As writing is being placed on the surface, successive images are acquired and compiled. Earlier images are matched to successive images until the entire text on the writing surface is compiled into a larger picture.
  • the images acquired can be existing images on a surface or can be the actual marks as they are being formed.
  • acquired images should overlap to the degree that consecutive images can be referenced back to previous images to build a larger picture of the writing surface.
  • FIG. 4-6 show cameras mounted in different locations on the stylus.
  • the viewing area of the cameras in represented by 17 in Figs. 4-6.
  • Other techniques could be used to determine the position of the writing element. For example, mirrors mounted at angles to detect the position of the writing element using triangulation techniques can be used. Additionally, two cameras can be used to capture different views of the writing surface and the frames of the two cameras can be calibrated and compared with each other to determine movement and position of the writing element. Other optical techniques are grid and moire triangulation.

Abstract

The present invention proposes the use of at least three points of position-related information for a indicating X-Y coordinates. The stylus or pen 2 in Fig. 1 comprising a writing element 6 and one of the three points of position-related information has a light source 16 illuminating a surface. A detector 3 comprising a charge-coupled device (CCD) located on the stylus detects the position-related information. The position-related information from the CCD is sent to a computer for processing and finally the desired information output to the user. During the process of writng, a position-related information is detected and sent to the computer analysis, and finally output to the user. The position of the writing element is determined using the principles of triangulation. Through the use of handwriting recognition software, the output can be converted to a "keyboard-typed" representation of the handwritten text.

Description

Patent Application of
Oral F. Se endur for
OPTICAL POSITION DETERMINATION
ON ANY SURFACE
Continuation in Part of
ABSOLUTE OPTICAL POSITION DETERMINATION
Appl . NO . 09/725, 301 AND
OPTICAL POSITION DETERMINATION
ON PLAIN PAPER
Appl. NO. 09/725,883
Field of the Invention
This invention relates to apparatus and method for generating position related computer data by obtaining and outputting the instantaneous position and/or movement of a moveable element on a surface, such as might be used for determining the position and/or movement of a pen/pencil on paper. In addition, this invention will allow the user to input graphical information (e.g., drawings or handwriting) and simultaneously provide an original hard copy of the information. Background-Description of Prior Art
The following United State Patents are believed to be most closely related to the present invention:
U.S. Pat. Nos.:
6,100,538 6,008,800 6,005,548 5,982,352 5,953,000
5,936,615 5,729,251 5,689,619 5,525,764 5,477,012
5.442.147 5,086,197 . 5,075,558 5,075,541 5,051,736
5,009,277 4,975,546 4,885,433 4,853,496 4,845,684
4,809,351 4,806,707 4,804,949 4,751,741 4,532,376
4,364,035 4,141,073
Many attempts have been made to determine the position of an object on a data surface in the form of computer data. Both two-dimensional and three-dimensional position determining devices now exist for inputting graphical data such as handwritten text, symbols, drawings, and so on. These devices determine the absolute position and/or movement of a stylus on a data surface by converting the position information into coordinates.
The use of a writing tablet and a stylus is common for inputting hand written data. Most two- dimensional devices require contact between the writing tablet and stylus. Three-dimensional devices usually do not require contact. They normally use a form of wave energy such as light, electromagnetic, or sonic energy. Generally, two relationships exist between the stylus and the writing tablet. The passive stylus/active tablet utilizes a passive stylus interfacing with an active receiving surface (e.g., resistive and capacitive methods), while the active stylus/passive tablet utilizes an active stylus interfacing with a passive surface (e.g., optical, acoustic, tactile, or electromagnetic). A third method using a mechanical linkage such as a pantograph is rarely used.
The passive stylus/active surface method has some significant shortcomings. The most significant is the active surface or tablet itself. Besides being complex, large, heavy, cumbersome, and difficult to transport, the tablet is expensive. Further, the tablet cannot usually distinguish between the stylus and another object pressing on it. Still further, active tablets are difficult to manufacture, subject to interference from outside factors, and have complex mechanical and electrical parts that are subject to malfunction.
The active stylus/passive surface method also has major drawbacks. Most significantly, this method generally requires an awkward tablet besides a separate transmitter and receiver (usually in different locations). Further, the transmitted signal can become obscured before reaching the receiver.
Another class of active stylus/passive surface devices provides relative position information. An example is the computer mouse that includes the mechanical mouse comprising a ball rolling on a surface, and the optical mouse comprising a surface with grid lines and an optical input means within the mouse.
Additionally* active stylus/passive surface methods comprise a form of transducer, gyroscope and/or accelerometer located in the stylus itself.
Both the passive stylus/active surface and active stylus/passive surface methods have the feeling of being unnatural and require a significant interface adjustment for the user from the conventional pen/pencil and paper. The amount and accuracy of information provided by these methods are limited. In addition, some of these methods require a physical connection between the stylus and the tablet. All the methods provide two-dimensional information. Some provide three-dimensional information. Further, they may provide one or more, but not all the following information: displacement, rotation, angle to tablet, and velocity. None provide all of this information.
The aforementioned methods can provide a printed hard copy, but they do not provide an original hard copy. Since the present invention teaches obtaining coordinate information by scanning a surface and simultaneously placing information on the surface by writing on the surface with a stylus, an original hard copy is produced by writing or drawing on the surface.
A significant advantage of the present invention is its interface. Overall, no matter how good a computer interface is, less of it would be better. The present invention allows for an interface that is almost identical to that of a pen/pencil and paper. The present invention is used in the same manner as a pen/pencil and paper and all of the computing is done in the background unnoticed by the user. The pen/pencil and paper are familiar and comfortable interfaces to the user.
This inventor has patented a device (U.S. Patent No. 5,477,012) comprising a passive coded surface and an active stylus comprising a video camera. The drawback to this system is the requirement of the passive coded surface. Availability of the passive coded surface and the ability of the active stylus to accurately read the passive coded surface are only some of the disadvantages of the patented device. The present invention overcomes these drawbacks by eliminating the need for the passive coded surface. Now, any surface can be used. For example, stylus and paper in the present invention will most closely simulate the familiar use of ordinary pen/pencil and paper.
Objects and Advantages
It is an object of the present invention to provide all of the aforementioned information.
It is an object of the present invention to overcome all of the aforementioned disadvantages.
It is an object of the present invention to provide an apparatus and method for obtaining and outputting the absolute position and/or movement of a moveable element on a surface. It is an object of the present invention to provide an apparatus and method for obtaining and outputting the absolute position and/or movement of a moveable element on a surface for acquisition and output of hand written data.
It is an object of the present invention to provide a system that most closely resembles using pen/pencil and paper.
It is an object of the present invention to provide an original hard copy of data as part of the input process of writing on a surface.
It is an object of the present invention to provide an apparatus and method of the character described in which the absolute position and/or movement of the movable element could be precisely determined relative to at least one fixed reference.
It is an object of the present invention to provide an apparatus and method of the character described in which the absolute position and/or movement of the movable element can be precisely determined relative to at least one fixed reference where the at least one fixed reference is automatically detected.
It is an object of the present invention to provide an apparatus for hand held use.
It is an object of the present invention to provide the aforementioned movable element in the shape of a stylus.
It is an object of the present invention to provide an apparatus of the character described which does not require the use of a special digitizing tablet.
It is an object of the present invention to provide an apparatus of the character described which does not require the use of a special surface.
It is an object of the present invention to provide an apparatus of the character described which does not require the use of a special transmitter.
It is an object of the present invention to provide an apparatus of the character described which could use a writing surface such as ordinary paper.
It is an object of the present invention to provide an apparatus and method for obtaining and outputting the position and/or movement of a moveable element on a surface comprising the surface, a detector means, a data processing means and a data output means.
It is an object of the present invention to provide an apparatus and method for precisely locating the absolute position and/or movement of a movable element within a plane. More particularly, it is an object of the invention to provide an input/output apparatus for use with a computer that includes a movable element, whose absolute position and/or movement within a plane can be determined with or without a physical connection between the movable element and the plane.
It is an object of the present invention to provide an apparatus and method for handwriting recognition.
It is an object of the present invention to provide an apparatus and method for optical character recognition (OCR).
It is an object of the present invention to provide an apparatus and method for signature verification.
It is an object of the present invention to provide an apparatus and method for handwriting verification.
It is an object of the present invention to provide an apparatus and method for graphical recognition.
It is an object of the present invention to provide an apparatus and method for graphical input.
It is an object of the present invention to provide an apparatus and method for forms processing. It is an object of the present invention to provide an apparatus and method for converting optically input data into coordinate data.
It is an object of the present invention to provide an apparatus and method for applying position related information to a surface.
It is an object of the present invention to provide an apparatus and method for applying position related information to a surface with by writing on it while scanning, then using the written information for position determination.
It is an object of the present invention to provide an apparatus and method for applying position related information to a surface by writing the surface while scanning, then using the written data as points of reference.
It is an object of the present invention to provide an apparatus and method for precisely locating the absolute position and/or movement of a movable element within a plane using well-known techniques of interpolation, extrapolation, and triangulation.
It is an object of the present invention to provide an apparatus and method for precisely locating the absolute position and/or movement of a movable element within a plane using well known techniques of pattern matching, sliding window correlation, and windowing.
It is an object of the present invention to provide an apparatus and method for precisely locating the absolute position and/or movement of a movable element within a plane using well-known techniques of pattern recognition and partial imaging.
It is an object of the present invention to provide an apparatus and method for providing analog data.
It is an object of the present invention to provide an apparatus and method for providing digital data. It is an object of the present invention to provide an apparatus and method for digitizing optical data.
It is an object of the present invention to provide an apparatus and method for learning a surface.
It is an object of the present invention to provide a writing surface made of a material selected from the group consisting of paper, plastic, glass, metal, synthetic fiber, synthetic material, natural material, and a paper like substance.
Description of Drawings
Fig. 1 is a perspective view of an embodiment using a pen, paper, CCD camera, and a computer.
Reference Numerals in Drawings
writing surface pen
CCD camera wireless interface
5a left corner 5b right corner
writing means adapter
8 universal joint computer
10 clipboard 11a left clipboard reference point
l ib right clipboard reference point 12a left external reference point
12b right external reference point 13 microcomputer 14 display 15 function keys
16 illumination 17 field of vision/viewing area
Description of Preferred Embodiments
The present invention aims to overcome the aforementioned disadvantages and to provide a system that most closely emulates the use of pen/pencil and paper.
Accordingly, the present invention proposes the use of a writing surface such as paper and a stylus. The stylus comprises an input means such as a charge-coupled device (CCD), a microcomputer, memory, power supply, and a communications device, whereby the CCD camera scans the writing surface. An output signal from the CCD or array of light sensitive elements is sent to a computer or processor and finally output to the user. The output can be in various forms including an image on a computer display or a computer printout.
Two methods are used independently or together to determine position. The first is triangulation and the second is pattern matching. These terms are used in their broadest sense and include methods that are more specific. For example, triangulation may include specific methods such as optical techniques of grid and moire triangulation. In some cases, other mathematical techniques such as interpolation, extrapolation, smoothing, and other compensating techniques may be appropriately used. Pattern matching may include sliding window correlation, windowing, pattern recognition, and partial imaging. For the purposes of this description, these various methods are incorporated.
Pattern matching is used to determine the instantaneous position and movement of the writing element relative to a pattern on a surface. While writing on a surface, the marks formed are instantaneously input by acquiring successive images of the writing surface as the writing element moves across the surface. Images are compiled into a picture of the writing surface scanned by the CCD camera. As writing is being placed on the surface, successive images are acquired and compiled. Earlier images are matched to successive images until the entire text on the writing surface is compiled into a larger picture.
The images acquired can be existing images on the surface or can be the actual writing as it is being formed. When using pattern matching without triangulation, acquired images should overlap to the degree that consecutive images can be referenced back to previous images to build a larger picture of the writing surface.
On the other hand, using the principal of triangulation, the instantaneous position and movement of the stylus relative to two fixed reference points is determined. The CCD camera automatically detects at least two fixed reference points, such as the corners of the writing surface, and triangulates them with a writing element on the stylus to determine the instantaneous position of the writing element. Alternatively, the user, using the writing element on the stylus can apply to the writing surface the reference points visible to the CCD camera. Still further, at least two fixed reference points can be pre-applied to the writing surface.
The reference points are dynamic, in that, the CCD camera will select two fixed reference points based on a set of pre-determined criteria. The two fixed reference points need only to be fixed instantaneously since the CCD camera is able to dynamically and instantaneously select a different set of two fixed reference points.
A first embodiment comprises a writing surface 1 in Fig. 1, a pen 2, a CCD camera 3, and a wireless interface 4. The CCD camera detects two fixed reference points based on a set of pre-determined criteria such as corners 5 a and 5b of the writing surface and triangulates the instantaneous position of the writing element 6 of the pen. The two fixed reference points are dynamic, in that, the CCD camera will detect any two points which may include the corners of the writing surface or any other points in its field of vision. The other points may include pre-applied marks on the surface or marks instantly applied using the writing element. The CCD camera may dynamically change the two fixed reference points it selects based on a set of pre-determined criteria.
The CCD camera is mounted on the pen using an adapter 7 that fits securely on the upper end of the pen. This position allows the CCD camera to view the writing surface, the writing element, and the two fixed reference points. Should the reference points become obscured from the field of view, the CCD camera will select other points of reference. A universal joint 8 allows maximum mobility for the CCD camera. Finally, position related information is sent by a wireless interface to a computer 9 for processing.
Further, a clip board 10 can be provided comprising the two fixed reference points, such as the left clip board reference point 11a and the right clip board reference point 1 lb. Still further, the two fixed reference points can be provided external to the writing surface or the clip board, such as the left external reference point 12a and the right external reference point 12b. A microprocessor 13 and a display 14 can be provided. Function keys 15 and an illumination means 16 can also be provided.
At least two points of coordinate-related information can be made to selectively reflect at least one selected frequency of light, and the detector means can be made to selectively detect at least one selected frequency of light.
While writing on the writing surface, the two fixed reference points are detected relative to the writing element, input, and analyzed. Thus, the instantaneous position and movement of the writing element is determined.
For greater resolution, a second embodiment uses both macro triangulation and micro triangulation techniques to determine the instantaneous position and movement of the stylus relative to the writing. Macro triangulation detects larger areas to triangulate such as the corners of the writing surface and the writing element. Micro triangulation detects smaller areas to triangulate such as points of the written text on the writing surface. The general position is determined by macro triangulation and the exact position is determined by micro triangulation.
A third embodiment uses both triangulation and pattern matching techniques to determine the instantaneous position and movement of the stylus relative to the writing. Triangulation determines the instantaneous position and movement of the writing element. Pattern matching inputs the written text by acquiring successive images of the written surface as the writing element moves across the surface and compiles the images into a picture of the writing surface scanned by the writing element. The two techniques in conjunction give extremely accurate and high-resolution data regarding the instantaneous position and movement.
With the camera placed on the upper end of the stylus, the instantaneous written text may not be immediately visible to the camera because the writer's fingers may be blocking the camera's view. In this case, triangulation determines the instantaneous position and movement of the stylus. As the stylus moves across the writing surface, the formerly blocked text will eventually come into view. At this point, this image is matched with previous images to form the overall picture of the writing surface.
In addition to a composite static image of the scanned writing surface, other information such velocity, acceleration, rotation, and attitude of the stylus can be obtained.
Thus far, the above embodiments use a single camera mounted on the upper end of the stylus for a greater perspective of the writing surface. Other embodiments are possible comprising a second camera mounted on the lower end of the stylus.
A forth embodiment in Fig. 2 utilizes a second camera mounted on the lower end to the stylus facing the same direction as the upper camera. The upper camera provides macro triangulation and the lower camera provides micro triangulation. The viewing area of the cameras in represented by 17 in Fig. 2.
A fifth embodiment in Fig. 3 comprises the upper and lower cameras mounted in opposite directions, whereby the upper camera uses macro triangulation and pattern matching and the lower camera uses micro triangulation and pattern matching. The viewing area of the cameras in represented by 17 in Fig. 3. The two techniques combined provide high-resolution data regarding the movement of the stylus over the writing surface.
A sixth embodiment uses an upper camera and a lower sensor facing the opposite direction. The lower sensor can be in the form of a track ball, pressure sensor, transducer, gyroscope, and/or accelerometer located in the stylus itself, whereby the upper camera uses macro triangulation, micro triangulation, and pattern matching, while the sensor provides movement information.
In a seventh embodiment, the stylus is incorporated into a cell phone 18 in Fig. 7 to form a Pencell™ in the shape of a stylus comprising a mouth piece 19, ear piece 20, display 21, function keys 22 and 23, pen 24, pen retractor switch 25, key pad 26, CCD camera 3, micro computer 27, memory 28, power supply 29, writing element 30, and communications device/cell phone 18. The display is on one side of the Pencell™ as in Fig. 7 and the keypad is on the other side as in Fig. 8.
The pen is retractable inside the cell phone to hide the writing element when not in use. The pen is extended only slightly when the writing element is closer to the user as in Fig. 7. When the writing element is further away from the user, the pen is extended further as in Fig. 8. The retractor switch is used to move the pen.
When used as a cell phone as in Fig. 9, the pen doubles as an antenna. The Pencell™ is flipped so that the pen is facing up, and the pen extended to act as an antenna. This puts the earpiece and mouthpiece in position for use. Of course, in another embodiment not shown, the positions of earpiece and mouthpiece can be reversed so that the pen is extended downward to act as an antenna. In this case, the Pencell™ does not have to be flipped.
Still further, the pen can be completely removed to write on the display as in Fig. 10. In this case, the pen will have two writing ends, one comprising the writing element 30 and the other comprising the writing tip 31. The writing tip is designed to write on the display while the writing element is intended to write on a writing surface. The writing tip fits into the Pencell™ first to allow the writing element to be extended for writing on a writing surface.
In an eighth embodiment as in Fig. 11, pattern matching is used to determine the instantaneous position and movement of the writing element relative to a pattern on a surface. While writing on a surface, the marks 32 formed are instantaneously input by acquiring successive images 33 of the writing surface as the writing element moves across the surface. Images are compiled into a picture of the writing surface scanned by the CCD camera. As writing is being placed on the surface, successive images are acquired and compiled. Earlier images are matched to successive images until the entire text on the writing surface is compiled into a larger picture.
The images acquired can be existing images on a surface or can be the actual marks as they are being formed. When using pattern matching without triangulation, acquired images should overlap to the degree that consecutive images can be referenced back to previous images to build a larger picture of the writing surface.
Other embodiments represented by Figs. 4-6 show cameras mounted in different locations on the stylus. The viewing area of the cameras in represented by 17 in Figs. 4-6.
Further embodiments include an overlay such as carbon paper, film template or plate for overlaying the surface with position related data, a pressure switch for turning on the CCD camera when pressure is applied to the writing element and a focusing method for focusing the surface.
Other techniques could be used to determine the position of the writing element. For example, mirrors mounted at angles to detect the position of the writing element using triangulation techniques can be used. Additionally, two cameras can be used to capture different views of the writing surface and the frames of the two cameras can be calibrated and compared with each other to determine movement and position of the writing element. Other optical techniques are grid and moire triangulation.
The above-described embodiments are simply illustrative of the principles of the invention. Various other modifications and changes may be made by those skilled in the art which will embody the principles of the invention and fall within the spirit and scope thereof.

Claims

I Claim:
1. An apparatus for obtaining and outputting the position and movement of a moveable element on a surface comprising:
o at least three points of position related data;
o a data input means for obtaining said at least three points of position related data;
o a data output means for outputting said at least three points of position related data;
o a data processing means for obtaining and analyzing said at least three points of position related data from said data output means;
o said at least three points of position related data comprising at least two fixed points of position related data and at least one movable point of position related data;
o whereby said movable element comprises said at least one movable point of position related data;
o whereby said at least three points of position related data are triangulated to obtain the position and movement of said moveable element.
2. An apparatus according to Claim 1 wherein:
o said surface comprises a writing surface;
o said movable element comprises a writing means for writing on said surface.
3. An apparatus according to claim 1 wherein:
o said surface comprises a substantially two dimensional planar face and further comprises said at least two fixed points of position-related data;
o said data input means is a detector means comprising an array of light sensitive elements for detecting said at least three points of position related data and an output means for generating at least one output signal thereof;
o said processing mean's comprises means for receiving and processing said at least one output signal form said detector means, thereby to determine the position and movement of said movable element relative to said surface.
4. An apparatus according to Claim 3 wherein:
o said moveable element comprises said detector means, whereby said movable element is movable relative to said surface;
o means for determining the path of said moveable element are provided by detecting a sequence of said position-related data.
o means for analyzing the path of said moveable element are provided.
5. An apparatus according to Claim 1 selected from the group consisting of:
said at least three points of position-related data comprising position-related data for a plurality of points out at least one point on said surface;
said apparatus whereby said at least two fixed points of position related data are printed on said surface; said apparatus whereby said at least two fixed points of position related data are written on said surface;
said apparatus whereby said at least two fixed points of position related data comprise existing features on said surface;
said apparatus whereby said at least two fixed points of position related data comprise existing features obtainable by said data input means;
said apparatus whereby said at least two fixed points of position related data comprise points outside the parameters of said surface;
said apparatus whereby said surface comprises a writing surface and said movable element comprises a writing means for writing on said surface and whereby said writing means comprises said at least one movable point of position related data.
6. An apparatus according to claim 1 selected from the group consisting of:
• said apparatus designed for use with computer;
• said apparatus comprising a computer display;
• said apparatus comprising a computer printer,
• said apparatus comprising a computer;
• said apparatus comprising a CCD camera;
• said apparatus comprising writing means for writing on said surface; • said apparatus comprising a microcomputer;
• said apparatus comprising a display;
• said apparatus comprising function keys;
• said apparatus comprising an illumination means;
• said apparatus whereby said processing means comprises a computer;
• said apparatus whereby said surface comprises a writing surface;
• said apparatus comprising a writing means for writing on said surface and further comprises an original hard copy means for forming an original hard copy made by said writing means on said surface;
• said apparatus whereby said surface is made of a material selected from the group consisting of paper, plastic, glass, metal, synthetic fiber, synthetic material, natural material, and a paper like substance,
• said apparatus whereby said at least-three two fixed points of position-related data comprise a reflecting means for reflecting said at least three two fixed points of position-related data to said data input means,
• said apparatus whereby said moveable element is selected from the group consisting of a stylus shaped moveable element for hand held use, and a hand held moveable element,
• said apparatus whereby said surface comprises an overlay means for overlaying said surface with said overlay means comprising said at least two fixed points of position related data selected from the group consisting of carbon paper, film template and plate, • said apparatus whereby said surface comprises an overlay means for overlaying said surface with said at least two fixed points of position related data;
• said apparatus whereby said data processing means uses the principle triangulation to determine the position of said moveable element relative to said surface,
• said apparatus whereby said at least thfee two fixed points of position-related data comprise a selective reflecting means for selectively reflecting at least one selected frequency of light,
said apparatus whereby said data input means comprises a selective input means far selectively inputting at least one selected frequency of light,
• said apparatus comprising a pressure switch means for turning on said apparatus when pressure is applied to said movable element,
• said apparatus including a focusing means for focusing said surface,
• said movable element comprising a writing means for writing on said surface,
• said movable element comprising a self contained optical stylus, a writing means for writing on said surface, said input means for inputting said at least three points of position related data, said output means for outputting said at least three points of position related data, a microcomputer, a user interface means for communicating with a user and a device interface means for communicating with other devices,
• said apparatus whereby said input means is selected from the group consisting of :
said input means mounted on said movable element; said input means mounted on a computer monitor;
said input means mounted on a portable computer;
said input means mounted to input said at least three points of position-related data.
7. A method of producing an electronic duplicate of markings made upon a surface wherein there is provided:
a position related means for designating the position of at least one point on said surface;
a movable element;
a detector means for detecting said position related means and for generating at least one output signal thereof;
a processing means for receiving and processing said at least one output signal, thereby to determine the position of said movable element relative to said surface;
whereby said position related means are triangulated to obtain the position and movement of said movable element.
8. A method of producing an electronic duplicate of markings made upon a surface according to claim 7 wherein there is provided:
o a movable element comprising a writing means for writing on said surface;
o said surface comprising a writing surface.
9. A method of producing an electronic duplicate of markings made upon a surface according to claim 7 wherein there is provided:
o at least three points of position related data;
o said at least three points of position related data comprising at least two fixed points of position related data and at least one movable point of position related data;
o said surface comprises a substantially two dimensional planar face and further comprises said at least two fixed points of position-related data;
o said movable element comprising said at least one movable point of position related data;
o said detector means comprising an array of light sensitive elements for detecting said at least three points of position related data and an output means for generating at least one output signal thereof;
o said processing mean's comprises means for receiving and processing said at least one output signal form said output means, thereby to determine the position and movement of said movable element relative to said surface;
o whereby said at least three points of position related data are triangulated to obtain the position and movement of said moveable element.
10. A method of producing an electronic duplicate of markings made upon a surface according to claim 9 wherein there is provided:
o said moveable element comprises said detector means, whereby said movable element is movable relative to said surface; o means for determining the path of said moveable element are provided by detecting a sequence of said position-related data.
o means for analyzing the path of said moveable element are provided.
11. A method of producing an electronic duplicate of markings made upon a surface according to claim 9 wherein there is provided:
said at least three points of position-related data comprising position-related data for a plurality of points out said surface;
said at least two fixed points of position related data are printed on said surface;
said at least two fixed points of position related data are written on said surface;
said at least two fixed points of position related data comprise existing features on said surface;
said at least two fixed points of position related data comprise existing features obtainable by said data input means;
said at least two fixed points of position related data comprise points outside the parameters of said surface;
said surface comprises a writing surface and said movable element comprises a writing means for writing on said surface and whereby said writing means comprises said at least one movable point of position related data.
12. A method of producing an electronic duplicate of markings made upon a surface according to claim 9: designed for use with computer;
comprising a computer display;
comprising a computer printer,
comprising a computer;
comprising a CCD camera;
comprising writing means for writing on said surface;
comprising a microcomputer;
comprising a display;
comprising function keys;
comprising an illumination means;
whereby said processing means comprises a computer;
whereby said surface comprises a writing surface;
comprising a writing means for writing on said surface and further comprises an original hard copy means for forming an original hard copy made by said writing means on said surface;
• whereby said surface is made of a material selected from the group consisting of paper, plastic, glass, metal, synthetic fiber, synthetic material, natural material, and a paper like substance, • whereby said at least three points of position-related data comprise a reflecting means for reflecting said at least three points of position-related data to said data input means,
• whereby said moveable element is selected from the group consisting of a stylus shaped moveable element for hand held use, and a hand held moveable element,
• whereby said surface comprises an overlay means for overlaying said surface with said overlay means comprising said at least two fixed points of position related data selected from the group consisting of carbon paper, film template and plate,
• whereby said surface comprises an overlay means for overlaying said surface with said at least two fixed points of position related data;
• whereby said data processing means uses the principle triangulation to determine the position of said moveable element relative to said surface,
• whereby said at least three points of position-related data comprise a selective reflecting means for selectively reflecting at least one selected frequency of light,
• whereby said data input means comprises a selective input means far selectively inputting at least one selected frequency of light,
• comprising a pressure switch means for turning on said apparatus when pressure is applied to said movable element,
• including a focusing means for focusing said surface,
• said movable element comprising a writing means for writing on said surface, • said movable element comprising a self contained optical stylus, a writing means for writing on said surface, said input means for inputting said at least three points of position related data, said output means for outputting said at least three points of position related data, a microcomputer, a user interface means for communicating with a user and a device interface means for communicating with other devices,
• whereby said input means is selected from the group consisting of :
said input means mounted on said movable element;
said input means mounted on a computer monitor;
said input means mounted on a portable computer;
said input means mounted to input said at least three points of position-related data.
• whereby said surface is adapted to function as writing surface;
•said detector means comprises a writing means for marking said writing surface visibly as said detector means and said writing mean move together relative to said surface; and
•whereby said processing means is adapted to produce a digital duplicate of visible marks made on said writing surface.
13. An apparatus comprising coordinate sensor means and processing means further comprising:
a position related means for designating the position of at least one point on said surface;
a movable element; a detector means for detecting said position related means and for generating at least one output signal thereof;
a processing means for receiving and processing said at least one output signal, thereby to determine the position of said movable element relative to said surface;
whereby said position related means are triangulated to obtain the position and movement of said movable element.
14. An apparatus according to claim 13 wherein there is provided:
o a movable element comprising a writing means for writing on said surface;
o said surface comprising a writing surface.
15. An apparatus according to claim 13 wherein there is provided:
o at least three points of position related data;
o said at least three points of position related data comprising at least two fixed points of position related data and at least one movable point of position related data;
o said surface comprises a substantially two dimensional planar face and further comprises said at least two fixed points of position-related data;
o said movable element comprising said at least one movable point of position related data;
o said detector means comprising an array of light sensitive elements for detecting said at least three points of position related data and an output means for generating at least one output signal thereof; o said processing mean's comprises means for receiving and processing said at least one output signal form said output means, thereby to determine the position and movement of said movable element relative to said surface;
o whereby said at least three points of position related data are triangulated to obtain the position and movement of said moveable element.
16. An apparatus according to claim 15 wherein there is provided:
o said moveable element comprises said detector means, whereby said movable element is movable relative to said surface;
o means for determining the path of said moveable element are provided by detecting a sequence of said position-related data.
o means for analyzing the path of said moveable element are provided.
17. An apparatus according to claim 15 wherein there is provided:
said at least three points of position-related data comprising position-related data for a plurality of points out said surface;
said at least two fixed points of position related data are printed on said surface;
said at least two fixed points of position related data are written on said surface;
said at least two fixed points of position related data comprise existing features on said surface; said at least two fixed points of position related data comprise existing features obtainable by said data input means;
said at least two fixed points of position related data comprise points outside the parameters of said surface;
said surface comprises a writing surface and said movable element comprises a writing means for writing on said surface and whereby said writing means comprises said at least one movable point of position related data.
18. A method of producing an electronic duphcate of markings made upon a surface according to claim 15:
• designed for use with computer;
• comprising a computer display;
• comprising a computer printer,
• comprising a computer;
• comprising a CCD camera;
• comprising writing means for writing on said surface;
• comprising a microcomputer;
• comprising a display; • comprising function keys;
• comprising an illumination means;
• whereby said processing means comprises a computer;
• whereby said surface comprises a writing surface;
• comprising a writing means for writing on said surface and further comprises an original hard copy means for forming an original hard copy made by said writing means on said surface;
whereby said surface is made of a material selected from the group consisting of paper, plastic, glass, metal, synthetic fiber, synthetic material, natural material, and a paper like substance,
• whereby said at least three points of position-related data comprise a reflecting means for reflecting said at least three points of position-related data to said data input means,
• whereby said moveable element is selected from the group consisting of a stylus shaped moveable element for hand held use, and a hand held moveable element,
• whereby said surface comprises an overlay means for overlaying said surface with said overlay means comprising said at least two fixed points of position related data selected from the group consisting of carbon paper, film template and plate,
• hereby said surface comprises an overlay means for overlaying said surface with said at least two fixed points of position related data;
whereby said data processing means uses the principle triangulation to determine the position of said moveable element relative to said surface, • whereby said at least three points of position-related data comprise a selective reflecting means for selectively reflecting at least one selected frequency of light,
• whereby said data input means comprises a selective input means far selectively inputting at least one selected frequency of light,
• comprising a pressure switch means for turning on said apparatus when pressure is applied to said movable element,
• including a focusing means for focusing said surface,
• said movable element comprising a writing means for writing on said surface,
said movable element comprising a self contained optical stylus, a writing means for writing on said surface, said input means for inputting said at least three points of position related data, said output means for outputting said at least three points of position related data, a microcomputer, a user interface means for communicating with a user and a device interface means for communicating with other devices,
• whereby said input means is selected from the group consisting of :
said input means mounted on said movable element;
said input means mounted on a computer monitor;
said input means mounted on a portable computer;
• said input means mounted to input said at least three points of position-related data.
• whereby said surface is adapted to function as writing surface; • said detector means comprises a writing means for marking said writing surface visibly as said detector means and said writing mean move together relative to said surface; and
• whereby said processing means is adapted to produce a digital duplicate of visible marks made on said writing surface.
19. An apparatus according to claim 1 whereby pattern matching is used in addition to triangulation.
20. An apparatus according to claim 1 whereby macro and micro triangulation are used.
21. An apparatus for obtaining and outputting the position and movement of a moveable element on a surface comprising:
o a data input means for obtaining position related data;
o a data output means for outputting position related data;
o a data processing means for obtaining and analyzing position related data from said data output means;
o whereby pattern matching is used to determine the instantaneous position and movement of said data input means relative to a pattern on a surface.
21. An apparatus according to claim 23 whereby triangulation is used in addition to pattern matching.
22. An apparatus according to claim 1 through 21 whereby said apparatus comprises a cell phone.
PCT/US2001/044854 2000-11-29 2001-11-29 Optical position determination on any surface WO2002058029A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001298118A AU2001298118A1 (en) 2000-11-29 2001-11-29 Optical position determination on any surface

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US09/725,301 2000-11-29
US09/725,301 US20020118181A1 (en) 2000-11-29 2000-11-29 Absolute optical position determination
US72588300A 2000-11-30 2000-11-30
US09/725,883 2000-11-30

Publications (2)

Publication Number Publication Date
WO2002058029A2 true WO2002058029A2 (en) 2002-07-25
WO2002058029A3 WO2002058029A3 (en) 2009-07-23

Family

ID=27111119

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/044854 WO2002058029A2 (en) 2000-11-29 2001-11-29 Optical position determination on any surface

Country Status (2)

Country Link
AU (1) AU2001298118A1 (en)
WO (1) WO2002058029A2 (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005076194A2 (en) * 2004-01-30 2005-08-18 Hewlett-Packard Development Company, L.P. Products with position identification pattern
JP2007524885A (en) * 2003-02-24 2007-08-30 エレクトロニック・スクリプティング・プロダクツ・インコーポレイテッド Device for optical estimation of information from a flat writing surface
US7646377B2 (en) 2005-05-06 2010-01-12 3M Innovative Properties Company Position digitizing using an optical stylus to image a display
US7826641B2 (en) 2004-01-30 2010-11-02 Electronic Scripting Products, Inc. Apparatus and method for determining an absolute pose of a manipulated object in a real three-dimensional environment with invariant features
US7961909B2 (en) 2006-03-08 2011-06-14 Electronic Scripting Products, Inc. Computer interface employing a manipulated object with absolute pose detection component and a display
US8542219B2 (en) 2004-01-30 2013-09-24 Electronic Scripting Products, Inc. Processing pose data derived from the pose of an elongate object
WO2013158944A1 (en) * 2012-04-19 2013-10-24 Motorola Mobility Llc A touchscreen writing system
US8692212B1 (en) 2012-10-29 2014-04-08 3M Innovative Properties Company Optical digitizer system with position-unique photoluminescent indicia
US8766954B2 (en) 2010-12-21 2014-07-01 Motorola Mobility Llc Active stylus for use with touch-sensitive interfaces and corresponding method
US9068845B2 (en) 2011-12-16 2015-06-30 3M Innovative Properties Company Optical digitizer system with position-unique photoluminescent indicia
US9229540B2 (en) 2004-01-30 2016-01-05 Electronic Scripting Products, Inc. Deriving input from six degrees of freedom interfaces
US9958954B2 (en) 2012-12-13 2018-05-01 3M Innovative Properties Company System and methods for calibrating a digitizer system
WO2018233920A1 (en) * 2017-06-22 2018-12-27 Staedtler Mars Gmbh & Co. Kg Electronic device for generating analogue strokes and for digitally storing the analogue strokes, and an input system and method for digitising analogue recordings
US10753746B2 (en) 2012-11-29 2020-08-25 3M Innovative Properties, Inc. Multi-mode stylus and digitizer system
US11577159B2 (en) 2016-05-26 2023-02-14 Electronic Scripting Products Inc. Realistic virtual/augmented/mixed reality viewing and interactions

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4991305A (en) * 1988-05-30 1991-02-12 Hoya Corporation Spectacle-lens-frame configuration measuring apparatus and article configuration measuring apparatus
US5376790A (en) * 1992-03-13 1994-12-27 Park Scientific Instruments Scanning probe microscope
US5819429A (en) * 1995-10-12 1998-10-13 Metrol Co., Ltd. Touch sensor
US6124845A (en) * 1992-04-21 2000-09-26 Alps Electric Co., Ltd. Coordinate input device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4991305A (en) * 1988-05-30 1991-02-12 Hoya Corporation Spectacle-lens-frame configuration measuring apparatus and article configuration measuring apparatus
US5376790A (en) * 1992-03-13 1994-12-27 Park Scientific Instruments Scanning probe microscope
US6124845A (en) * 1992-04-21 2000-09-26 Alps Electric Co., Ltd. Coordinate input device
US5819429A (en) * 1995-10-12 1998-10-13 Metrol Co., Ltd. Touch sensor

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007524885A (en) * 2003-02-24 2007-08-30 エレクトロニック・スクリプティング・プロダクツ・インコーポレイテッド Device for optical estimation of information from a flat writing surface
WO2005076194A2 (en) * 2004-01-30 2005-08-18 Hewlett-Packard Development Company, L.P. Products with position identification pattern
US9939911B2 (en) 2004-01-30 2018-04-10 Electronic Scripting Products, Inc. Computer interface for remotely controlled objects and wearable articles with absolute pose detection component
US7826641B2 (en) 2004-01-30 2010-11-02 Electronic Scripting Products, Inc. Apparatus and method for determining an absolute pose of a manipulated object in a real three-dimensional environment with invariant features
US9235934B2 (en) 2004-01-30 2016-01-12 Electronic Scripting Products, Inc. Computer interface employing a wearable article with an absolute pose detection component
US8542219B2 (en) 2004-01-30 2013-09-24 Electronic Scripting Products, Inc. Processing pose data derived from the pose of an elongate object
WO2005076194A3 (en) * 2004-01-30 2006-01-05 Hewlett Packard Development Co Products with position identification pattern
US10191559B2 (en) 2004-01-30 2019-01-29 Electronic Scripting Products, Inc. Computer interface for manipulated objects with an absolute pose detection component
US9229540B2 (en) 2004-01-30 2016-01-05 Electronic Scripting Products, Inc. Deriving input from six degrees of freedom interfaces
US7646377B2 (en) 2005-05-06 2010-01-12 3M Innovative Properties Company Position digitizing using an optical stylus to image a display
US7961909B2 (en) 2006-03-08 2011-06-14 Electronic Scripting Products, Inc. Computer interface employing a manipulated object with absolute pose detection component and a display
US8553935B2 (en) 2006-03-08 2013-10-08 Electronic Scripting Products, Inc. Computer interface employing a manipulated object with absolute pose detection component and a display
US8766954B2 (en) 2010-12-21 2014-07-01 Motorola Mobility Llc Active stylus for use with touch-sensitive interfaces and corresponding method
US9068845B2 (en) 2011-12-16 2015-06-30 3M Innovative Properties Company Optical digitizer system with position-unique photoluminescent indicia
US9557827B2 (en) 2011-12-16 2017-01-31 3M Innovative Properties Company Optical digitizer system with position-unique photoluminescent indicia
WO2013158944A1 (en) * 2012-04-19 2013-10-24 Motorola Mobility Llc A touchscreen writing system
US9075452B2 (en) 2012-10-29 2015-07-07 3M Innovative Properties Company Optical digitizer system with position-unique photoluminescent indicia
US9836164B2 (en) 2012-10-29 2017-12-05 3M Innovative Properties Company Optical digitizer system with position-unique photoluminescent indicia
US8692212B1 (en) 2012-10-29 2014-04-08 3M Innovative Properties Company Optical digitizer system with position-unique photoluminescent indicia
US10753746B2 (en) 2012-11-29 2020-08-25 3M Innovative Properties, Inc. Multi-mode stylus and digitizer system
US9958954B2 (en) 2012-12-13 2018-05-01 3M Innovative Properties Company System and methods for calibrating a digitizer system
US11577159B2 (en) 2016-05-26 2023-02-14 Electronic Scripting Products Inc. Realistic virtual/augmented/mixed reality viewing and interactions
WO2018233920A1 (en) * 2017-06-22 2018-12-27 Staedtler Mars Gmbh & Co. Kg Electronic device for generating analogue strokes and for digitally storing the analogue strokes, and an input system and method for digitising analogue recordings
US11073921B2 (en) 2017-06-22 2021-07-27 Staedtler Mars Gmbh & Co. Kg Electronic device for generating analogue strokes and for digitally storing the analogue strokes, and an input system and method for digitizing analogue recordings

Also Published As

Publication number Publication date
WO2002058029A3 (en) 2009-07-23
AU2001298118A1 (en) 2009-07-29

Similar Documents

Publication Publication Date Title
US20020163511A1 (en) Optical position determination on any surface
US20020118181A1 (en) Absolute optical position determination
US5852434A (en) Absolute optical position determination
EP0892971B1 (en) Absolute optical position determination
CN1774690B (en) Implement for optically inferring information from a planar jotting surface
US8542219B2 (en) Processing pose data derived from the pose of an elongate object
CN1175344C (en) Pen like computer pointing device
US6281878B1 (en) Apparatus and method for inputing data
KR100465241B1 (en) Motion recognition system using a imaginary writing plane and method thereof
WO2002058029A2 (en) Optical position determination on any surface
US20050156915A1 (en) Handwritten character recording and recognition device
US20060028457A1 (en) Stylus-Based Computer Input System
KR100777107B1 (en) apparatus and method for handwriting recognition using acceleration sensor
US20040114834A1 (en) Handwritten character recording and recognition device
US20020158848A1 (en) Optical position determination on plain paper
KR100360477B1 (en) Wireless electronic pen
AU759166B2 (en) Device and method for recording hand-written information
JP4292927B2 (en) Pen-type data input device and program
WO1999026191A1 (en) Drawing rendering device
JP3812906B2 (en) Whiteboard eraser
KR200207639Y1 (en) X-y point detect for computer monitor
RU2166796C2 (en) Pen for entering alphanumeric and graphical information in computer
JP2713135B2 (en) pointing device
KR200417164Y1 (en) Computer motion-detecting input device
JPH08314627A (en) Input device

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): JP US

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2001988207

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 2001988207

Country of ref document: EP

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP