WO2000025293A1

WO2000025293A1 - Pen-input device

Info

Publication number: WO2000025293A1
Application number: PCT/US1998/022477
Authority: WO
Inventors: Raphael Cohen
Original assignee: Raphael Cohen
Priority date: 1998-10-23
Filing date: 1998-10-23
Publication date: 2000-05-04

Abstract

A method for determining a motion of a tip on a surface (21) having a pattern of distinct features marked thereon, comprising, imaging the pattern at least in the vicinity ot the tip (24), using a two-dimensional camera (22) coupled to the tip, comparing successive images acquired by the camera and analyzing the images to determine the motion of the tip. Preferably, at least a portion of the pattern is formed by ink from the tip.

Description

PEN-INPUT DEVICE FIELD OF INVENTION

The present invention relates generally to pen input devices, in particular to pen input devices using a two-dimensional camera for determining movement of the pen. BACKGROUND OF THE INVENTION

Personal computers (PC) have become almost indispensable. As such, flexibility in entering data to them is important. In addition to PCs there are many other types of electronic devices which require simple and/or flexible data entry, for example, PDAs, cell phones, watches, pocket dictionaries, set-top boxes, network computers, writing boards and remote controls.

Gyroscopic handwriting recognition is known. British patent GB 2,306,669, "Manual Entry Interactive Paper and Electronic Document Handling and Processing System", the disclosure of which is incorporated herein by reference, utilizes one or more gyroscopes to determine motions of a pen and recognize handwriting. The pen input device also includes a camera for optical recognition of pre-printed bar codes.

Optical recognition of printed matter is known. US patent 4,538,072, "Optical Wand for Reading OCR Characters and Bar-Code Marks", the disclosure of which is incorporated herein by reference, describes an optical wand which has a camera for scanning bar codes and characters, for OCR (optical character recognition). Handwriting recognition on a girded surface, is known. US patent 5,420,943,

"Universal Computer Input Device", the disclosure of which is incorporated herein by reference, describes an input device which can be used as a bar-code scanner, a mouse, a text scanner or a handwriting input device. A linear CCD array is used to determine movement of the pen tip relative to a specialized printed grid. US patent 4,731,526, "Process for Picking up Navigation Data", the disclosure of which is incorporated herein by reference, describes a process for determining relative movement on a map using a photocell to detect changes in reflection from a grid overlay and/or a printed grid. A resistive type overlay is suggested for determining absolute positioning. This process is used for scanning but not for writing. US patent 4,906,843 "Combination Mouse, Optical Scanner and Digitizer Puck", the disclosure of which is incorporated herein by reference, describes a combination mouse, digitizer puck and scanner which can operate as a scanner, a mouse or a digitizer puck. The puck utilizes flux detectors and the mouse utilizes optical movement encoders. SUMMARY OF THE INVENTION

One aspect of some preferred embodiments of the invention relates to determining a motion of a pen tip utilizing a two dimensional camera which images a vicinity of the tip. Preferably, movements of the tip are used to recognize handwriting inputted by the pen. In a preferred embodiment of the invention, the motion determination compares successive images acquired by the camera to determine motion. The motion determined may include position and/or orientation, and/or first and/or second derivatives thereof (i.e., pen velocity and pen acceleration). Preferably the images are compared by correlation. Alternatively or additionally, the motion determination utilizes an image of a trace marked by the pen itself as it moves on an imaged writing surface. Thus, while a gridded or even an arbitrarily patterned writing surface may be used, movements of the pen tip may be detected even on an (previously) unmarked writing surface. In a preferred embodiment of the invention, the position is a two dimensional orientation. However, in some embodiments, only a single dimension is required and/or determined. Additionally, even though three orientation angles are available (rotation, horizontal slant and vertical slant), any number may be determined by the pen, including zero and one. Alternatively or additionally, a composite pen slant may be determined.

In a preferred embodiment of the invention, the pen includes a pressure transducer to detect when the pen is pressed against a writing surface and/or an amount of applied pressure and/or that the pressure is over a threshold. Preferably, the contact determination is used to decide if writing is to be detected. Alternatively or additionally, the relative positions of the pen and surface and/or relative movements thereof are determined using a range finder or by optical processing of the image acquired by the camera.

An aspect of some preferred embodiments of the invention relates to identification of an absolute position of a pen. Two types of absolute positioning may be identified. In a first type of positioning, an area in which the pen is positioned is identified. In one example, a preprinted form may include a plurality of areas, each with a different background pattern. When the pen is positioned in one of the areas, the background pattern of the area may be analyzed by the pen to determine the identification of the area. A second type of absolute positioning optically encodes a position in a grid. Thus, an image of a grid portion underlying the pen may be analyzed to determine an absolute position and/or orientation of the pen relative to the grid.

An aspect of some preferred embodiments of the invention relates to using the same pen device for handwriting recognition, scanning of bitmap images and/or inputting of graphical data, by tracing. Alternatively or additionally, the pen may be used as a pointing device (e.g. emulating a mouse). In a preferred embodiment of the invention, a pre-printed pad is provided with differently patterned areas for a mouse pad, a writing area and/or function keys. When the pen is positioned over the mouse pad, it operates as a mouse, when it is positioned over the writing surface, it functions as a handwriting input device and when positioned over a function key, the pen activates the key, as if pressed on a keyboard. In a preferred embodiment of the invention, the grid is invisible to a human eye, for example being reflective mainly in the infrared spectrum and/or it being printed using a fluorescent material which is activated by a special light source on the pen. In a preferred embodiment of the invention, the grid is provided by overlaying a transparency having a grid printed thereon, over a work surface. An aspect of some preferred embodiments of the invention relates to using a pen device to annotate printed text. In one example, the text comprises a form, possibly partially filled, and a user enters text and/or corrections to existing text using a pen input device. In another example, the text comprises a printed page.

An aspect of some preferred embodiments of the invention relates to controlling the functionality of a pen input device. Preferably, various functions of the pen and/or of a software application controlling the pen are controlled and/or initiated using switches on the pen. Alternatively or additionally, the pen recognizes gestures and/or written symbols made by the pen as indicating specific function requests. Such function requests may include control of the pen itself, entering non-printable characters (such as "delete") and/or indications of actions to perform on a text, for example to highlight a circled text section.

An aspect of some preferred embodiments of the invention relates to handwriting recognition on multiple surfaces, including for example, paper, computer screens (in which a pattern may be actively displayed and/or may be passively formed by the display elements themselves), black-board, white-board and/or pre-printed forms. In a preferred embodiment of the invention, the pen device is provided as a camera attachment to an existing writing instrument, for example a pen or a marker.

An aspect of some preferred embodiments of the invention relates to electronically controlling the flow of ink in a pen (or engraving action in an engraving tool). Preferably, ink is allowed to flow only if writing is allowed, for example on a white sheet of paper. Preferably, the allowance of writing on a surface is responsive to an identification of a pattern printed on the writing surface.

An aspect of some preferred embodiments of the invention relates to various levels of connection between a pen device and a target computer. Although the pen may be connected to a computer by wire, in a preferred embodiment of the invention, the connection is a wireless connection. The processing ability of the pen may be provided on the pen itself, on a connected computer or utilizing a dedicated processing hardware, preferably connected between the pen and the computer.

In a preferced embodiment of the invention, the pen includes an internal memory and/or processor for storing handwriting and/or commands. Thus, the pen may be used as a stand alone device for marking up a printed copy of a document. When the pen is connected to a computer, an electronic copy of the document is updated and, optionally printed. Alternatively or additionally, the pen may comprise a computer, with an external connection required to print out a document. There is thus provided in accordance with a prefened embodiment of the invention a method for determining a motion of a tip on a surface having a pattern of distinct features marked thereon, comprising: imaging the pattern at least in the vicinity of the tip, using a two-dimensional camera coupled to the tip; comparing successive images acquired by the camera; and analyzing the images to determine the motion of the tip.

Preferably, the method comprises forming at least a portion of said pattern by ink released from said tip. Alternatively or additionally, comparing comprises correlating at least parts of the images. Alternatively or additionally, comparing comprises feature extraction and matching of said images.

Alternatively or additionally, the method comprises providing said pattern on the surface prior to the imaging, wherein said pattern includes a coded pattern for absolute positioning of the tip. Preferably, the coded pattern comprises a plurality of areas, each with a different background pattern and said camera has a field of view larger than an area of a single pattern. Preferably, the plurality of areas comprises a plurality of square bar codes that form a checkerboard.

Alternatively or additionally, the coded pattern is a coded grid, wherein each line of the grid is a specific combination of lines of varied thickness and spaces of different widths. Preferably, the coded grid has a likeness of a woven lattice. In a prefened embodiment of the invention, at least part of said pattern is invisible to a human eye. Preferably, said pattern is reflective in the infrared spectrum. Alternatively or additionally, said pattern comprises a fluorescent material. Alternatively or additionally, said surface comprises a transparency having said pattern printed thereon. Alternatively, said surface comprises a plain paper with a pattern printed thereon. Alternatively or additionally, said pattern comprises a grain of the surface.

In a preferred embodiment of the invention, determining the motion of the tip includes determining relative translation. Alternatively or additionally, determining the motion of the tip includes determining relative orientations. Alternatively or additionally, determining the motion of the tip includes determining an absolute position. Preferably, said position comprises a one-dimensional position.

Alternatively or additionally, said position comprises a two-dimensional position.

In a preferred embodiment of the invention, determining the motion of the tip includes determining an absolute orientation.

In a prefened embodiment of the invention, said orientation comprises a rotation angle of said tip around an axis perpendicular to the surface. Alternatively or additionally, said orientation comprises an angle between said tip and said surface.

In a preferred embodiment of the invention, determining the motion comprises determining a velocity of the motion. Alternatively or additionally, determining the motion comprises determining a rate of change of the velocity of the motion.

In a preferred embodiment of the invention, the method comprises utilizing said detected motions to recognize handwriting. Preferably, handwriting recognition comprises handwriting recognition utilizing motion vectors. Alternatively, handwriting recognition comprises utilizing both motion vectors and images of the surface for handwriting recognition.

Alternatively or additionally, handwriting recognition comprises: generating an image utilizing said determined motion; and recognizing at least one symbol in said generated image.

Alternatively or additionally, handwriting recognition comprises utilizing a user- dependent database to perform said handwriting recognition.

In a preferred embodiment of the invention, the method comprises utilizing said determined motions for graphical input. Alternatively or additionally, the method comprises utilizing said determined motion for controlling a cursor. Preferably, controlling a cursor comprises emulating a mouse. In a preferred embodiment of the invention, the method comprises reconstructing a scanned image from said acquired images, utilizing said determined motions. Preferably, the method comprises analyzing said images to recognize special codes.

In a preferred embodiment of the invention, said tip forms a part of a pen input device, and the method comprises: recognizing a particular pattern in at least one of said acquired images; and modifying an operational mode of said pen input device responsive to said recognized particular pattern.

Alternatively or additionally, said tip forms a part of a pen input device, and the method comprises: receiving a voice input by said pen; and modifying an operational mode of said pen input device responsive to said voice input.

Alternatively or additionally, said tip forms a part of a pen input device, and the method comprises: recognizing an entry of a symbol utilizing said determined motions; and modifying an operational mode of said pen input device responsive to said recognized symbol.

Alternatively or additionally, said tip forms a part of a pen input device, and the method comprises: recognizing said determined motions as gestures; and modifying an operational mode of said pen input device responsive to said recognized gestures.

Alternatively or additionally, said tip forms a part of a pen input device, and the method comprises: detecting when the tip in contact with said writing surface; and modifying an operational mode of said pen input device responsive to said detection.

Preferably, said modifying comprises detecting motion of the tip responsive to said detected contact. Alternatively or additionally, said modifying comprises operating said camera responsive to said detected contact. In a prefened embodiment of the invention, modifying comprises selectively operating said pen input device in at least one of the following input modes: handwriting input, graphical input and pointer emulation. Preferably, the input modes include a hypertext link selection mode, in which the pen may be selectively operated.

In a preferred embodiment of the invention, modifying comprises controlling said camera. Alternatively or additionally, modifying comprises controlling the release of ink.

There is also provided in accordance with a prefened embodiment of the invention, a method of controlling ink flow in a pen, comprising: determining when said pen is in contact with a surface whose marking is allowed; and automatically electronically controlling a flow of ink, responsive to said determination. Preferably, determining comprises determining if said pen is operating in a pointer emulation mode.

There is also provided in accordance with a prefened embodiment of the invention, a pen input device comprising: a tip; a two dimensional camera, coupled to said tip, which images a surface at the vicinity of the tip; and circuitry for determining motion of said tip relative to said surface, by analysis of successively acquired images.

Preferably, the pen comprises an electrical wire connection to a computer. Alternatively, the pen comprises a wireless connection to a computer for data generated in the pen.

In a prefened embodiment of the invention, at least some of said circuitry is comprises in a dedicated processing unit. Alternatively or additionally, at least some of said circuitry comprises software operable with on said computer.

In a prefened embodiment of the invention, said pen comprises a memory. Preferably, said memory stores at least portions of said images. Alternatively or additionally, said memory stores data responsive to said movement determination. In a prefened embodiment of the invention, said pen comprises an internal power source. Alternatively or additionally, the pen comprises a light source, coupled to the pen input device, to illuminate the surface. Alternatively or additionally, the pen comprises a pressure transducer which determines if the tip is in contact with the surface. Alternatively or additionally, the pen input device is selectively operable in a plurality of input modes, including at least handwriting recognition and pointer emulation. Preferably, the input modes include pattern recognition. Alternatively or additionally, the input modes include hypertext link selection.

In a prefened embodiment of the invention, the pen comprises an external software, and said mode is determined by the software. Preferably, said software controls the pen to operate in said different modes. Alternatively or additionally, said software analyses signals from the pen, responsive to said mode.

In a prefened embodiment of the invention, the pen comprises a personal computer on which said software is executed. In a prefened embodiment of the invention, the pen comprises at least one switch for controlling the operation of the pen. Alternatively or additionally, said tip comprises an ink dispenser. Preferably, the pen comprises an ink flow control which selectively allows ink to be dispensed from said tip onto said surface. In a prefened embodiment of the invention, said circuitry comprises a memory for reconstructing virtual pen marking from said determined movements.

Alternatively or additionally, said circuitry recognizes handwriting patterns based on said determined movements.

Alternatively or additionally, said circuitry determine a three-dimensional position of said tip relative to said surface.

Alternatively or additionally, said circuitry determines a three-dimensional orientation of said tip relative to said surface.

Alternatively or additionally, said pen comprises a rechargeable power source.

There is also provided in accordance with a prefened embodiment of the invention, an implement mountable camera, comprising: a camera having a field of view; and a bracket for attaching said camera to a writing implement, such that said field of view images at least a vicinity of a tip of said elongate writing implement.

Preferably, the camera comprises circuitry for analyzing images acquired by said camera to determine motion of said tip.

There is also provided in accordance with a prefened embodiment of the invention, a controllable pen having a pen tip, comprising: an ink dispenser; and an ink flow control which selectively blocks ink from being dispensed from said ink dispenser. Preferably, said ink flow controller comprises an electrical flow controller.

There is also provided in accordance with a prefened embodiment of the invention, a pen input device comprising: a tip; a two dimensional camera coupled to said tip and which images a surface at the vicinity ofthe tip; and a rechargeable power source for powering said camera. Preferably, the pen comprises a connector to a computer, wherein said connector is adapted to hold said pen while not is use and wherein said connector comprises a data communication adapter for said computer and a power adapter to recharge said rechargeable power source. BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be more clearly understood with reference to the following detailed descriptions of non-limiting prefened embodiments of the invention in which same reference numbers are used to denote same articles throughout the figures and in which: Fig. 1 is a schematic representation of a pen input device in accordance with a prefened embodiments of the present invention;

Fig. 2 is a schematic representation of an alternative embodiment of a pen input device;

Figs. 3A - 3E illustrate a handwriting recognition method in accordance with a prefened embodiment of the invention; Fig. 4 is a schematic representation of a data entry pad, in accordance with a prefened embodiment of the present invention;

Figs. 5A-5F comprise a flowchart of a program for controlling a pen input device, in accordance with a prefened embodiment of the invention;

Fig. 6 is a schematic illustration of a system configuration including a computer and a pen input device, in accordance with a prefened embodiment of the invention;

Fig. 7A and 7B are block diagrams of a wired pen input device in accordance with a prefened embodiment of the invention; and

Fig. 8A and 8B are block diagrams of a wireless pen input device in accordance with a prefened embodiment of the invention. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Fig. 1 is a schematic representation of a pen input device 20 in accordance with a prefened embodiments of the present invention. Pen 20 includes a body 28, an ink dispenser 30 and a camera 22 which images an area 24 of a writing surface 21, adjacent the tip of ink dispenser 30, preferably using a lens 26. Images acquired by camera 22 are preferably analyzed (as described below) to determine movements of pen 20. Preferably, the signals are sent to a processor (not shown in this Fig.), via a connector 38. In some prefened embodiments of the invention, the connector is a wired connector. Alternatively or additionally, the connector comprises a wireless connection.

In a prefened embodiment of the invention, pen 20 includes a pressure sensor and/or switch 36 which generates a signal responsive to the pen being put to paper. In one embodiment, sensor 36 is coupled to ink dispenser 30 using a spring 34. Alternatively or additionally, pen 20 includes a sensor (not shown) which generates a signal responsive to ink dispenser 30 being in a writing position. In Fig. 1, camera 22 is shown mounted on a bracket 32 external top pen 20. In some embodiments, camera 22 and bracket 32 are provided as a stand-alone device which may be attached to many writing implements, for example a favorite pen. Fig. 2 is a schematic representation of an alternative embodiment of a pen input device, 40, in which a camera 42 is located inside the body of pen 40. A lens 46 is preferably ananged at a writing tip of the pen, to image area 24. Alternatively or additionally, the camera may be provided as a wrap around for an existing pen, so that when the camera is mounted on the pen, an imaging configuration similar to that of Fig. 2 is created.

It should be appreciated that the term "pen" is used herein for convenience. As used herein, a pen may be any of a plurality of writing devices known in the art, for example, a pen, a chalk stick, a marker, an engraving tool and/or a paintbrush. Additionally, as described below, pen 20, in some embodiments thereof, may be used to generate signals which are substantially the same as those generated by a gyroscopic pen input device or by other types of pen input devices. Thus, pen 20 may be used in their place in many applications. Alternatively or additionally, pen 20 may use a surface of a display, such as an LCD or a CRT as a writing surface. It should be noted that generating a pattern on a display is straight forward and, in some ways, more flexible than using a pre-printed sheet (described below). Also, many display devices have a noticeable grain which may server as a pattern. Furthermore, in some prefened embodiments of the invention pen 20 selectively leaves no mark on the writing surface. Lens 26 (Fig. 1) or lens 46 (Fig. 2) are preferably regular imaging lenses. Alternatively or additionally, a fish-eye lens is used, so that better resolution is achieved at portions of area 24 nearer the tip of pen 20 and/or while imaging a larger area 24. Alternatively or additionally, lens 46 (or lens 26) includes a fiber optic wave guide to channel light to camera 42. In a prefened embodiment of the invention, pen 20 (or pen 40) includes a light source (not shown) to illuminate area 24. It should be noted that for many applications a low resolution camera, a camera with a wide dispersion of gain for light detectors and/or a camera in which some of the light detectors are damaged may be used. Preferably, the camera quality and/or resolution are selected to match a task for which the pen is used.

Alternatively or additionally, lens 26 is a magnifying lens. In a preferred embodiment of the invention, the magnification is set so that a grain of a writing surface will be visible. For example, a hand and/or a table will appear to have a pattern marked thereon if the magnification is sufficient.

In a prefened embodiment of the invention, lens 26 comprises a zoom lens, having variable magnification settings (and/or focus settings). Preferably the lens is a manual lens. Alternatively or additionally, lens 26 comprises an electronic zoom lens. In a prefened embodiment of the invention, the zoom setting is set during a calibration and/or responsive to the task to be performed. Alternatively or additionally, the zoom setting is automatically determined based on an imaged grain size of a writing surface. Thus, in some cases, the zoom setting may change during a single task.

In a preferred embodiment of the invention, movements of pen 20 are determined by comparing consecutive images acquired of areas 24 and analyzing the images to determine a relative movement and/or a relative rotation between the images. In a prefened embodiment of the invention, the images are compared by correlation of, at least parts of, the images. Alternatively or additionally, the images are compared using other techniques known in the art, for example feature extraction and then feature matching.

In some prefened embodiments of the invention, writing surface 21 includes a grid marking, possibly in which each grid line comprises a plurality of lines. The number, thickness and/or distance between the lines may be used to determine a relative and/or absolute position. In a prefened embodiment of the invention, the lines in the horizontal and vertical directions are overlaid, to form a weave-like pattern. Alternatively or additionally, the lines simply meet at vertexes of the grid. Alternatively or additionally, the marking comprises a checkerboard of bar-codes. Alternatively or additionally, the marking comprises an arbitrary, possibly nonrepeating pattern. Thus, images of two overlapping areas 24 are more easily conelated to determine movement. However, it should be appreciated that such marking is not necessary in all prefened embodiments of the invention. Rather, the ink dispensed by ink dispenser 30 can provide enough information content (possibly by itself) in areas 24 for a successful movement detection.

In a preferred embodiment of the invention, sensor 36 is used to determine if pen 20 is in contact with writing surface 21. Preferably, movement detection is performed only when pen 20 is in contact with surface 21. Alternatively or additionally, when pen 20 is not in contact with surface 21, pen 20 works in a different mode. In one example, a focal length of lens 26 changes so that camera 22 images objects further away, such as a nearby writing surface. In another example, pen 20 changes to a volume motion mode, in which a three-dimensional position is determined, for example by measuring a three-dimensional vector of motion flow. An image of surroundings acquired by camera 22 may be analyzed to determine motion in up to six degrees of freedom: rotation of the image, scale of the image, X and Y axis movement and distortion along X and along Y axes. Thus, spatial gestures made with the pen may also be captured. Alternatively or additionally, sensor 36 may be used to signal other changes in the mode of operation of pen 20, for example, operating in a manner analogous to a mouse button and/or a pressure pad.

Handwriting recognition may then be performed utilizing the acquired images and/or the determined movements. Two basic types of handwriting recognition may be applied, vector based and bit-mapped based. In the bit-mapped based handwriting recognition method, individual images of areas 24 are accumulated into a larger bit map, using the conelation and/or the movement analysis. An OCR algorithm, many of which are known in the art, may then be applied to the large bit-map, to determine the character. The pen may also buffer the larger bit map, so that entire words are OCRed in a single operation.

In the vector-based handwriting recognition method, the movement vectors themselves are used to identify the letter. In addition, sensor 36 and/or an analysis of the acquired images may be used to determine when pen 20 did and/or did not contact surface 21. The frame rate of the acquired images (and movement analysis) is preferably determined to match an expected pen motion speed, letter complexity (which is different for English and Chinese) and/or geometry of area 24. Preferably, a desired frame rate is between 10 and 24 frames per second. It should be noted however that the snapshot time for an image may be very short, for example one millisecond. Alternatively or additionally, the snapshot time is longer and the acquired images are analyzed to detect motion based on a direction and/or magnitude of smear in the image.

Alternatively or additionally, a combination of methods may be used, for example using vector information as hints for the OCR process or using an image of the actual motion, in association with the motion vector, to better define the exact trajectory of the pen tip.

Figs. 3A-3E illustrate a handwriting recognition method in accordance with a prefened embodiment of the invention. In Figs. 3A-3D, pen 20 generates consecutive line segments 50a- 50d and camera 22 acquires consecutive images 52a-52d. Each pair of consecutive images (e.g., 52a,52b) are analyzed to determine relative motion of the tip of pen 20. Alternatively or additionally, non-consecutive images may also be analyzed to determine relative motion therebetween. Non-consecutive image comparison is especially useful to conect for drift enors which may accumulate when serially comparing a large number of images.

Fig. 3E illustrates an exemplary application of the present invention, namely, a word processor 60. An area 64 is a main display area and includes a cursor 66, to the right of which a new character is to be entered. A handwriting recognition area 62 may be used to display a character as it is being entered. Once the character entry is complete, the character is moved to the next character location to the right of cursor 66.

Alternatively or additionally, pen 20 may be used for Internet browsing. In a prefened embodiment of the invention, the pen is used to select hypertext links, for example by indicating a link on the screen (e.g., circling them) and/or by scanning a link. In a prefened embodiment of the invention, the image of the link is within the field of view of the camera, when the link is circled. Alternatively or additionally, the word in the link is written out to activate the link. Alternatively or additionally, the camera image is used to identify patterns, for example those which indicate specific links. Alternatively or additionally, pen 20 is used for data entry in preparing of e-mail, to select and/or enter a telephone number to be dialed, as part of a chat session and/or for multiuser games, on a network or on a single computer (with multiple pens connected thereto). It should be noted that these applications may thus be used without requiring a keyboard, in accordance with some prefened embodiments of the invention. Alternatively or additionally, the data entry using pen 20 may be used for handwriting analysis, for example for signature verification, for example for secure electronic commerce. Alternatively or additionally, the handwriting analysis may be used for graphology and/or for education, e.g., for teaching of handwriting.

Area 62 is not required in all embodiments of the invention, since, in some embodiments, any surface may be used for data entry and an ink feedback is provided to the user regarding his actual pen motions.. Thus, display 62 may be a low resolution display, for example a low resolution numeric display. Such a low resolution display is usually cheaper, lighter and/or requires less power to operate than a high-resolution graphic display.

Alternatively or additionally, to using sensor 36 to determine contact of pen 20 with surface 21, other technologies may be used. In one example, a quality of focusing of the acquired images is used to determine a distance between camera 22 (and hence pen 20) and surface 21. Alternatively or additionally, a range finder, such as an ultrasonic or laser range finder, may be used. Alternatively or additionally, if the pattern on surface 21 is known (for example having been acquired in a pervious image), the scale of the cunently acquired pattern may be used to determine contact with the paper. Alternatively or additionally, it should be noted that when writing, the tip of the pen usually travels in a plane, unlike in spatial gesturing. An analysis of the acquired images may be used to determine that the pen is traveling on a plane and, hence, on a writing surface. Alternatively or additionally, it is noted that when holding a pen, axial movements are generally used to bring the pen to a writing surface, while transverse movements are usually used to guide the pen over the surface. Movement with limited or no axial component may be assumed to be connected with a writing surface. Alternatively or additionally, it is noted that ink is generally dispensed only when ink dispenser 30 is in contact with surface 21. Analysis of the acquired images may be used to determine if any additional ink was dispensed.

In a prefened embodiment of the invention, a single pen 21 may be used for multiple modes of data input. Fig. 4 is a schematic representation of a data entry pad 70, in accordance with a prefened embodiment of the present invention. In a preferred embodiment of the invention, the pad is printed on a plain or a special paper sheet, for example using a laser printer and/or a photo-copier. Pad 70 may include a designated mouse pad portion 72, a designated writing portion 74 and/or one or more designated function key portions 76. Preferably, pen 20 operates in different modes over each of the portions. In mouse pad portion 72, relative movement of pen 20 moves a cursor on a computer screen (not shown). Preferably, mouse pad 72 (and possibly other portions of entry pad 70) is coated with a material which resists ink from ink dispenser 30. Alternatively or additionally, mouse pad 72 is erasable. Alternatively or additionally, when mouse pad 72 is all marked up, a new data entry pad 70 is used. Alternatively or additionally, a disappearing ink is used. Mouse pad portion 72 may be automatically detected based on a background pattern printed thereon and identified by pen 20. Alternatively or additionally, a switch on pen 21 is used. Alternatively or additionally, a gesture (described below) is used to change modes in pen 20.

When pen 20 is placed over a function key portion 76, pen 20 preferably generates a signal indicating that the function key was "pressed". Preferably a degree of hysteresis is provided so that a second depressing requires moving the pen a significant distance away. A particular function key may be identified based on OCR of the text printed in the key area. Alternatively or additionally, a special pattern is imprinted on the background of the key area. Alternatively or additionally, a one or two dimensional bar-code is imprinted and read by pen 20. Alternatively or additionally, a control or a gesture is used, as for the mouse function described above.

Writing portion 74 is preferably used for handwriting input, as described above. Figs. 5A-5F comprise a flowchart of a program for controlling a pen input device, in accordance with a prefened embodiment of the invention. It should be appreciated that this flowchart is only an example of one possible control structure for a pen input device, many other control structures will occur to a person skilled in the art. Fig. 5A illustrates a main loop, in which depressing of two mouse button simulating switches (mouse 1, mouse2) is determined. In Figs. 5B and 5C, a signal is sent to a host computer responsive to pressing and/or releasing of the buttons.

Fig. 5D illustrates the operation when camera 22 is operational, for example responsive to a camera switch being activated. If imaged area 24 includes a pattern of mouse pad 72, a mouse advance process is activated. If imaged area 24 includes a pattern of a function key, the key is detected and then sent to the host computer keyboard driver. Otherwise, if imaged area 24 includes a pattern of a writing portion 74, the pattern is used to assist in determining movement of pen 20. Preferably, no ink is released if any of the above patterns are identified, since none are required. Ink dispenser 30 is preferably electronically controlled to restrict flow of ink when not needed. If imaged area 24 does not include any of the above background patterns, ink dispenser 30 is preferably electronically controlled to allow ink to flow onto a writing surface.

In Fig. 5D, handwriting recognition utilizes a "replica algorithm", described below, which identifies movements of the pen and/or whether a continuous line was created. In a prefened embodiment of the invention, especially when bit-mapped based OCR is used, an image is generated in a bit-map image memory, which image includes the contribution of the individual pen motions. Such an image is preferably filtered to remove any underlying grid. Alternatively or additionally, if ink dispenser 30 was electronically controlled not to dispense ink, the bit map is modified to include a line which would have been drawn if ink had been allowed to flow. Thus, the bit map may be generated by assessing the contributions of pen motions rather then by combining images. Removing an underlying pattern is thus greatly simplified or not required at all. Additionally or alternatively, removing markings generated by the pen itself is also facilitated, by knowledge of the pen dispensing ink and/or the pen movements.

The image memory (which may be maintained for example on the pen and/or on an attached computer) is then analyzed to detect a character and/or a gesture. If no character is identified, further pen movements may be required in order to recognize a character. Preferably, a temporal threshold is used to determine when no further movements should be accepted. If no character is identified, an enor code may be sent to the host computer or a default character may be recognized. Alternatively another recognition library is checked. Once a character is recognized, its code is preferably sent to a keyboard driver of the host computer and the bit-map memory is preferably erased. Fig. 5E illustrates a "replica algorithm". A new and a previous images are conelated and the conelation peak determined. Additionally, a determination is made to see if there is a contiguous drawn line from between an origin in the previous image and the origin in the new image. The movement and the existence of a line are preferably returned to a calling program. Fig. 5F illustrates a "mouse advance algorithm". A new and a previous image are conelated and the peak conelation determined. The amount shift required for the peak conelation is returned as a movement.

Electronic ink control may be activated, as described above, by an underlying pattern. Alternatively or additionally, the activation may be controlled by an activation mode of the pen, for example, responsive to movement of the pen, such as a gesture and/or using a physical control on the pen (which is electrically coupled to ink dispenser 30). In a prefened embodiment of the invention, the electronic ink dispenser comprises a valve which controls the flow of ink. Alternatively or additionally, the electronic ink dispenser comprises an actuator which retracts an ink-dispensing tip relative to the tip of pen 20, so that the ink dispenser does not contact a writing surface and dispense ink.

In a prefened embodiment of the invention, other types of data input are facilitated by pen 20. In one example, pen 20 may be used for scanning images, with the scanned areas 24 being assembled as a mosaic, preferably using conelation. In another example, pen 20 is used to enter graphics, which may be stored as bit maps and/or as vectors. In another example, pen 20 is used for filling out and/or amending forms. Preferably each field on the form has a different background pattern, so that they can be individually identified. Alternatively or additionally, each filed includes a bar-code and/or an alphanumeric code which can be acquired by camera 22.

In a prefened embodiment of the invention, pen 20 may be used to enter gestures, which are movements of pen 20 which do not generally conespond to any identifiable character. Examples of gestures include editing marks (e.g., swap letters), selection of text, special commands to pen, for example to increase resolution or control ink flow and/or gestures to change the mode of operation of the pen (e.g., handwriting mode -> mouse mode). In a prefened embodiment of the invention, gestures are associated with a location relative to printed text, for example for editing of forms.

As can be appreciated, various less or more complicated and/or automated logics may be defined for a pen input device, utilizing the myriad of available inputs: physical switches, imaged patterns, text and or bar codes, gestures and/or movements and/or utilizing one or more of the myriad of available functional modes, such as handwriting recognition, scanning, function key entry, pen control and/or mouse function. Additionally, pen 20 may provide one or more of a plurality of feedback functions, including, sound (from a host computer or the pen), display of input received at a host computer, a display on the pen itself and/or messages on the host computer. In a prefened embodiment of the invention, pen 20 is used to mark up a pre-printed sheet of paper, the content of which is preferably known to a host computer. If the content is not known, the paper may be scanned using pen 20. Various methods may be provided for absolute localization of pen 20 on the paper. In one example, the paper is pre-printed with a varying pattern which is unique at each point on the paper. In another example, such a pattern (or grid) is provided as a transparency. In another example, the printed matter is utilized as such a varying pattern, providing that area 24 is large enough to always include a uniquely printed section of paper. In some cases, especially if the imaged text is unique, it is not necessary to determine a position. Rather, the annotations performed by pen 20 are directly applied to the text, which text is determined based on its being unique. In another example, pen 20 is moved along the paper from a landmark position, so that the position of pen 20 relative to the landmark position is known. In some of the above cases, text on the pre-printed paper is identified. In other cases, only a low-resolution image of the text needs to be used. In another example, a forest (of unconnected graph segments) of local annotations and/or images of sections of the paper are acquired. Gradually, individual localities on the paper are merged, as areas 24 which overlap two such localities are acquired. It should be appreciated that this type of merging of imaged localities may also be useful in handwriting recognition, if the pen moved more than the size of area 24 in one step.

In a prefened embodiment of the invention, when a grid or other pre-printed pattern is provided for the writing surface, the pattern is preferably printed using an invisible and/or marginally visible ink, for example an IR ink, which is visible to camera 22. Alternatively or additionally, a fluorescent ink is used, which preferably emits light in response to light, such as UV light, from a light source on pen 20.

In a prefened embodiment of the invention, pen 20 is calibrated prior to use. Calibration preferably includes setting the field of view of camera 20 and/or its focusing. In a prefened embodiment of the invention, when camera 22 is provided as a separate bracket mounting as shown in Fig. 1 , for example, camera 22 is automatically calibrated using the tip of the pen as a point of reference. Alternatively or additionally, such automatic calibrations may utilize a grid or other patterned portion of a data entry pad. In a preferred embodiment of the invention, a pen may be personalized for a particular user, language, target application program and/or style of use. The following items may be customized, for example: the language, writing direction, shape of characters, use of shorthand, prefened gestures and their meaning (e.g., application control, entering of non-printing characters, and/or assigning scripts commands and/or macros to gestures), writing angle, and/or rotation of the pen. In a prefened embodiment of the invention, a user can view such a personalization on a computer screen, so that he can keep track of them. In some embodiments of the invention, the pen will upload the gestures to the computer, to be displayed. Alternatively or additionally, different resolution mouse pads may be used for different people. In some cases, the same mouse pad is used to emulate different resolution steps. Alternatively, the pen identifies a step size on the mouse pad and uses this step size to select a mouse movement resolution and/or step. In a preferred embodiment of the invention, personalization information is stored in a database on the computer and/or on the pen. When the information is stored on the pen, a display and/or controls on the pen are used to view and/or modify the information. Alternatively or additionally, the maintenance of the information is via a computer connection, which preferably either directly controls the pen or downloads information and/or settings to it.

It should be noted that pen 20 may be used, in some embodiments of the invention, to enter characters in non-Latin languages and/or pictogram-based languages, such as Arabic, Japanese and Chinese. An advantage of pen 20, in some embodiments thereof, over prior art devices, is that the combination of imaging and motion detection may be used to determine and track high resolution features of the writing, for example stroke width.

In a prefened embodiment of the invention, especially when entering a gesture, the motion of the pen may be differently interpreted depending on what is already printed on the surface. In one example, a small "v" over a line of printed text will be interpreted as a gesture to insert the following text, while on a blank page it will be interpreted as a letter "V. In another example, the written markings may combine with existed pre-printed (or pre-written) marking to firm a single symbol, character and/or pattern to be recognized.

Fig. 6 is a schematic illustration of a system configuration, in accordance with a prefened embodiment of the invention including, a computer 82, a pen input device 20 and/or a dedicated hardware 80 which preferably includes a DSP. The following three dimensions of the configuration may be identified:

(a) Is required processing performed in pen 20, computer 82, using dedicated hardware 80 (possibly installed in computer 82) or in a combination of the above. (b) Are the connections between pen 20, computer 82 and dedicated hardware 80 wired, wireless or a combination.

(c) Does pen 20 require real-time communication with computer 82 or are off-line operations supported and/or sufficient. Referring to dimension (a), in the current state of the art, the division of processing between pen 20, computer 82 and dedicated hardware 80 is a design question; any contemplated division may be implemented, at a cost, using cunent technology, with a possible speed tradeoff. In some embodiments, especially if a high powered CPU is available no dedicated hardware and/or processing on the pen is used at all. Referring to dimension (b), again, any contemplated interconnection scheme may be implemented, however, it is noted that processing and/or image acquisition require a significant amount of power, thus, it may be desirable for pen 20 to be connected by a wire to one of the other elements of the system. Alternatively or additionally, a quick-charge docking station may be provided, so that pen 20 is only required to store power for short periods of time, between docking instances, referring to dimension (c), it is noted that a main usage of pen 20 is as a real-time input device to computer 82. However, it should be noted that for marking up a preprinted paper, for marking up or filling a form, for map navigating, for measuring a distance along a route on a map, for making measurements on an architectural drawing (or display) and/or for scanning text, images, graphics and/or handwriting, there is no need to communicate with a host computer. In addition, if feedback means, such as a display, are provided on the pen, the pen can form a complete computer system in itself. A pen display suitable for incorporation in a pen 20, in some prefened embodiments of the invention, is described for example in EP patent 0094867, the disclosure of which is incorporated herein by reference.

Referring to dimension (c), in a prefened embodiment of the invention, pen 20 can operate in a stand alone mode and upload data to a computer when it is connected to one. the data may be partially analyzed by the pen. Alternatively the pen may utilize an online connection with the computer. Alternatively or additionally, the computer may be used to download updated programming, configuration files and/or personalization settings to the pen.

Fig. 7A and 7B are block diagrams of a wired pen input device in accordance with a prefened embodiment of the invention. Fig. 7A is a block diagram of a dedicated device 80 suitable for a wired pen. Although an analog interface to the pen is shown, a digital or a hybrid interface may also be using in some prefened embodiments of the invention. Alternatively or additionally, the image is conveyed from the pen using a fiber-optic cable. Fig. 7B is a block diagram of the pen itself. Special note should be taken of the power supply management blocks, which may be used to put the pen and/or dedicated hardware 80 in a power-saving hibernating state if the pen is motionless. In such a state, the image acquisition rate may also be reduced. Alternatively or additionally, pen 20 includes a movements sensor, so that image acquisition may be stopped. Alternatively or additionally, the pen includes an on/off switch. Alternatively or additionally, the pen includes a "hibernate switch". Alternatively or additionally, the pen includes a timer which reduces the electrical activity in the pen after a certain time has elapsed since a pen movement and/or a control activation. Other power saving techniques may also be applied.

Fig. 8A and 8B are block diagrams of a wireless pen input device in accordance with a prefened embodiment of the invention. Fig. 8 A is a block diagram of a dedicated device 80 suitable for a wireless pen. Special note should be take of the connector for pen charging and the modem transceiver. The transmission may be optical, IR, RF, ultrasonic and/or other short- range transmission techniques known in the art. Fig. 8B is a block diagram of the pen itself. Special note should be take of the battery which may be required unless power is beamed to the pen, for example using micro-waves or magnetic field coupling. Preferably, power saving techniques, as described herein are also utilized in this embodiment.

Various types of hardware interactions between the pen and computer 82 are possible. Some limited examples follow: on a simplest level, pen 20 may be connected (optionally via dedicated hardware 80) to computer 82 instead of a keyboard and/or a mouse. Alternatively or additionally, pen 20 may be connected to a serial port, preferably a USB (universal serial bus), to a parallel port, to a SCSI connector, to a Fire Wire port (1394 port) and/or as a network device. Additionally, dedicated hardware 80 may be implemented as a card in the host computer and the pen may be connected thereto.

Various types of software interaction may be used. Some limited examples follow: on one end of a spectrum is the pen generating hardware signals of a mouse and/or keyboard, possibly emulating a standard device, such as a Microsoft® mouse. On the other end of the spectrum is allowing pen 20 to interact only with a dedicated program. In between possibilities include a driver for the pen replacing a keyboard and/or mouse driver and the driver for the pen providing input to an operating system and/or to the drivers, through a system provided data entry hook.

It will be appreciated that the above described methods and apparatus of pen input may be varied in many ways, including, changing the order of steps, which steps are performed online and which steps are performed off-line. In addition various distributed and/or centralized configurations may be used to implement the above invention, preferably utilizing a variety of software tools. In addition, a multiplicity of various features, both of methods and of devices have been described. It should be appreciated that different features may be combined in different ways. In particular, not all the features shown above in a particular embodiment are necessary in every similar prefened embodiment of the invention. Further, combinations of the above features are also considered to be within the scope of some prefened embodiments of the invention. Also within the scope of the invention are computer readable media on which software, for performing part or all of a prefened embodiment of the invention, are written. It should also be appreciated that many of the embodiments are described only as methods or only as apparatus. The scope of the invention also covers hardware and/or software adapted and/or designed and/or programmed to carry out the method type embodiments. In addition, the scope of the invention includes methods of using, constructing, calibrating and or maintaining the apparatus described herein.

It will be appreciated by a person skilled in the art that the present invention is not limited by what has thus far been described. Rather, the scope of the present invention is limited only by the following claims. As used in the following claims, the words "comprises", "comprising," "includes", "including" or their conjugates shall mean "including but not necessarily limited to".

Claims

1. A method for determining a motion of a tip on a surface having a pattern of distinct features marked thereon, comprising: imaging the pattern at least in the vicinity of the tip, using a two-dimensional camera coupled to the tip; comparing successive images acquired by the camera; and analyzing the images to determine the motion of the tip.

2. A method according to claim 1, comprising forming at least a portion of said pattern by ink released from said tip.

3. A method according to claim 1, wherein comparing comprises conelating at least parts of the images.

4. A method according to claim 1, wherein comparing comprises feature extraction and matching of said images.

5. A method according to claim 1, comprising providing said pattern on the surface prior to the imaging, wherein said pattern includes a coded pattern for absolute positioning of the tip.

6. A method according to claim 5, wherein the coded pattern comprises a plurality of areas, each with a different background pattern and wherein said camera has a field of view larger than an area of a single pattern.

7. A method according to claim 6, wherein the plurality of areas comprises a plurality of square bar codes that form a checkerboard.

8. A method according to claim 5, wherein the coded pattern is a coded grid, wherein each line of the grid is a specific combination of lines of varied thickness and spaces of different widths.

9. A method according to claim 8, wherein the coded grid has a likeness of a woven lattice.

10. A method according to claim 1, wherein at least part of said pattern is invisible to a human eye.

11. A method according to claim 10, wherein said pattern is reflective in the infrared spectrum.

12. A method according to claim 1, wherein said pattern comprises a fluorescent material.

13. A method according to claim 1, wherein said surface comprises a transparency having said pattern printed thereon.

14. A method according to claim 1, wherein said surface comprises a plain paper with a pattern printed thereon.

15. A method according to claim 1, wherein said pattern comprises a grain of the surface.

16. A method according to claim 1, wherein determining the motion of the tip includes determining relative translation.

17. A method according to claim 1, wherein determining the motion of the tip includes determining relative orientations.

18. A method according to claim 1, wherein determining the motion of the tip includes determining an absolute position.

19. A method according to claim 16, wherein said position comprises a one-dimensional position.

20. A method according to claim 16, wherein said position comprises a two-dimensional position.

21. A method according to claim 1, wherein determining the motion of the tip includes determining an absolute orientation.

22. A method according to claim 17, wherein said orientation comprises a rotation angle of said tip around an axis perpendicular to the surface.

23. A method according to claim 17, wherein said orientation comprises an angle between said tip and said surface.

24. A method according to any of claims 1-23, wherein determining the motion comprises determining a velocity of the motion.

25. A method according to any of claims 1-23, wherein determining the motion comprises determining a rate of change of the velocity of the motion.

26. A method according to any of claims 1-23 comprising utilizing said detected motions to recognize handwriting.

27. A method according to claim 26, wherein handwriting recognition comprises handwriting recognition utilizing motion vectors.

28. A method according to claim 26, wherein handwriting recognition comprises utilizing both motion vectors and images of the surface for handwriting recognition.

29. A method according to claim 26, wherein handwriting recognition comprises: generating an image utilizing said determined motion; and recognizing at least one symbol in said generated image.

30. A method according to claim 26, wherein handwriting recognition comprises utilizing a user-dependent database to perform said handwriting recognition.

31. A method according to any of claims 1-23, comprising utilizing said determined motions for graphical input.

32. A method according to any of claims 1-23, comprising utilizing said determined motion for controlling a cursor.

33. A method according to claim 32, wherein controlling a cursor comprises emulating a mouse.

34. A method according to any of claims 1-23, comprising reconstructing a scanned image from said acquired images, utilizing said determined motions.

35. A method according to claim 34, comprising analyzing said images to recognize special codes.

36. A method according to any of claims 1-23, wherein said tip forms a part of a pen input device, and comprising: recognizing a particular pattem in at least one of said acquired images; and modifying an operational mode of said pen input device responsive to said recognized particular pattern.

37. A method according to any of claims 1-23, wherein said tip forms a part of a pen input device, and comprising: receiving a voice input by said pen; and modifying an operational mode of said pen input device responsive to said voice input.

38. A method according to any of claims 1-23, wherein said tip forms a part of a pen input device, and comprising: recognizing an entry of a symbol utilizing said determined motions; and modifying an operational mode of said pen input device responsive to said recognized symbol.

39. A method according to any of claims 1-23, wherein said tip forms a part of a pen input device, and comprising: recognizing said determined motions as gestures; and modifying an operational mode of said pen input device responsive to said recognized gestures.

40. A method according to any of claims 1-23, wherein said tip forms a part of a pen input device, and comprising: detecting when the tip in contact with said writing surface; and modifying an operational mode of said pen input device responsive to said detection.

41. A method according to claim 40, wherein said modifying comprises detecting motion of the tip responsive to said detected contact.

42. A method according to claim 40, wherein said modifying comprises operating said camera responsive to said detected contact.

43. A method according to claim 36, wherein modifying comprises selectively operating said pen input device in at least one of the following input modes: handwriting input, graphical input and pointer emulation.

44. A method according to claim 43, wherein the input modes include a hypertext link selection mode, in which the pen may be selectively operated.

45. A method according to claim 40, wherein modifying comprises controlling said camera.

46. A method according to claim 36, wherein modifying comprises controlling the release of ink.

47. A method of controlling ink flow in a pen, comprising: determining when said pen is in contact with a surface whose marking is allowed; and automatically electronically controlling a flow of ink, responsive to said determination.

48. A method according to claim 47, wherein determining comprises determining if said pen is operating in a pointer emulation mode.

49. A pen input device comprising: a tip; a two dimensional camera, coupled to said tip, which images a surface at the vicinity of the tip; and circuitry for determining motion of said tip relative to said surface, by analysis of successively acquired images.

50. A pen input device according to claim 49 comprising an electrical wire connection to a computer.

51. A pen input device according to claim 49 comprising a wireless connection to a computer for data generated in the pen.

52. A pen input device according to claim 50, wherein at least some of said circuitry is comprises in a dedicated processing unit.

53. A pen input device according to claim 50, wherein at least some of said circuitry comprises software operable with on said computer.

54. A pen input device according to any of claims 49-53, wherein said pen comprises a memory.

55. A pen input device according to claim 54, wherein said memory stores at least portions of said images.

56. A pen input device according to claim 54, wherein said memory stores data responsive to said movement determination.

57. A pen input device according to any of claim 49-53, wherein said pen comprises an internal power source.

58. A pen input device according to any of claims 49-53, comprising a light source, coupled to the pen input device, to illuminate the surface.

59. A pen input device according to any of claims 49-53 comprising a pressure transducer which determines if the tip is in contact with the surface.

60. A pen input device according to any of claims 49-53, wherein the pen input device is selectively operable in a plurality of input modes, including at least handwriting recognition and pointer emulation.

61. A pen input device according to claim 60, wherein the input modes include pattern recognition.

62. A pen input device according to claim 60, wherein the input modes include hypertext link selection.

63. A pen input device according to claim 60, comprising an external software, wherein said mode is determined by the software.

64. A pen input device according to claim 63, wherein said software controls the pen to operate in said different modes.

65. A pen input device according to claim 63, wherein said software analyses signals from the pen, responsive to said mode.

66. A pen input device according to claim 63, comprising a personal computer on which said software is executed.

67. A pen input device according to any of claims 49-53, comprising at least one switch for controlling the operation of the pen.

68. A pen input device according to any of claims 49-53, wherein said tip comprises an ink dispenser.

69. A pen input device according to claim 68, comprising an ink flow control which selectively allows ink to be dispensed from said tip onto said surface.

70. A pen input device according to any of claims 49-53, wherein said circuitry comprises a memory for reconstructing virtual pen marking from said determined movements.

71. A pen input device according to any of claims 49-53, wherein said circuitry recognizes handwriting patterns based on said determined movements.

72. A pen input device according to any of claims 49-53, wherein said circuitry determine a three-dimensional position of said tip relative to said surface.

73. A pen input device according to any of claims 49-53, wherein said circuitry determines a three-dimensional orientation of said tip relative to said surface.

74. A pen input device according to any of claim 49-53, wherein said pen comprises a rechargeable power source.

75. An implement mountable camera, comprising: a camera having a field of view; and a bracket for attaching said camera to a writing implement, such that said field of view images at least a vicinity of a tip of said elongate writing implement.

76. A camera according to claim 75, comprising circuitry for analyzing images acquired by said camera to determine motion of said tip.

77. A controllable pen having a pen tip, comprising: an ink dispenser; and an ink flow control which selectively blocks ink from being dispensed from said ink dispenser.

78. A pen according to claim 77, wherein said ink flow controller comprises an electrical flow controller.

79. A pen input device comprising: a tip; a two dimensional camera coupled to said tip and which images a surface at the vicinity of the tip; and a rechargeable power source for powering said camera.

80. A pen input device according to claim 79, comprising a connector to a computer, wherein said connector is adapted to hold said pen while not is use and wherein said connector comprises a data communication adapter for said computer and a power adapter to recharge said rechargeable power source.