WO1999003009A2

WO1999003009A2 - Method for controlling microscopes and apparatus connected thereto by direction of view analysis using an apparatus for measuring direction of view and a suitable eyepiece attachment

Info

Publication number: WO1999003009A2
Application number: PCT/DE1998/002010
Authority: WO
Inventors: Klaus SCHÖNENBERGER; Klaus Rink; Guy Delacreetaz; Winfried Teiwes; Kevin Strommer
Original assignee: Sensomotoric Instruments Gmbh; STROMMER, Latefa
Priority date: 1997-07-13
Filing date: 1998-07-13
Publication date: 1999-01-21
Also published as: DE19731301A1; WO1999003009A3; AU9252698A; DE19731301C2; EP0995142A2

Abstract

The invention relates to a method for controlling microscopes and apparatus connected thereto by analysis of the direction of view. It further relates to an apparatus for measuring the direction of view and an eyepiece attachment for carrying out said method. The aim of the invention is to provide a method, an apparatus for measuring the direction of view and an attachment of the type described above which make it possible to control the microscope and any apparatus operating in the working area of said microscope exclusively by means of the optical data of the observer's eyes without resulting in disruptive reflexes or incorrect handling of the microscope. To this end, a highly diffuse light with a wavelength of between 800 and 1000 nm is used as infrared light which is generated coaxially around the eye of the observer at the level of the eyepiece tube. In addition, in the beam path of the optical system at least one other infrared signal is generated which is independent of the site of production of the diffuse light and directed at the eye of the observer, which infrared signal has an intensity different from that of the diffuse light and a wavelength of between 800 and 1000 nm and is subjected to the diffuse light so as to produce at least one additional corneal reflex.

Description

Methods for controlling microscopes and connected devices using line-of-sight analysis,

Direction of view measuring device and eyepiece attachment therefor

The invention relates to a method for controlling microscopes with coupled devices by means of line-of-sight analysis, in which the viewer's eye is illuminated with infrared light, imaged by an optical system, recorded by an image sensor and the image thus obtained is then determined in a processor for determining the gaze position the middle of the pupil and corneal reflexes are processed.

The invention further relates to a direction-of-view measuring device for controlling microscopes and devices coupled thereto, having infrared diodes illuminating the eye of the viewer, an image sensor imaging the eye, a processor calculating the direction of view of the viewer and a monitor indicating the pupil center and the corneal reflexes. The invention also relates to an attachment for the eyepiece tube of a microscope, for example a surgical microscope, by means of line-of-sight analyzes with at least one eyepiece system for visual observation of the operation field imaged by the object lens by the viewer, an illumination source illuminating the eye of the viewer and at least one eyepiece lens arranged in the eyepiece tube.

From DE 28 22 277 C2 a binocular microscope for medical purposes is known which is particularly suitable for use when sewing a cut made in the cornea, in particular for restoring the spherical curvature of the cornea of an eye which has an input lens which is at a fixed distance can be brought into position by the eye, an annular light source being provided near the input lens, by means of which an original image is projected onto the cornea. Additional imaging optics are partially arranged in the beam path of the microscope with at least one prism or a lens or a mirror, so that part of the light is not deflected by the additional imaging optics and a reference image is generated. By means of the additional imaging optics, at least one further comparison image is generated, which is essentially consistent with the reference image, but offset from this. The microscope has an adjustment device by means of which the position and / or size of the reference image and at least one comparison image can be adjusted. The ring-shaped light source described in this publication is assigned to the objective lens and not to the eyepiece and serves to illuminate the patient's eye and not the viewer.

Also known is an operating microscope with an objective of variable focal length and a housing which is fastened to a carrier and which contains means for beam splitting and deflection and two observation tubes (DE 31 05 018 AI), in which a mirror or a prism is used as the beam deflecting means. Furthermore, from DE 36 23 613 C2

Illumination device for an operating microscope with an illumination system is known, which is arranged outside the optical axis of the microscope objective and emits the illuminating light perpendicular to the optical axis of the microscope objective. The illuminating light originating from the illumination system and running perpendicular to the optical axis of the microscope objective is reflected with a plane beam splitter and the light coming from the object and extending to the microscope objective is transmitted.

The beam splitter is arranged centrally to the optical axis of the microscope objective on its side facing away from the object and is inclined relative to the optical axis of the microscope objective by an angle which ensures that the illuminating light is mirrored in a manner which is strictly coaxial with the observation beam path.

According to DE 41 34 481 AI is a surgical microscope and a method for computer-aided Stereotactic microsurgery is known, in which a device for reflecting intermediate images in at least one of the two stereo observation beam paths, detectors for detecting the optical system data, a position detection system and a process control device for evaluating the signals of the process detection system is provided.

This known surgical microscope and method allows manipulation in all six degrees of freedom in relation to the surgical field, but the data for position detection and detection are obtained from the position of the objective lenses and the zoom on the patient side, so that the position and orientation of the patient in the Space is determinable. As before, the surgeon must control the setting mechanically. In addition, the space on the

Lens side, i.e. relatively limited to the patient side.

Furthermore, from EP 0 596 868 A2 a method for determining the direction of view is known which consists of the steps of determining a position and a direction of the head from a face image, determining a feature point of an eye and calculating the eye position in accordance with the determined position and direction of the head and the feature point of the eye.

The eye is illuminated with infrared light, which is emitted by diodes arranged in a ring around the lens of the camera. EP 0 596 749 A1 also describes an ophthalmological device which contains irradiation means for irradiating the eye to be examined with infrared light, image recording means for recording the eye to be examined, comparison and storage means for comparing the image information obtained from the image recording means with one threshold value determined for the progressive image information, calculation means for calculating the values of the eye to be examined on the basis of the image information which is stored in the comparison and storage means.

US Pat. No. 5,231,674 also discloses a method and an apparatus for determining the direction of view, in which an optical, i.e. analog image is recorded, which is analyzed in an image processing system in order to obtain information about the point of view and / or the direction of the eye. The IR light source that illuminates the eye to be examined is located in the optical axis of the lens system of the camera.

According to EP 0 350 967, an image recording device and a method for determining the viewing direction are also known, in which the eye of the viewer is illuminated by infrared light, imaged by an optical system, from one

Image sensor recorded and the image thus obtained is then processed in a processor for determining the gaze position by determining the pupil center and corneal reflexes. All these known solutions have the disadvantage in common that on the one hand the position control data are always obtained from the optical system data on the lens side (DE 41 34 481) and on the other hand the eye image is often disturbed by shadows, for example by eyelashes. This means that the position of the lighting diodes has to be readjusted frequently, especially when working on one and the same examination object at different times.

DE 43 37 098 AI describes a visual axis determination device which, using an reflected eye image generated by illuminating the eye of a photographer with infrared light, determines an axis pointing in the direction of an observation point of the viewer or a visual axis when the viewer looks at a viewing plane on the means of an object image is generated in a recording device contained in an optical device such as a camera. In this known solution, the eye is illuminated by two infrared light-emitting diodes, which are arranged on the optical axes of the projection lenses, the light-emitting diodes being symmetrical to an optical axis.

This state of the art is on the one hand away from microscopy and on the other hand there is no indication of the teaching according to the invention. The lighting for controlling microscope functions and operating devices connected to the microscope is not suggested. From WO 96/13743 there is also a microscope for a user, in particular a surgical microscope with at least one tube, at least one operating element and at least one controller for at least one remote-controlled actuator and with a sensor for detecting the eye or pupil position of an observer's eye for the controller the actuator known. The actuator includes an auto zoom and / or other peripheral devices or devices connected to devices for changing the position of the microscope. In the area of the tube, operating symbols inside the tube are visibly assigned to the user. The eye is illuminated by a single IR LED, which images the pupil of the observer's eye on a CCD.

Due to the IR illumination of the eye, the pupil of the human eye appears as a coherent gray-black surface. However, shadow zones can also be caused by lighting, for example when the light rays are not only reflected in one direction on the corneal surface or also from other objects that are close to the eye. If these objects are opposite the eye, as described, for example, in WO 96/13743, they are reflected and appear on the eye image. The corneal surface of the eye is a mirror that reflects about 2% of the incident light. Therefore, every object positioned in front of the eye has its reflex on the cornea. Every reflex or shadow zone, the gray tone of which approximates the gray tone of the pupil, leads to falsifications or errors in the determination the center of the pupil in the previously described methods for gaze direction analysis. This represents a considerable risk of incorrect operation of the known microscope.

Knowing the disadvantages of this state of the art, the object of the invention is to provide a method, a direction-of-view measuring device and an attachment of the type mentioned at the outset, which allows the microscope and devices operating in the working area of the microscope to be used without disruptive action To control reflexes and operating errors of the microscope only through the optical data of the observer's eye.

This is achieved with the method of the type mentioned in the present invention in that a strongly diffuse light with a wavelength of 800 to 1000 nm is used as infrared light, which is generated coaxially around the eye of the viewer on the eyepiece tube, and in the beam path of the optical system At least one further infrared light signal, which is independent of the point of generation of the diffuse light and is directed at the viewer's eye, is generated with a different intensity and a wavelength of 800 to 1000 nm compared to the diffuse light, which is used to generate at least one additional corneal reflex for the diffuse light.

According to a preferred embodiment of the method according to the invention, the analog eye image detected by the image sensor is converted by analog / digital Implementation divided into different shades of gray (grayscale image generation), or broken down into events by an edge detector logic, wherein each video line is searched for dark-light or light-dark transitions, and that the information obtained is incorporated into an event table that the Processor for determining the coordinates of the pupil center, more corneal

Reflections and from there the direction of the eye.

In a further preferred embodiment of the method according to the invention, the visible light is separated from the infrared light by a wavelength-dependent beam splitter, preferably a half mirror with an anti-reflection coating, a prism, or a polarizing beam splitter, which reflects more than 80% of the infrared light used to illuminate the observer's eye and for more than 80% of the visible light with a wavelength below 800 nm is transparent.

The illumination of the eye and the generation of the corneal reflexes take place separately in the method according to the invention, whereby a very good image quality of the analog eye image with sharp dark-light transitions without disturbing reflections is achieved. This is the basis for the fact that the analog eye image recorded by the image sensor can be flawlessly scanned at high image frequencies up to 250 Hz and can be broken down into digital sequences of events that precisely determine the coordinates of the pupil center, the corneal reflexes and thus necessary for the control enable the line of sight. In a further essential feature of the method according to the invention, in addition to the first corneal reflex at a distance d, a second corneal reflex is generated with a further infrared light crucible from the former, which is used for auto-focusing the eye image of the viewer. The grayscale images obtained are passed through a video memory, in which the images with a

Histogram function can be analyzed. The individual gray levels are determined over all video lines and compiled in the form of a frequency distribution, which enables the threshold values for the detection of the pupil and corneal reflexes to be determined automatically.

The user's gaze movements are calibrated to the work area by using a support with marking points in the object plane or an image reflected in the intermediate image plane of the beam path, which are converted to the physical sizes of the work area and then stored temporarily, so that the microscope magnification changes each time the calibration data is automatically adapted, which always guarantees an optimal calibration without manual readjustments.

Suitable devices or systems for coupling to the microscope are surgical instruments, lasers, inspection systems for wafers, software programs, image-bound systems or cameras.

The object is further achieved by a line of sight measuring device which consists of a video multiplexer, on the inputs of which a number of video signals are applied, to which image sensors are assigned, and the output of which is either directly via a oo tt

Oh o o on o

Hi tö Φ N d = Φ μ- d ri M P ri φ Φ

P. d o μ- Ω ö fr

Φ tr μ- d rt Hi M

5_! d Hi PJ

PJ P d ri ω LQ CQ M

Φ CQ 0 μ- ri P p. µ- CQ CQ

Φ P Ω Φ D LQ tr

Φ C- tu J μ- μ-

Φ P tr Ω rt LQ φ tr ri Φ rt

PJ 0 PJ

Ω ri er ⁾ tr p. LQ P rt P Φ Φ

Φ φ μ- ri rt Φ P

CD Ω PJ D for P

PJ μ- rt d P Φ Φ

Hl P- ri

C M

Φ P J μ- - Hi P

CQ Φ LQ rt H PJ Φ

~ ri 0

Φ 0 ri d μ- rt P *

P P P

Φ μ- φ

0 rt

> Φ

PJ d Φ - tr- CQ P

CQ

Ό N d J d ri ri Ω d Φ tr

P μ-

LQ P φ

3

o oo to to | _- on o on o H o Ut α LQ HH ^l O ö H tr ≥! rrj PJ σ NJ> f PJ φ P. ri J. tö tö LQ JJ 3 h μ- φ P PJ = ^ μ- ri Φ J od μ- d P d μ- PJ P μ- d φ d φ φ Φ PP μ- Φ

Φ 3 P d Φ P ⁾ CD rt M CQ Φ LQ LQ LQ Ω Ω LQ P ri Hl CD LQ rt ri LQ LQ P Ω

CQ P ⁾ = s- LQ MP rt d = ^ φ Φ φ tr tr Φ Φ Ω M rt i ri μ- Φ Φ C- tr tΛ Φ CQ PJ H CD Φ ri 0 rt α 0 0 P rt LQ 0 3 tr Φ μ- Φ PJ Ω 0 0 φ rt tr Φ μ- J ri P 3 tr MX tr μ- ri ri tr Φ ri M ^ Φ P Ω tr r ri CQ ≤

PJ P rt Ω J Hl μ- φ μ- ^ Φ Hi & α μ- φ 0 P. ≤ PJ: rt ri N tr rt rt μ- rt Φ tr Ω ri CQ P Ω 3 ri Hi PPM 3 ri P μ- CD μ- er d rt Φ PP φ P

> ri CD tr PJ CQ tr Φ 3 d φ φ • Ö Φ P CD Φ PJ P Φ rt Φ Φ P

C- d Φ Φ CQ ri μ- rt 0 co rt rt φ Hi P rt μ- i h LQ ri rt rt CD M

Φ Hi ri P Φ 0 0 P. F? P ^J Ό 0 CQ μ- φ φ LQ φ CQ μ- Φ μ-

P CD rt P PJ J ^ M ri μ- μ- P rt μ- M Φ P μ- CD μ- φ LQ

P ⁾ er for LQ C- CQ PMP ⁾ CQ CD CD d P. Φ CD i P. CQ Φ M rt O CQ μ-

<rt φ 0 rt μ- PJ P Φ CQ Ω rt rt P M P rt <l φ rt ri Φ - • rt P N

0 ESI CQ J Φ 0 d P. P rt tr • - P. μ- - 0 ^ ri - P fö - φ d μ Φ 0 X ri Hi μ- P. μ- Φ Ω P Φ -

Φ € φ μ- 3 D μ- P- P. tr s ö fr P. • 3 H rt CQ P μ- P d M d μ- Φ μ- 1 C

Φ p. J Hi P Φ Φ μ- Ω tr Φ φ φ er P (^ M Hi P r- P Φ r Φ H 0 μ- tr Φ M o = μ- td rt tr φ 3 ri CQ μ- a. LΠ Φ d = - Hl o ri d Ό

M PJ ri ri CQ LO μ- ö Φ M 0 P ri d C- P Φ er rt rt φ ESI 3 μ- rt LQ μ- tr 3 tr φ CQ P. - M CD Ξ J P. P d μ- ri d μ- P. φ Hi 0 Φ J ri φ d PJ tsi PJ = CQ CD. P- ri LQ P. P Hl t «CQ CD tsi PP CΛ d P> 0 0 CD φ CD μ- tΛ ri i P er PH Ω J Φ CQ d φ P rt φ CQ LQ LQ Ω CQ Φ H CD fö Φ tr tΛ ri d OJ μ- Ω CQ 3 P φ td 0 CQ μ- φ tr Φ Ω μ- fö 0 1 μ- Φ

CQ for P P tr 0 CQ tr M LQ ri φ P 0 P rt μ- tr 1 ri f P P

P- H LQ d LQ PJ ri rt rt μ- rt CD σ PJ tr P Φ C- tr ¹ M μ- φ μ- d Φ M φ Φ Hi CQ ri CQ Ω φ ^ Φ Φ rt er PJ μ- PJ μ - μ- Ω>

CD er CD PJ 0 μ- rt Ω μ- rt P fr P CD P μ- μ- tr ri α er CQ Ω P t ≤ Ω d rt ri ri P tr CQ 0 PJ rt Φ P CQ PJ = φ μ- Φ tr CD rt Φ d CQ PJ PJ P. φ Φ Φ rt Hi Ω μ- Φ φ Ω tr t μ- CD

3 M Ω P i μ- μ- PJ Hi tr Ω 0 3 3 ü tr P CQ φ rO rt φ

LQ CQ P> ri J rt Φ

• ö Φ rt tr φ P er M tr ri μ- φ μ- α P tr * φ P μ- PJ d d Φ

Hi μ- d CQ rt i Φ Φ M Φ rt P Hι P LQ μ- co CQ CQ d P Hl ri μ- P P φ M M μ- tr μ- d φ d = P. Φ Φ Ω Φ Hi P- Φ φ

P φ LQ J = PJ CQ Φ P P PJ ri Φ ri> 3 d P ri

P Ω tr d PJ μ- φ LQ M CD Ω tr M 0 PJ φ CQ p. PJ rt tr P ri P. φ μ- H

M <! P. d P. tr O Ω rt CQ 3 CD φ rt tsi CO Hi & P J μ- P P o μ- J Φ CQ Φ tr tr μ- Ό 3 P μ- Φ $. CQ rt OJ. ü P PJ CQ P φ St.

Ω ri CQ LQ 3 Φ 0 μ- £. PJ = φ P μ- Φ Φ J φ tr ri d tr μ- Φ μ- CD PJP Φ Φ VÄ fö CQ CD J P. ri rt Φ rr μ- JM for PJ Φ ri rt φ d CD M μ- LQ φ td Ω μ- tr o 0 d ri 3 3 li PJ

Μ er μ- P P. CD LQ CQ rt Φ lh i Φ tr M Ω rt d p. LQ μ- 0 μ. μ- M Hl Ω μ- ^• ^ μ- ri = ^> Φ ri M Hl μ- φ rt tr M co rt rt rt Φ μ- tr tr i Hi Φ PJ Φ ri fjj: φ PJ POP Φ Φ

P φ μ- for P. ri 3 rt P. d

P rt ri Φ Hi μ- P. μ- Φ rt X CQ φ μ- tr Ω d P. P ri μ- er p. f P- Φ Φ Ω d P • co P φ CQ PJ tr M 0 φ - 'tr μ- P> 0 d

Φ PJ = d rt P tr CD φ φ d d φ 1 Φ Ω rt P CD tr P. CQ

CQ P P rt Φ 3 2 P Hi P J ri tr M Φ LQ LQ P Φ LQ φ ri P φ μ- Φ CD Ω rt φ μ- i d P P

• 1 d M for P. f tr P φ LQ

H PJ 3 Φ 1 d P PJ

CQ J CQ P

M CQ LQ

System is adjustable on head movements. The longer the tube length, the less sensitive it is

View direction detection from the viewer's head movement.

In a further preferred embodiment of the attachment according to the invention, a further tube is provided at right angles on the eyepiece tube, in which a further infrared diode for generating the second corneal reflex is arranged, which can be used to automatically focus the user's eye image on the image sensor.

The half mirror has an anti-reflective coating made of light metal fluorides, which ensures high transmission for visible light.

The method according to the invention, the direction-of-view measuring device according to the invention and the attachment according to the invention enable the observer's eye to be illuminated uniformly without reflection and shadow. As a result, optimal contrast formation between the pupil and iris of the viewer, which is insensitive to ambient light, is achieved, so that the center of the pupil and the position of the corneal reflexes can be determined exactly. It also has the advantage that the viewing direction of the viewer determines and automates the readjustment of devices such as lasers, endoscopes, inspection systems for wafers, software programs, image-bound systems, cameras and the like. is made possible. The article according to the invention is equally suitable for binocular or monocular viewing direction analysis. The attachment according to the invention can be placed on all common microscope types by means of an adapter.

From this, the advantage of the high flexibility and variability of the method and the article according to the invention when performing medical operations or technological processes for microchip production can be seen.

Further advantages and details emerge from the following description of a preferred exemplary embodiment with reference to the attached drawings.

The individual shows:

1 is a sectional side view of the attachment according to the invention,

2 shows a sectional illustration in top view of the lighting system on the attachment according to the invention,

3 is a sectional side view of the lighting system on the attachment according to the invention,

Fig. 4 is a diagram of the mapping of the analog image through the lens system of the invention

Essay, Fig. 5 is a schematic representation of the calculation of the influence of head movements

User on the gaze direction measurement,

6 shows a family of characteristic curves for the sensitivity of the head movements as a function of the position of the separate infrared diode,

7 shows a representation of the analogous eye image according to the inventive method with overlays of crosshairs for the determined pupil center and a corneal reflex,

8 shows a basic representation for the focusing of the surgeon's eye by means of autofocus when using two corneal reflexes,

9 is a block diagram of the electronics of the method according to the invention,

10 shows an illustration of an application of the method according to the invention in laser surgery.

The eyepiece tube 1 shown in FIG. 1 of the attachment 2 according to the invention is locked by an adapter 3 with the receptacle of a commercially available surgical microscope 57. In the eyepiece tube 1 there is a half mirror 5 which contains more than 80% of the infrared light (wavelength of

d Q n_ ö PJ tö er P- φ ≥! > σ tö td td pr = φ tö er tu n PJ p ri Φ J d Φ ri φ μ- μ- ⁾ P μ- φ μ- μ- ri μ- μ- Φ PJ tr P

PJ CQ CQ CQ M Φ ri 0 P Ω Ό φ ri P P φ Φ P M <M H- J d rt Φ μ- P. Φ tr J Φ Hi μ- μ- Φ P. 0 tr ri M p. rt CD CQ rr d rt o φ ri CD TJ μ- JH CQ tr Hi CO ri CQ d 0 J 0 = rt T 0 Ω Φ p P = co 0 Ω MP φ Φ Φ Ό ri LQ tr P Φ M tr P d LQ μ- d CQ tr M Hi M μ- tr P Φ μ- LQ φ

Φ Φ M rt ^ rt MH ri iQ P μ- Φ Φ ri Hi rt μ- LQ CQ φ Φ CQ ri • «M Φ 0 d tö PJ H 0 = d LQ rt CQ PJ 0 = CQ Φ 0 LQ P d ri XP φ ri tr- ri μ- ri H LQ ri ri <φ tö p. P- P LQ MM r Φ 0 J <0 3 ri 0 M s: μ- μ- μ- LQ P g CD Φ μ- Φ ri to Φ P d 0 rt μ- JP l- ¹ 3 μ- M

Φ Φ 0 Φ ri d P M Φ ri Hi P LQ μ- o cπ ri P.

P. Ω rt μ- Ω CQ Φ P d P- ü φ Φ Ω O

CQ P Φ tr tr P tr φ d P α Φ P. tr ¹ d P tr PJ er μ- lO μ- PJ ri * <LQ rt Ω P. μ- H Φ μ- ri t rt he P P. o

Ω tr P M d cn tr P 0 P Ω Ω φ tr LQ φ d

^ Φ tö PJ = φ MP rt tsi OO tr tr ≤ Φ P. Φ fr φ ri φ M tr P to LQ Φ Φ PJ rt ^ tö rt LQ Φ Hi J σ rt μ- Ω φ ri P. μ- φ CQ P rt - Ξ ri d μ- PJ LQ Φ ri μ- μ- P tr CQ φ Φ Ω ri J μ- ri LQ M μ- μ- MM P. P d CQ LQ M <! φ

CD er CQ μ- - ri φ PJ C- φ LQ P rt Φ PP> t ri tsi LQ rt PJ & P. P ri co CQ CQ LQ μ- CQ φ C- ti μ- d φ o μ- d Φ LQ P μ- er NM Φ Hi Φ LQ rt rt Φ s; φ LQ

CQ ri Ω p. 3 Φ Φ Φ P. J μ- P d μ- P rt φ ri μ- Φ rt PJ tr Φ Φ μ- H μ- μ- PJ tr P CD M M ~ M s; ω f CQ μ- d rt Ω rt t P CQ CQ M CQ 0 rt rt N M μ- <Ω μ-

3 rt d M for J 0 T Φ ri ri Φ d rt ri μ- tr P ^ J

3 0 P fjj: rt Φ tr P PJ μ- P- PJ ri M μ- φ P. Φ CQ d P LQ d • μ- o P. X φ d Φ σ «H <! P ⁾ = φ P • φ P φ α

P φ rt P tr φ μ- M LQ CD O μ- to CΛ 0 er P φ

LQ co Φ Φ Φ Φ ri PJ CD Φ μ- • 0 rt M tu ü for μ- CQ ri ri ri ri M 5! dead 3 sts μ- PJ μ- PJ M σ »

P * P. for P J Φ -J Φ Φ <! P M Φ 3 P. tö φ φ φ P. α ö P ESI μ- rt O rt φ Φ rt φ φ d Φ

CD ü P Φ μ- μ- LQ d CQ P. fr rt PJ Φ <ri er P rt

P Hi Ω φ ri Φ φ PJ d Φ tr ri H μ- PJ Φ ü

• ö rj rt <Hi tr 0 CQ Ω M P μ- φ d & P PJ d d Φ d rt ri 0 <! tr PJ P P φ O Φ j ^ Ω

O Ό CD ri CQ Φ Ό 00 tö rt ri d P- d LQ 0 tr μ- μ- Ω l-h 0 P rt Φ Φ rt P φ P for PJ tsi d = rt

M M tr PJ ri er φ μ- rt ri d P. ri & 3 J d σ Φ

M M PJ tr CQ μ- rt CQ H h CD tr rt μ- 3 Hl Φ ri φ φ rt ri Ω M Ω P J d CQ rt φ PJ CD

P rt Φ tr tr Hi Ω <1 CQ μ- P. ri P. er

3 M φ P φ Φ Φ Φ ri tr PJ Ω PJ LΠ J φ LQ & er μ- μ- P μ- P μ- P PJ rt ri H tr CD P Φ er φ N rt φ N er P i Φ μ- rt P s £ rt LQ J d Φ φ> 0 ri P ⁾ μ- σ n μ-

Φ Φ M ri Ü P Ω rt CD tr CD PJ μ- n M

MP LQ H Φ tr φ rt ri CD d C- ω CD 3 M φ rt Φ φ μ- • P rt

rt d PJ td ri P

point can disturb. The same applies to additional reflexes on the cornea, which interfere with the detection of the corneal reflexes.

Through the diffuser disc 13, the light emitted by the infrared diodes 11 is scattered and strongly diffuse, so that a uniform, shadow-free and reflex-free, symmetrical illumination of the eye of the viewer about the optical axis thereof is ensured and the aforementioned disturbances no longer occur.

The infrared diodes 11 emit infrared light in a narrow spectrum within the wavelength range from 800 to 1000 nm. Commercially available LEDs, for example LD 261 from Siemens with a wavelength of approximately 950 nm, are used as infrared diodes 11, so that a more detailed description of these infrared diodes can be omitted.

The diffuser disc 13 and the lighting ring 12 have a cutout 14 for the nose of the viewer and at least one further opening 18 for the passage of the infrared light for generating the corneal reflexes 25 and 28, which will be discussed in more detail in the further course of the exemplary embodiment.

A further tube 15, in which a separate individual infrared diode 16 is positioned, is located on the eyepiece tube 1 perpendicular to the optical axis B of the eyepiece 6. The infrared light from this diode 16 falls in the tube 15 of length 1 through a CR condenser lens 17 merges the beam path, and on a prism 19, which the infrared rays through the opening 18 in

Steer the lighting ring 12 and in the diffuser disc 13 onto the surgeon's eye. There, this infrared light generates the clearly defined corneal reflex required for the calculation method for determining the viewing direction.

The infrared diode 16 is also a commercially available LED (SFH 487 from Siemens) with a wavelength of approximately 880 nm.

The angular position of the prism 19 is adjustable and the length 1 of the tube 15 is variable. The length 1 of the tube 15 thus positions the separate infrared diode 16 and at the same time determines the sensitivity of the attachment according to the invention to head movements of the viewer.

A clearly corneal reflex 25 is generated by the infrared light of the individual infrared diode 16.

The formation of the analog image 20 is explained in FIG. 4 and the optical dimensioning of the article according to the invention is discussed.

The eye 7 of the viewer is at a distance l in front of the eyepiece lens 6 with the focal length f6 and an object size of yl. After passing through the distance k between the eyepiece lens 6 and the lens 4 with the focal length f4, an image size of y3 of the image of the eye 7 is produced at an image width w3 to the lens 4 in the intermediate image plane Z _E The outer edge of the eyepiece lens 6, with the diameter y6, imaged by the lens 4, determines the exit pupil of the lens combination with the focal lengths f6 and f4. The following applies to the distance w4 of the exit pupil from the lens 4 and the size y4 of the exit pupil:

fyk w ₄ (equation 1)

and

^ 4 = ^• y ₆ (equation 2) k -

The intermediate image in the intermediate image plane Z _E is imaged by the lens of the video camera 9 with a focal length f5 with an object width of w7 and an image width of w5 on the image sensor 10 in the imaging plane 8 with an image size of y5. Sizes 5 and y5 can be set individually for each video camera type by selecting the object width w7.

The calculation of the influence of the head movements of the microscope user is explained in more detail in FIG. 5.

The vector representation represents the eye 7 of the viewer, the separate individual infrared diode 16 and the attachment 2 according to the invention according to FIG. 1 in a three-dimensional space. The vector & ■ points to the coordinate of the infrared diode 16 in the earth-bound coordinate system O, and the vector o points from the origin of the earth-bound coordinate system to the origin O 'of the variable eye coordinate system.

By using the auxiliary vector a "from the origin of the eye coordinate system O 'to the coordinate of the

Infrared diode 16, the line of sight vector g can be expressed by the coordinates of the terrestrial system:

R

Gy- ^• (V - o _x ) (Equation 3)

2A - - RGG _γγ (Equation 4)

A denotes the distance between the origin of the eye coordinate system and the coordinate of the infrared diode 16, and R the radius of the curvature of the cornea.

The transition to the vector § enables the calculation of the viewing direction coordinates for any head movements if the eye 7 remains fixed on the center of the microscope slide.

The sensitivity of the coordinates of the viewing direction vector to the head movements and thus the shifts to the earthbound coordinate system O is given by the equations

dG _γ dO _v d0 ₇ (equation 5)

dG d0 ₇ (equation 6)

expressed

The partial derivatives of the line of sight vector therefore result in:

For the X coordinate:

A- (2A-R) -2- (S _x -O _x Y

= -R (Equation 7) X> _x A- (2A- -R) ²

(Equation 8)

dG _Λ (S _x

= 2R -o _x ) (S _z - -o _z ) (Equation 9)

A- (2A-RY and for the Y coordinate:

SG _γ (S _χ -O _χ ) - (S ~ Oy) dG,

-2R (Equation 10)

A- (2A-R) ² δO _v dG ^, A- (2A-R) -2- (S _γ -O _y ) ²

= -R (Equation 11) dO _y A- (2A-R) ²

6G _y (S _y -O _γ ) - (S _z -O _z )

-2R (Equation 12) A- (2A-R) ² The dependence of the head movements is shown by the family of curves according to FIG. 6. For long lengths S _z , which correspond to the length 1 of the tube 15, the sensitivity is significantly reduced.

The separate illumination for determining the center of the pupil and the corneal reflexes achieves an excellent image quality of the eye image 7, as shown in FIG.

Superimposed on the eye image 7 are the crosshairs 22 for the pupil center 24 and the crosshairs 23 for the corneal reflex 25 as overlays.

A second tube 26, like the tube 15, is arranged at right angles to the eyepiece tube 1. This contains a further infrared diode 27, which also directs its infrared light with a lens 17 and an adjustable prism 19 through the opening 18 in the diffuser disc 13 onto the cornea of the surgeon's eye 7 and generates a second corneal reflex 28 there, which is used for auto-focusing the Eye image 7 is used.

FIG. 8 shows the focus of the surgeon's eye 7 in the event that the two infrared diodes 16 and 27 are coplanar. Since the tubes 15 and 26 open into the eyepiece tube 1 in a vertical alignment, this condition is fulfilled. In addition to the first corneal reflex 25, there is a second corneal reflex 28, which are at a distance d from one another, which is a measure of the focus ΔS _{0 of} the video camera 9. For the coplanar level:

1 1 1 2

- + - = 7 = - (Equation 13) ύ, o ₀ JK and

(Equation 14]

This results in the auto focus function:

2aR

Ad = - ~ - r - ΔS ₀ (Equation 15),

(R + 2S ₀ ) where a is the distance between the two infrared diodes 16 and 27.

In Figure 9 is the block diagram of the invention

Blickmeßgeräts 46 shown.

A video multiplexer 29 with a maximum of three video inputs is implemented on a PC ISA bus-compatible plug-in card.

CCD image sensors 10 with a vertical sampling frequency of 50 Hz for the standard video refresh rate or with a vertical sampling frequency of 250 Hz for higher refresh rates are optionally available for these three video inputs.

The analog signals arrive on the one hand via an 8-bit analog / digital converter 30 as a grayscale image to the line of sight processor 32. The distribution of the Different shades of gray can be analyzed with a histogram function.

On the other hand, the analog video input signals, in parallel to the analog / digital conversion, are first broken down into events by edge detection logic 34, which events are characterized by black-and-white (dark-light) transitions or white-black (light-dark) transitions characterize. These transitions are stored and stored in the form of events for each video line separately for the pupil and the corneal reflex in an event table before they are fed to the direction-of-view processor 32. With the event table, it is possible to process significantly more frames per second than is necessary for standard video sampling frequencies. For this reason, the division of the images into events is used for maximum refresh rates. The video output 35 is for the eye 20 with

Overlays 62 provided. In addition, a video output 38 with a standard VGA is available. With the help of a RAMDAC 39, the digital / analog conversion of the signals is carried out for this output, with the possibility of also mixing false colors into the output signal. The area of the video image which is covered with an image overlay is stored in the video memory 41. The video input signal from the video camera 9 is looped through to the video inputs until a pixel arrives that contains overlay information. This is colored with a selectable overlay color.

Four 12-bit DAC ports 43 with a 100 kHz conversion rate enable the control of external functions, for example from switches via relays. A 12-bit ADC port 44 for eight channels in multiplex mode can be used to acquire further measured values.

The 16-bit ISA bus 48 establishes the connection to additional PC plug-in cards and the PC processor when used in a PC. An 8-bit digital input / output 60 and serial ports 61 are also provided for the stand-alone operation of the gaze measuring device 46.

Furthermore, two infrared diode power supplies (12 V, 200 mA) 45 are provided, which supply the required current for the infrared diodes 11, 16 and 27 when the eye 7 of the viewer is scanned by the video camera 9.

The expansion port 49 enables the connection of further function modules, for example additional logic and coprocessors.

The method according to the invention is explained below using the example of laser surgery according to FIG. 10. Instead of the right microscope tube, an attachment 2 according to the invention in the form of an eyepiece tube 1 with the adapter 3 is placed on the surgical microscope 57 and locked. A tube 15 for the video camera 9 is integrated at right angles in the beam path of this eyepiece tube 1.

The camera signal is fed to the line of sight measuring device 46 in accordance with FIG. 9. The PC-compatible slide-in board of the viewing direction measuring device 46 is located in the eye tracking computer 59. The output signals of the viewing direction measuring device 46 are sent to a control unit via a parallel or serial link 50 51 transmitted, which takes over the control of an operation laser 53 via a driver card 52.

Before the actual operation is started, the surgeon's eye movements on the operating field 54 must be calibrated. A slide 55 with a small number of defined points (5 points are sufficient) is used for this. While the surgeon is looking at these points, the output data of the line of sight measuring device 46 are related to the physical quantities of the operating field 54. The calibration data are buffered by the line of sight measuring device 46 in the eye tracking computer 59.

Each time the microscope magnification is changed, the system automatically adjusts the calibration data, which always ensures optimal calibration without manual readjustments.

The laser beam for the laser treatment is fed in under the object table 56 of the surgical microscope 57 and directed onto the working field 54 via an optical deflection system in the object table 56.

List of the reference symbols used

Eyepiece tube 1

Article 2

Adapter 3

Lens 4

Wavelength-dependent beam control 5 eyepiece lens 6

Eye of the beholder

Imaging plane

Video camera 9

Image sensor 10 infrared diodes 11

Illumination ring 12

Diffuser disc 13

Cutout for nose 14

Tube 15 Separate infrared diode 16

C.R. -Condenser lens 17

Opening 18

Prism 19

Analogous eye image 20 infrared pass filter 21

Target Crosshairs Pupil Center 22

Target crosshair corneal reflex 23

Pupi11enmi11e1punkt 24

First corneal reflex 25 tube 26

Separate infrared diode 27

Second corneal reflex 28

Video multiplexer 29 8-bit analog / digital converter 30

View direction processor 32

Edge detector logic 34 video output 35

Video inputs 36.37

Additional video output 38

RAMDAC with false color overlay 39

Memory module 40 video memory 41

12-bit DAC port 43

12-bit ADC port 44

Power supply for infrared diodes 45

Line of sight measuring device 46 16-bit ISA bus 48

Expansion port 49

Parallel or serial connection 50

Control unit 51

Driver card 52 operational laser 53

Working area 54

Slide 55

Object table 56

Surgical microscope 57 eye tracking computer 59

8-bit digital input / output 60

Serial ports 61 distance infrared diode 16 from the origin of the

Eye coordinate system A distance of the infrared diodes 16 and 27 a

Optical axis of the eyepiece B Distance between the corneal reflexes 25 and 28 d

Focal length of the lens 4 f4 Focal length of the lens of the video camera f5

Focal length eyepiece lens 6 f6 viewing direction vector ^~ ζ

Length of tube 15 1, S _z optical distance eyepiece lens 6 to

Lens 4 k

Vector ^" o ^" origin of the space coordinate system 0

Origin of the eye coordinate system O '

Radius of corneal curvature R

Measure of the autofocus of the video camera ΔS ₀

Distance S ₀ Distance Si

Vector s

Auxiliary vector ^' s "

Object distance wl

Intermediate image distance w3 distance exit pupil from lens 4 w4

Image width w5

Object distance of the eyepiece lens 6 w6

Item range w7

Coordinates x, y, z object size yl

Intermediate image size y3

Exit pupil size y4

Image size y5

Outer lens diameter 6 y6

Claims

claims

1. Method for controlling microscopes with connected devices by means of line-of-sight analysis, in which the eye is illuminated with infrared light, imaged by an optical system with an eyepiece tube, recorded by an image sensor, and the image thus obtained is then determined in a processor for determining the gaze position the pupil center and the coordinates of the generated corneal reflexes are further processed, the view position data of the viewer obtained by the processor being calibrated on the work field of the viewer and with this calibrated data the automatic focusing and / or the illumination and / or the adjustment of the microscope slide and the connected devices are controlled. characterized in that a strongly diffuse light with a wavelength of 800 to 1000 nm is used as infrared light, which is generated coaxially around the eye of the viewer on the eyepiece tube, and that in the beam path of the optical system at least one other independent of the place of generation of the diffuse light the viewer's eye directs an infrared light signal with a different intensity than the diffuse light and a wavelength of 800 to 1000 nm, which is used to generate at least one additional corneal reflex for the diffuse light.

2nd The method of claim 1, d a d u r c h g e k e n n z e i c h n e t that the of the

Image sensor (10) captured analog eye image (20) is divided into different shades of gray by analog / digital conversion (grayscale image generation), or is broken down into events by an edge detection logic, each video line according to dark-light or light-dark Transitions are searched, and the information obtained in an event table are incorporated, which are fed to the processor (32) for determining the coordinates of the pupil center, corneal reflexes and, from there, the viewing direction.

3. The method of claim 1, d a d u r c h g e k e n n z e i c h n e t that the visible light from the infrared light through a wavelength-dependent beam splitter, e.g. a half mirror with an anti-reflective coating, a prism, or a polarizing beam splitter is separated, which reflects more than 80% of the infrared light used to illuminate the eye and more than 80% of the visible

Light with a wavelength below 800 nm is transparent.

4. The method according to claim 1 to 3, d a d u r c h g e k e n n z e i c h n e t that constant with a second infrared light signal

Intensity a second corneal reflex at a distance (d) from the first corneal reflex for auto-focusing the eye image is generated.

5. The method of claim 1, d a d u r c h g e k e n n z e i c h n e t that the

Image sensors (10) of the video camera (9) are connected both to the analog / digital converter (30) and to an edge detector logic (34).

6. The method according to claim 1 to 5, characterized in that the grayscale images are guided via a video memory (41) in which the images are analyzed with a histogram function to ensure automatic detection of the pupil.

7. The method according to claim 1 to 6, so that the digital image output signals are converted into analog output signals to which false colors can be mixed.

8. The method according to claim 1 to 7, d a d u r c h g e k e n n z e i c h n e t that for

Calibration of the eye movements an object carrier (55) with markings in the object plane or an image reflected in the intermediate image plane of the microscope is used, which are each related to the physical quantities of the operation field (54).

9. Direction-of-view measuring device for controlling microscopes and devices coupled thereto, with an infrared light illuminating the eye, an image sensor imaging the eye, a processor calculating the direction of view and a monitor indicating the pupil center and the corneal reflexes, characterized in that it consists of a video multiplexer (29) , on the inputs (36, 37) of which several video signals are assigned, to which image sensors (10) are assigned, and the output of which either directly via an analog / digital converter (30) or via edge detection logic (34) is connected to the direction of view processor (32) and consists of a digital / analog converter (RAMDAC) (39) for the image signals, at least one video output (35) for displaying the eye image with superimposed markings on the pupil and corneal reflexes is present, and via a serial or a parallel Interface (50) is connected to a control unit (51) for the control of devices acting on the working field.

10. line of sight measuring device according to claim 9, d a d u r c h g e k e n n z e i c h n e t that

Image sensors (10) have a vertical scanning frequency of 50 Hz or higher.

11. Line of sight measuring device according to claim 9 and 10, so that an additional input of the line of sight measuring device (46) is provided for simultaneous line of sight analysis of the second eye by means of a further image sensor.

12. Attachment for the eyepiece tube of a microscope, for example a surgical microscope, with at least one eyepiece system for visual observation of the operating area imaged by the objective lens through the

User, a lighting system for one or both eyes of the user and at least one eyepiece lens arranged in the eyepiece tube, for controlling a microscope and connected devices by means of viewing direction analysis, characterized in that infrared diodes (coaxially arranged to the eyepiece lens (6) and covered by a diffuser disc (13) 11) are arranged to form a lighting ring (12) which has a recess (14) for the user 's nose and which is at right angles to the optical axis (B) of the eyepiece tube (1) in At least one tube (15) is arranged a further infrared diode (16), the IR light of which is directed to a CR condenser lens (17) arranged in the beam path and a prism (19) which direct the IR light onto the user's eye , The prism (19) being arranged near the inner boundary of the diffuser disc (13) and being adjustable in this position, and that a wavelength-dependent beam splitter (5) reflecting this infrared light but highly permeable to visible light is at an angle of 45 ° to the optical one Axis (B) is arranged in the eyepiece tube (1), and that in an imaging plane (8) downstream of the intermediate plane (Z _E ) at least one IR-sensitive sensor system (10) for detecting the

Eye image and the generated corneal reflex is arranged to which the direction-of-view measuring device (46) is assigned.

13. Article according to claim 12, d a d u r c h g e k e n n z e i c h n e t that the

Diffuser disc (13) made of thermoplastic polymers, for example polyoxymethylene with a high degree of crystallization.

14. Article according to claim 12, d a d u r c h g e k e n n z e i c h n e t that the

Diffuser disc (13) made of glass with high diffuse reflection.

15. An attachment according to claim 12 to 14, characterized in that the infrared diodes (11) are arranged under the diffuser disc (13) in a star shape around the eyepiece axis (B).

16. Article according to claim 12 to 15, d a d u r c h g e k e n n z e i c h n e t that the

Infrared diodes (11) are aligned under the diffuser disc with their longitudinal axes coaxial (parallel) to the eyepiece axis (B).

17. Article according to claim 12, d a d u r c h g e k e n n z e i c h n e t that the

Tube (15) has a variable length.

18. An attachment according to claim 12, d a d u r c h g e k e n n z e i c h n e t that in addition to the tube (15) a further tube (26) is provided at right angles on the eyepiece tube (1) in which a further separate infrared diode (27) is arranged.

19. Article according to claim 123, so that the wavelength-dependent beam splitter (5) has an anti-reflection coating made of light metal fluorides.