DE102006042324B4 - Method for generating computer-generated video holograms in real time by means of sub-holograms - Google Patents

Abstract

Method for generating computer-generated video holograms in real time, whereby - from image data with depth information, hologram values for the representation of a three-dimensional scene (S) structured by object points are coded on a spatial light modulator (SLM), - in such a way that those wavefronts which would emit an object, is reconstructed in one or more virtual observer windows (OW), the observer window (OW) lying in a reference plane that is assigned to the observer, and only a partial hologram (TH ) is used as a subset of the SLM, - where for each object point (P) the contributions of the partial holograms (TH) to the entire reconstruction of the scene (S) from its position through its lateral x and y coordinates and its depth distance and current viewer position can be determined from look-up tables.

Description

Field of the invention

The invention relates to a method for generating computer-generated video holograms CGVH from three-dimensional image data with depth information in real time. In the field of generating holographic data, the invention relates to the transformation of subregions of the scene, wherein the transformation describes the propagation of the light waves. In the holographic representation of 3D objects or 3D scenes, the light wavefront is generated by the interference and superposition of coherent light waves.

Unlike classical holograms, which are photographically or otherwise stored as interference patterns, CGVH exist as a result of the calculation of hologram data from sequences of a three-dimensional scene and their storage by electronic means. Modulated interference-capable light propagates in the space in front of the eyes of a viewer as a controllable by the amplitude and or phase values light wave front to reconstruct a three-dimensional scene. The activation of a spatial light modulator SLM with the hologram values of the video holograms causes the pixel-modulated wave field emanating from the display screen to reconstruct the desired three-dimensional scene by interfering with the space. A holographic display typically includes an array of controllable pixels, the pixels reconstructing object points by electronically influencing the amplitude and / or phase of illuminating light. Such an arrangement is a form of Spatial Light Modulator SLM. The display can also be continuous rather than matrix-shaped. For example, it may be a continuous SLM, including a continuous matrix-controlled SLM or acousto-optic modulator AOM. A suitable display device for the reconstruction of video holograms by spatial amplitude modulation of a light pattern is, for example, a liquid crystal display LCD. However, the invention may also be applied to other controllable devices that use coherent light to modulate a light wavefront.

In this document, the term 'pixel' denotes a controllable hologram pixel in the SLM; a pixel is individually addressed and driven by a discrete value of a hologram point. Each pixel represents a hologram point of the video hologram. In an LCD, therefore, the term 'pixel' is used for the individually addressable pixels of the screen. In a DLP, the term 'pixel' is used for a single micromirror or a small group of micromirrors. In a continuous SLM, a pixel is the transition region on the SLM representing a complex hologram point. The term 'pixel' therefore generally refers to the smallest unit that can represent a complex hologram point, that is, to display. Finally, the term holographic coding or short coding designates the generation of complex hologram points from image data with depth information.

Technical background and state of the art

The computer-generated video holograms can be reconstructed, for example, with a hologram display, which the applicant already in the document WO 2004/044659 A2 has described. For this purpose, the viewer looks through at least one corresponding virtual viewer window, which is larger than an eye pupil, to the display screen.

Other methods for computer-generated video holograms CGVH from three-dimensional image data with depth information, however, in the publications Lucente, M: Computational holographic bandwidth compression. In: IBM Systems Journal, IBM Corp. Armonk, New York, US, Vol. 35, No. 3 & 4, 1996, pp. 349-365. - ISSN: 0018-8670 and WO 02/39192 A1 described. However, these pamphlets do not work with viewer windows, but instead make a scene visible at any point in a fixed, spatially extended viewing zone.

A 'viewer window' is a limited virtual space through which the viewer can view the entire reconstructed 3D scene with sufficient visibility. The viewer window is located on or near the eyes of the viewer. The viewer window can be moved in the directions X, Y and Z. Within the observer window, the wave fields overlap in such a way that the reconstructed object becomes visible to the viewer. The windows are close to the eye of the viewer, can be tracked with known position detection and tracking of the current viewer position. Thus, they can be advantageously limited to a size that is little greater than the pupil size. It is possible to use two viewer windows, one for each eye. More elaborate arrangements of observer windows are also possible. It is also possible to encode video holograms that contain objects or entire scenes that the viewer sees behind the SLM.

The term 'transformation' is to be construed as including any mathematical or computational technique that approximates or approximates a transformation. Transformations in the mathematical sense are merely Approximations of physical processes, which are described in more detail by Maxwell's wave propagation equations. Transformations such as Fresnel transforms or the special group of transforms known as Fourier transforms describe second order approximations. Transformations usually lead to algebraic and non-differential descriptions and can therefore be handled computationally efficiently and performant. Moreover, they can be used precisely in optical systems.

The WO 2006/066906 A1 ( PCT / EP 2005/013836 Applicant describes a method of calculating computer-generated video holograms. It assigns objects with complex amplitude values of a three-dimensional scene in grid points of parallel, virtual cutting planes to define a separate object data set with discrete amplitude values in grid points for each cutting plane and to calculate from the image data sets a holographic coding for a spatial light modulator of a hologram display ,

• • aus jedem Objektdatensatz jeder tomographischen Szeneschnittfläche wird ein Beugungsbild in Form einer separaten zweidimensionalen Verteilung von Wellenfeldern für eine Betrachterebene mit einem endlichen Abstand parallel zu den Schnittebenen berechnet, wobei die Wellenfelder aller Schnitte für mindestens ein gemeinsames virtuelles Fenster berechnet werden, das in der Betrachterebene nahe den Augen eines Betrachters liegt und dessen Fläche gegenüber dem Videohologramm reduziert ist,
• • die berechneten Verteilungen aller Schnittebenen werden zur Beschreibung eines gemeinsamen Wellenfeldes für das Fenster in einem zur Betrachterebene referenzierten Datensatz addiert und
• • der Referenzdatensatz wird zum Erzeugen eines Hologrammdatensatzes für ein gemeinsames Computer generiertes Hologramm der Szene in eine von der Referenzebene endlich entfernte, parallele Hologrammebene transformiert, wobei in der Hologrammebene der Spatial Light Modulator liegt, mit welcher nach entsprechender Kodierung die Szene in den Raum vor den Augen des Betrachters rekonstruiert.

The solution of the problem utilizes according to the invention the basic idea of carrying out the following steps computer-assisted:

• From each object data set of each tomographic scene intersection, a diffraction image is calculated in the form of a separate two-dimensional distribution of wavefields for a viewer plane with a finite distance parallel to the intersecting planes, the wavefields of all intersections being calculated for at least one common virtual window close to the observer plane is the eyes of a viewer and whose surface is reduced compared to the video hologram,
• • the calculated distributions of all cutting planes are added to describe a common wave field for the window in a dataset referenced to the observer plane, and
• The reference data set is transformed into a hologram of the scene generated by the computer for generating a hologram data set in a parallel hologram plane finally removed from the reference plane, wherein the hologram plane contains the spatial light modulator with which, after appropriate encoding, the scene enters the space before the hologram Eyes of the viewer reconstructed.

The above methods and displays are based on the idea not to primarily reconstruct the object of the scene, but to reconstruct that wavefront that would send out the object into one or more virtual observer windows.

Through the virtual viewer window, the viewer can see the scene. The virtual observer windows cover the pupils of the observer and are tracked with known position detection and tracking devices of the current observer position. Between the Spatial Light modulator of the hologram display and the observer windows, a virtual truncated pyramid-shaped observer area, the so-called frustum, is spanned, the SLM forming the base area and the observer window forming the top. For very small observer windows, the truncated pyramid can be approximated as a pyramid. The viewer looks through the virtual viewer window in the direction of the display and takes in the viewer window on the wavefront, which represents the scene.

Due to the large number of necessary transformations, the holographic coding is associated with high computational complexity. Real-time coding would require high-performance and costly computing units.

Object of the invention

The object of the invention is to provide a method which allows real-time video holograms to be generated from three-dimensional image data with depth information. The generation should be able to be carried out by simple and inexpensive computer systems.

Summary of the invention

The object is achieved by the method according to claim 1. The method according to the invention serves to generate computer-generated video holograms in real time. From image data with depth information, hologram values for displaying a three-dimensional scene structured by object points are encoded on a spatial light modulator, SLM.

Analogously to the cited prior art, the method according to the invention is based on the principle of not primarily reconstructing the object of the scene, but of reconstructing that wavefront which would emit the object into one or more virtual observer windows. From sufficiently coherent light, a modulated wave field is generated by a hologram-controlled Spatial Light Modulator SLM and a desired real or virtual three-dimensional scene is reconstructed by interferences in space. Based on the SLM, virtual observer windows are generated in truncated pyramidal reconstruction spaces. The windows are close to the viewer's eye and can be tracked with known position detection and tracking the current viewer position. It One starting point of the invention is that the area in which a viewer sees a scene is given by a truncated pyramid-shaped reconstruction space, the so-called frustum, which extends from the SLM to the viewer window. The truncated pyramid can be replaced by a pyramid in an approximation because the observer window is much smaller than the SLM.

Furthermore, the method is based on the principle that the reconstruction of a single object point requires only a partial hologram as a subset of the SLM. The information about each scene point is thus not distributed over the entire hologram, but only in limited areas, the so-called sub-holograms. Following this idea, a single object point of the scene is reconstructed only by a limited pixel area of the SLM, the so-called sub-holograms. The invention is based on the idea that for each object point the contributions of the sub-holograms to the entire reconstruction of the scene can be determined from look-up tables and these sub-holograms are accumulated to form a total hologram for the reconstruction of the entire scene.

• – Schritt (1): Ermittlung der Lage des Teilhologramms je Objektpunkt. Aus der Position eines Objektpunktes, also den lateralen x, y-Koordinaten und dessen Tiefenabstand, werden die Position und die Ausdehnung des zugehörigen Teilhologramms abgeleitet.
• – Schritt (2): Ermittlung der Beiträge des zugehörigen Teilhologramms aus Look-Up-Tables
• – Schritt (3): Wiederholen der beiden Schritte für allen Objektpunkte, wobei die Teilhologramme zu einem Gesamthologramm zur Rekonstruktion der gesamten Szene akkumuliert werden.

In a particularly preferred embodiment of the method, a view of the scene is determined by the position of each observer and his viewing direction. Each observer is assigned at least one virtual observer window located near the eyes in a viewer plane. A three-dimensional discretization of the scene into visible object points takes place in a prepared method step. If necessary, these data are already taken over by an interface. The following process steps according to the invention comprise

• Step (1): Determination of the position of the partial hologram per object point. From the position of an object point, ie the lateral x, y coordinates and their depth distance, the position and the extent of the associated sub-hologram are derived.
• Step (2): determination of the contributions of the associated sub-hologram from look-up tables
• Step (3): repeating the two steps for all object points, the partial holograms being accumulated into a total hologram for reconstructing the entire scene.

The size and position of a partial hologram assigned to an object point is evident in a simplest possible solution due to the set of rays. In this case, the observer window or a partial area covering the pupils is imaged by the object point onto the hologram plane, ie the SLM. Furthermore, this results in the indices of the pixels of the sub-hologram, which are required for the reconstruction of this scene point.

In a further aspect of the invention, additional correction functions are applied to the partial or total hologram in order, for example, to compensate for positional or shape-related tolerances of the SLM or to achieve an improvement in the reconstruction. For example, correction values are added to the data values of the partial holograms and / or the total hologram. Moreover, since each object point is determined by the current position of the observer window, special look-up tables can also be generated for more extraordinary observer windows. For example, if the viewer sees from a lateral position very obliquely on the display.

The principle of look-up tables can be extended with advantage. For example, parameter data for color and brightness information can be stored in separate look-up tables. In addition, data values of the partial holograms and / or the entire hologram can be modulated with brightness and / or color values from look-up tables. A color representation is based on the idea that the primary colors can be determined from respective look-up tables.

The look-up tables on which the method according to the invention is based are preferably used in accordance with FIG WO 2006/066906 A1 ( PCT / EP 2005/013836 ) or WO 2006/066919 A1 ( PCT / EP 2005/013879 ) and stored in corresponding data carriers and memory modules.

The method allows the generation of computer-generated video holograms in real time, for example, for a holographic display according to WO 2004/044659 A2 or WO 2006/027228 A1 , The advantage of this method according to the invention is that the complex generation of the complex hologram values in accordance with WO 2006/066906 A1 ( PCT / EP 2005/013836 ) or WO 2006/066919 A1 ( PCT / EP 2005/013879 ), where numerous transformations of the discretized object into the viewer window and a final inverse transformation into the SLM take place correspondingly complex. In summary, at a commercially available resolution of the SLM, which already enables a high-quality representation of the hologram, the hitherto very high and cost-intensive requirements on the computing unit for generating the holographic data are permanently reduced. The calculation effort can be reduced by several orders of magnitude using the look-up tables. The inventive method thus allows the execution of common PC systems. This ensures that for holographic applications, hologram generation is interactive and real-time. Finally, the reliable generation of the holograms in real time ensures that the resulting undesirable reaction time when tracking the viewer's pupils or windows can be reduced. Since the generation of the holograms for a viewer is definitely done in real time even with simple arithmetic units, the method according to the invention ensures that holograms separated in time or space sequentially can also be displayed for several or many observers. Since the generation of the holograms requires little computational effort, it is conceivable, for example, not to carry out the calculation by the central processing unit (CPU) of a computer. In an alternative solution, the generation of the holograms is created on the components of the graphics card, wherein preferably a Graphics Central Processing Unit GPU and / or specially configured computing units are used. The invention contributes sustainably to the general application and acceptance of holographic displays and is of high economic value.

Preferred embodiments of the invention

1a illustrates the basic idea of the invention on the basis of a single observer. By the position of the viewer (O) and its viewing direction, a view of a scene (S) is set. The viewer is assigned at least one virtual viewer window (OW) lying in a reference plane near the eyes. From sufficiently coherent light, a modulated wave field is generated by a hologram-controlled Spatial Light Modulator (SLM).

The method and the displays derived therefrom are based on the idea not to primarily reconstruct the object of the scene, but rather to reconstruct the wavefront that would emit the object into one or more virtual observer windows (OW). The object is shown isolated in the figure by an object point (P). Through the virtual viewer window (OW) the viewer (O) can see the scene (S). The virtual observer windows (OW) cover the pupils of the observer (O) and are tracked with known position recognition and tracking devices of the current observer position. The activation of the Spatial Light Modulator (SLM) with the hologram values of the video holograms causes the pixel-modulated wave field emanating from the display screen to reconstruct the desired three-dimensional scene by interfering with the space. How out 1a Obviously, according to the underlying principle, a single object point (P) of the scene (S) is reconstructed only by a limited pixel area of the spatial light modulator (SLM), the so-called partial hologram (TH). The size and position of a partial hologram (TH) is as if 1a evident in a simplest possible solution due to the set of rays, resulting in the following the indices of the reconstruction of this object point (P) required pixels. The position and the extent of the partial hologram (TH) are derived from the position of an object point (P), that is to say the lateral x, y coordinates and their depth or z distance. In the look-up table LUT, the hologram values required for the reconstruction of the point (P) are read out. The partial hologram (TH) is modulated with a brightness and / or color value and accumulated in the hologram plane at the corresponding position to the so-called total hologram. The data of the named look-up tables are generated in advance. Preferably, the data is determined according to the method mentioned in the prior art WO 2006/066906 A1 ( PCT / EP 2005/013836 ) and stored in corresponding data carriers and memory modules. On the basis of the position and properties of the object point, the associated partial holograms are calculated in advance and thus the look-up tables of the partial holograms, the color and brightness values and the correction parameters are generated.

1b illustrates this principle further and shows the partial holograms (TH1, TH2) which are respectively associated with the object points (P1, P2). Out 1b It can be seen that these sub-holograms are limited and are a small and contiguous subset of the total hologram, so the entire Spatial Light Modulator (SLM), are. In addition to, as in the 1a it can be seen that the location and extent of the partial holograms determined from the set of rays make further functional relationships conceivable.

Claims (10)

Method for generating computer-generated video holograms in real time, wherein - from image data with depth information, hologram values for representing a three-dimensional scene structured by object points (S) are encoded on a spatial light modulator (SLM), such that those wavefront, the an object would be transmitted, reconstructed into one or more virtual viewer windows (OW), the observer window (OW) being located in a reference plane associated with the viewer, - and only a partial hologram (TH) for reconstructing a single object point (P) ) is used as a subset of the SLM, - wherein for each object point (P) the contributions of the sub-holograms (TH) to the entire Reconstruction of the scene (S) whose position is determined by its lateral x and y coordinates as well as its depth distance and the current observer position from look-up tables. Method according to claim 1, wherein a view of the scene (S) is defined by the position of a viewer (O) and its viewing direction, and at least one virtual viewer window (OW) located in a viewer plane (OP) near the eyesight (O). and after a three-dimensional discretization of the scene (S) into visible object points (P), the method comprises the following steps: a) determination of the position of the partial hologram (TH) for an object point (P); b) determination of the contributions of the partial hologram (TH) belonging to the object point (P) from the look-up tables; c) repeating steps a) and b) for all object points, wherein the associated sub-holograms (TH) are accumulated into a total hologram for reconstruction of the entire scene (S). The method of claim 2, wherein data values of the partial holograms, which are determined from look-up tables, and / or the entire hologram are modulated with brightness and / or color values. Method according to Claim 2, wherein correction values are added to the data values of the sub-holograms and / or to the overall hologram. Method according to claim 1, wherein the position of the partial hologram of an object point is determined on the basis of the position of the object point and the observer window. Method according to claim 5, wherein the observer window is imaged by the object point on the SLM by means of the beam set, and the size and position of the sub-hologram associated with the object point result from the beam set. Method according to claim 1, wherein the indices of the pixels of the SLM which refer to the partial hologram of an object point are determined on the basis of the position of the object point and the observer window. The method of claim 7, wherein the indices of the pixels are determined from the set of rays and the set of rays images the viewer window through the object point on the SLM. Method according to one or more of the preceding claims, wherein for color representation, the primary colors can be determined from respective look-up tables. Method according to one or more of the preceding claims for a holographic display.

Images