DE10341822A1 - Three dimensional object photogrammetry recording method, e.g. for use in geological survey, involves storing picture information in polar coordinate system, where information allows eventual turning or tilting of camera - Google Patents

Abstract

Three dimensional object photogrammetry recording method involves storing extracted picture information in a polar coordinate system. The picture information allows eventual turning, tilting of a camera (2). An automatic extrapolation of geometrical primitive, e.g. surfaces, edges, and vertices, takes place during operating conditions. The information is corrected geometrically, such that the picture is free of distortion from the point of view. An independent claim is also included for an arrangement for photogrammetry recording, comprising an optoelectronic camera.

Description

operation area and technical background

The This invention relates to the field of true-to-life photographing three-dimensional objects in space and the automatic extraction their spatial Coordinates without procedural image angle limits with the means the computationally supported electronic image acquisition and processing. Deputy Areas of application are all tasks of photogrammetric data acquisition, for example for structural allowances, industrial plant documentation, facility management or geological Surveying, but also the faithful takeover real scenarios in computer construction systems, modeling, Simulation and visualization tasks, as well as various multimedia Applications.

The Invention affects different technical topics, so that the prior art separately for these individual areas separately is analyzed.

1) Photogrammetric admission process

Photogrammetric Shooting takes place with the help of recording cameras, at least from an imaging lens and a generally planar electronic Image sensor, or a mechanical support with a photo emulsion, consist.

als Tangensfunktion, also nichtlinear, zu dieser Vorzugsrichtung auf. Hierbei ist α der Abstand des abgebildeten Objektpunktes P' vom Bildhauptpunkt H' und α der Einfallswinkel des Objektpunktes P, während c als Kamerakonstante bekannt ist und bei Einstellung auf unendliche Objektentfernung annähernd mit der Objektivbrennweite übereinstimmt.The outer orientation of a according 1 on one level ( 3 ) projecting recording camera ( 1 ) is determined by three translational spatial coordinates x ₀ , y ₀ , z _{0 of} the projection center in the objective ( 2 ) and three rotational position coordinates (pivoting φ, inclination ω, cant κ) of the image coordinate system ( 4 ) the camera. Even if a subsequent correction via displacement and rotation matrix correctly arranges the image into the object coordinates, 6 degrees of freedom are already fixed during image acquisition. The recording direction, given by the optical axis of the lens, determines a fixed preferred direction during image acquisition. As in 2 is shown, the image geometry is based on the equation

as a tangent function, ie non-linear, to this preferred direction. Here, α is the distance of the imaged object point P 'from the main image point H' and α is the angle of incidence of the object point P, while c is known as the camera constant and when set to infinite object distance approximately coincides with the objective focal length.

each change of position the recording axis of the camera leads in planar illustrations to change the linear image geometry. This fact becomes, for example, unsuccessful Try clearly, wide-angle panoramas of planar captured frames correctly assembled with different pivoting angles.

at The mapping of planes in object space is always inevitable perspective distortions when the optical axis during the Recording was not perpendicular to these levels. The condition of a Vertical position is almost impossible to guarantee in practice. The named distortions are caused by non-parallel imaging visible from the original ansich parallel lines. For the perspective Equalization methods are known, but only then can be geometrically correct, if the original one Orientation of the camera in the object space and the inner orientation of the Camera, including of the image main point, are known. Otherwise, only approximate results are to expect.

die von der radialen Verzeichnung verursacht werden und als Kissen- (a) oder Tonnenverzerrungen (b) bekannt sind.It must also be considered that all technical objectives produce aberrations that lead to deviations from the idealized central projection. Will, as in 3 , a rectangle shown, so show deviations in the ratio of the side lengths to the longer diagonal between the original and image, ie

which are caused by the radial distortion and as cushion (a) or barrel distortion (b) be are known.

Just the more common observed barrel distortion leads to the said Tangensfunktion is linearized.

While in the past the construction of the measuring lenses photogrammetric Measuring chambers were optimized to the smallest aberrations, corresponds it now the usual state technology, camera equipment to be used for general photographic purposes, the Objective but have significant aberrations. For photogrammetric Applications, these cameras are calibrated using test patterns, so that the result of the calibration is used to correct all image recordings.

As a general rule there is the disadvantage of all planar imaging systems in an angle limitation, which is a complete capture of the entire Situation in the object space around the shooting location prevents and in fixing the camera orientation at the time of recording, which severely restricts a universal reprocessibility of the images.

2) panoramic photography

The so-called real panorama photography captures horizontally endless Shooting with a 360 degree angle. Certain admission procedures allow thereby also an extension of the vertically detectable angle of view to a total of 180 degrees.

The basic form of the electronic scanning panoramic camera, consisting of a line sensor in the image plane of an imaging lens - collectively referred to as a line camera - and a rotary drive is off DE 44 28 055 A1 known. These and other solutions, eg in DE 199 21 734 C2 or WO 01/80550 A3, produce a cylindrical image projection, that is, in the vertical direction, the central perspective plane projection remains with the disadvantage of the principle image angle limitation. To overcome this limitation, be in DE 299 18 951 U1 or US 6,545,701 B2 Proposed method in which the line camera is inclined relative to the axis of rotation, so that in fact results in a truncated cone projection. Such recordings have the problem that no universally reusable imaging model is available, but for each angle of inclination a separate equalization rule is needed. US 2003/0128276 A1 describes a correction rule for panoramic images with non-vertical standing axis. However, merging into fully presentable panoramas is impossible in this way. The use of fisheye lenses with scanning panoramic cameras is well known, which thus captures a very large vertical angle of view, which can also reach 180 degrees. Because of the associated lens properties, no high resolution recording with accurate coordinate assignment is achievable. Methods for geometric calibration of such cameras are not known. Also, no arrangements are known with which a complete horizontal view in both the horizontal and in the vertical direction is generated automatically by a scanning line scan camera.

DE 197 46 319 A1 proposes a device in which a point projection onto a sensor takes place via two mirrors rotating perpendicular to one another, so that, as a result, a spherical image is achieved. DE 199 16 305 A1 extends this arrangement by a laser distance measuring device. Disadvantage of these solutions is an exceptionally high mechanical manufacturing effort to achieve photogrammetrically useful recordings, and a very long recording time.

One method of simultaneous, ie simultaneously recording in all directions, panoramic photography is to project on an optical axis with their vertical upward surface area camera an annular representation of the environment, for which in numerous publications, including in WO 03/027766 A2, EP 908 053 A1 . US 6,028,719 . US 6,313,865 B1 . US Pat. No. 6,480,229 B1 convex aspheric mirrors are used. DE 100 00 673 A1 describes solutions in which this mirror has a spherical shape. An interesting recording and processing methodology is in EP 0 971 540 B1 proposed where a fisheye lens serves to accommodate a hemispherical field of view and a continuous retrieval of random and perspective equalized monitoring images is offered from this field of view. In this case, a mapping geometry is postulated, which in reality only approximately true, which is why the perspective equalization succeeds only approximately. In order to avoid the main drawback of the insufficient solubility of the image for the latter methods, integrated EP 816 891 A1 In addition, a movable, higher-resolution periscope optics in the center of the lens path, but at the same time loses the advantage of the universality of the all-round imaging.

Another method of simultaneous panoramic photography is according to WO 03/038752 A1 or after DE 199 25 159 A1 using an array consisting of multiple area cameras. In US 2002/0075258 A1, such a camera arrangement is additionally supplemented by a higher-resolution camera with a movable deflection mirror. Since the projection centers of the individual cameras in camera arrays can not lie in one common point, the error-free transformation into a universal image is not possible. This disadvantage avoids a device after US Pat. No. 6,560,413 B1 at least for a limited vertical angle range, by adding flat mirrors into the beam path of the cameras.

Furthermore, so-called stitching methods are known in which a number of images recorded by means of a surface camera one after the other in different directions are combined to form a common image. The most widely used application of this kind is known from the computer software product Quick-Time VR from Apple Computer, Inc., Cupertino, CA 95014-2084. There, the image information is stored on the surface of a virtual cube, resulting in different partial resolutions for different viewing angles. Further proposals of similar solutions can be found in US 2001/803802 and WO 01/93199 A1. In US 2003/0128975 A1 a multifunctional tripod head is presented, which facilitates the adjustment of the projection center of the single camera on the axis of rotation. To simplify the application of the method, the equipment of the recording camera with an additional motion sensor is proposed in WO 99/51027. US Pat. No. 6,456,323 B1 Specifies a solution for the color matching of the individual images when merging to the panoramic image. A problem for all known stitching methods is the fact that generally no correct and universal assignment between the original and the stored image coordinates takes place and the imaging properties of the lenses are estimated at best. Thus, the transition areas between adjacent frames can only be approximated.

A certain relationship with panoramic cameras have so-called surveillance cameras, which are remotely mounted on a pan-tilt head. A particularly sophisticated system of such control is in US 5,633,681 presented. Surveillance cameras are not suitable because of their overall concept for panoramic shooting.

In summary is not a practical solution for a panoramic camera with a associated Image memory principle known, so a photogrammetrically evaluable Picture without angle limitation automatically recorded can.

3) Videotheodolite, video tachymeter and pan-tilt cameras

In DE 199 22 321 C2 and DE 199 22 341 C2 An advanced state-of-the-art video tachymeter is described which, like all other known video theorems and tachymeters, decouples the telescope image or uses a parallel lens to image onto an electronic camera sensor. This camera image is used to visualize the target image or targeting. An image storage for photogrammetric evaluation is basically not provided in such devices and not possible with the camera technology used. From the publications of the German Society for Photogrammetry and Remote Sensing, Volume 2, Augsburg 1993, pp 61-70 a pan-tilt camera is known, the recording camera by means of a servo-controlled pan-tilt system reproducible in their Lage controls and records frames from the various positions. A solution for the adoption of these frames in a universal image system is not specified.

4) Photographic extraction distance information

Here be first only methods without structured or collimated light, ie with diffuse object lighting, considered. Base for all known Method forms the inclusion of the object space of at least two different, a parallax causing camera locations. in this connection it is initially without Meaning in which coordinate system the image data are recorded. The object coordinates of object points that are in at least two Recordings are identified and assigned, in a known manner by Triangulation and bundle adjustment be calculated.

A special case of such methods is the stereoscopic image, which is characterized by the assignment of homologous points lying in a core line and considerably simplifies the image analysis because of additional boundary conditions. The long-known planar projecting recording cameras with two lenses at a base distance from each other are not suitable for shooting without frame angle limitation. Out DE 44 28 054 C1 . DE 199 21 734 C2 and DE 101 32 399 C1 are arrangements for stereoscopic Panoramic image with horizontally oriented base known that allow with known presentation means, such as anaglyph or polarization method, a direct stereoscopic viewing of such images and their processing in three-dimensional space. In JP 11325895 A (Patent Abstracts of Japan) and DE 100 08 520 A1 For example, vertical-based arrangements are proposed which, however, do not allow natural stereoscopic viewing. The main problem in the automated evaluation of photogrammetric multiple or stereo recordings is in finding corresponding pixels - the so-called matching process. For discrete objects that are detached from the background, there are discontinuous edge transitions, whose imaged environment has different contents depending on the camera position. If objects obscure each other in space, the resulting shadowing depends on the perspective. In two different images, this results in partial areas, which are only visible in one image and thus can not be measured. In such scenarios, at least three images are always required to measure all visible areas. In practice, many more views are often needed. The overview of the completeness is lost quickly, which is why certain parts of the image for the measurement are not available or the effort for the evaluation is immeasurable.

Become attached easily recognizable or even optically coded marks in the object, so can a reliable automatic readability can be achieved. This sets but extensive Marking ahead, the cost-effectiveness of shooting total relativize. Also, the object to be measured is often not accessible in its entire extent.

Without such markings or other simplifying boundary conditions the automatic measurement in the Nahbereichsphotogrammetrie despite numerous solutions for this Problem so far not mastered. Another problem exists also in that an object or Surface assignment take place must, so from a lot of point coordinates in the end spatial Objects can be. The object assignment of pictorially recorded surface elements can only be automated to a limited extent and requires the regular Intervention of a serving person.

5) Obtaining distance information using structured lighting

The Recurring principle of operation consists in the projection of light patterns, lines, grids or marks on the object surface whose Photographic image and analytical evaluation of the of object surface deformed projection image from one or more observation points.

Here, methods are first of all known in which, instead of target plates to be mechanically attached, optical marks are projected onto the object surface. So also inaccessible object areas for the measurement can be reached. Corresponding proposals for using such light marks, which are also formed as a surface grid or grid pattern, are representative in DE 101 37 241 A1 . DE 10149 750 A1 . DE 195 02 459 A1 or US 6,501,554 B1 contain. As a limiting special form for punctual distance measurement, the triangulation method could be designated, as for example in DE 197 21 688 A1 is described. Instead of the static projection of patterns is in US 6,529,627 B1 proposed the scanning with a projected light line while recording as an image sequence. Because of their universality, projection methods that employ switchable, binary-coded light patterns are particularly useful. DE 198 10 495 A1 gives an example. In order to obtain an even higher resolution and avoid shading by object-own surface profiles, the objects are additionally moved within arrangements of pattern projector and camera. So is located at DE 198 52 149 C2 the object on a turntable. Photographs with structured light projection allow, in principle, good accuracy in coordinate determination and the application of effective algorithms for automatic evaluation. With increasing object expansion, however, their areal lighting becomes more and more complicated because an ever increasing light output and an ever more complex structure is required. This is especially true for wide-angle or panoramic shots.

6) Direct distance measuring method

Methods are known for polar scanning of the object surface with discrete measurement beams at known angles while simultaneously determining the respective distance, for example by transit time, phase comparison or interference measurement. Such tasks are fulfilled by so-called laser scanners. Although among others EP 0 999 429 A1 Arrangements are already known which couple an image recording device with such laser scanners, they themselves are not suitable for image acquisition, but capture only coordinate clouds. Because of their very high manufacturing overhead in the current state of the art In addition, with high demands on the precision of moving assemblies, such devices also have a high price.

Technical task

• – Es soll keine prinzipbedingte Bildwinkelbegrenzung auftreten.
• – Alle Ansichten aus einem Beobachtungsstandort sollen in einem gemeinsamen, universellen Modellkoordinatensystem gespeichert werden können.
• – Es soll ein hohes geometrisches Auflösungsvermögen erreichbar sein.
• – Systematische Abbildungsfehler sollen im Rahmen der mechanisch realisierbaren Toleranzen vollständig korrigiert werden.
• – Einzelansichten sollen nach der Aufnahme durch geometrisch korreke Messbildentzerrung in beliebige Flächen zentralperspektivisch entzerrt werden können.

The object underlying the invention is to produce photogrammetrically evaluable photographic images from an observation site with the following properties:

• - There should be no principle limitation of the picture angle.
• - All views from an observation site should be able to be stored in a common, universal model coordinate system.
• - It should be achievable a high geometric resolution.
• - Systematic aberrations should be completely corrected within the scope of the mechanically realizable tolerances.
• - Single views should be able to be rectified in a central perspective after acquisition by geometrically correct image equalization.

Further should in certain embodiment and under certain conditions of use an automatic extrapolation of geometric primitives, e.g. surfaces, Edges and vertices, done so that in particular a economical solution for fast and accurate Measurement of buildings and interiors as an application example is derivable.

Farther should be a solution be shown for calibration of the recording device.

Troubleshooting

The Problem becomes with the characterized in the claims 1, 3 and 19 Method and in the claims 22 and 25 marked arrangement solved. Advantageous embodiments are in the further claims specified.

The Invention basically uses as a coordinate system for storing all image information the polar coordinate system. Across from All other coordinate systems have this crucial Advantage, not to need any preferred directions or image main points and Thus, at the time of recording no preconditions to set, the one restrict subsequent processing with optional projection directions.

According to 4 is placed around the center O, which is identical to the camera location and is referred to as an observation point, a virtual sphere K. The radius of the sphere has the length 1 as a unit radius. A Cartesian coordinate system (x, y, z) with its origin at the center O describes an arbitrary directional orientation. A superimposed spherical coordinate system (λ, φ) lies with its equatorial plane E in the plane (x, y). The imaging beam from an arbitrary object point P to the observation point O penetrates the spherical surface in a pixel P 'whose coordinates are unambiguously described by the horizontal angle λ _P and the vertical angle φ _P. All object points visible from the observation point O are thus imaged in a homogeneous manner on the imaginary spherical surface, without the position in relation to the orientation of the coordinate systems being of importance. The latter serve only as a basis for assigning measurable coordinates and are arbitrarily set. Because the entire spherical image is related to the coordinate system (x, y, z) with its angular coordinates (λ, φ) around an observation point, this is referred to below as a model coordinate system, in contrast to the image coordinate system, even though this term is still somewhat different Context is in use.

geneigt.The extraction of the image data which are to be imaged on the spherical surface, according to the invention does not happen at a single time for the entire spherical surface, but from a combination of individual, successively recorded sections of this surface. The recording camera, which records only a fragment of the spherical surface delimited in the angle of view, is adapted for this purpose 5 pivoted about its projection center with a given horizontal angle 0 ⩽ α <2π and with a certain vertical angle

inclined.

Starting from a limited projection area B of the camera, the image information for a section F of the spherical surface is obtained in this camera position. Corresponding 12 the camera K is advantageously mounted in a precisely executed gimbal suspension, which allows a pivoting about the vertical standing axis VV and inclination about the horizontal tilting axis HH whose intersection point in the projection center of the camera on the optical axis ZZ.

Without restriction of generality the z-axis is placed in the pivot axis of the camera, while the x-axis has an initial angle α = 0 the pivoting movement marks. Such a constellation makes it possible a principle to be considered Canting angle of the camera around its optical axis generally zero which makes the analytic equations significant simplify.

A recording camera projecting onto a flat sensor matrix generates a picture with discrete rectangular coordinates. Already the view of 5 illustrates, however, that an association between the planar camera image B and the curved portion of the spherical surface F would require a mathematical conversion rule.

abgebildet. Zur Vereinfachung wird bei dieser, wie auch allen weiteren Bildprojektionen die Bildfläche in der sogenannten Positivlage dargestellt. Selbstverständlich entstehen die Bilder physikalisch durch das Projektionszentrum hindurch in der Negativlage, haben aber dabei zur Positivlage völlig identische Maßbeziehungen.This conversion is to be avoided if the pixel coordinates of the camera are not calibrated as usual in a rectangular coordinate system, but basically in polar angle coordinates. During a calibration process, therefore, a value pair of horizontal and vertical angles is assigned to each discrete picture element of the sensor and stored in a correction matrix. 6 illustrates the geometric context. An object point P with known angular coordinates (λ ' _P , φ' _P ) in the image coordinate system (x ', y', z ') becomes on the image surface B with the pixel coordinates

displayed. To simplify this, as well as all other image projections, the image area is displayed in the so-called positive position. It goes without saying that the images physically emerge in the negative position through the projection center, but have absolutely identical dimensional relationships to the positive position.

The mentioned calibration also comes the radial distortion of the Most lenses, where the ansich to tangent proportional Mapping function in the direction of an angle-proportional image is linearized.

führt. Die Ausdehnung G×H entspricht der Anzahl physikalischer Sensorelemente des Bildsensors. Diese Matrix wird als Ergebnis der Kamerakalibrierung als Korrekturmatrix in einer Datei abgelegt.Usually, the pixel coordinates are present in discrete form as the position number of the associated pixels. However, the radiometric values (color value, gray value) to be assigned to each image pixel can be interpolated between the discrete coordinate steps according to a suitable rule to a continuous function. In this way it is also possible to measure the pixel coordinates of the measuring marks, which are imaged for camera calibration, in continuous coordinates. It is even simpler to assign angular pairs measured in discrete coordinates to the discrete pixels, resulting in a matrix of the form

leads. The extent G × H corresponds to the number of physical sensor elements of the image sensor. This matrix is stored as a correction matrix in a file as a result of the camera calibration.

If the camera is a line scan camera with a vertically aligned line sensor, the effort for calibration and storage is reduced considerably because it only needs to measure one-dimensionally and in shape ((Φ ') 0 (Φ ') 1 ... (φ ') H ) T to save. Here, H is the number of physical sensor elements arranged in a row.

In order to reduce the size of the correction matrix, it is of course also possible to reduce the number of support points for the correction function below the number of sensor elements G × H or H, and the remaining coordinates are suitable during correction, taking into account the errors to be accepted interpolate. This could go so far that in general a higher-order function is found which approximates the geometric correction with sufficient accuracy. Since the most essential aberration If technical objectives are caused by the radial distortion, it would also be possible to find a one-dimensional correction function dependent on the radius from the main image point of the camera for the two-dimensional imaging.

The apply with the help of calibration determined angle assignments for the Image coordinate system (x ', y', z ') and under the condition {Α = 0; β = 0} too for the Model coordinate system (x, y, z). For the normal case is a coordinate transformation between both systems of common origin.

7 shows the transformation task. The model coordinate system (λ, φ) is discretized into a predefined angle grid with the constant angular steps (Δλ, Δφ). As far as appropriate image data are available, each intersection of this angle grid should be occupied with a radiometric image value. From the available frame the Verschwenkungswinkel α and the inclination angle β are known. To calculate the angle pair (λ ', φ') from the model coordinates (λ, φ), the operator T is used with the rotation equation deducible for the Cartesian coordinate system

With the polar coordinate substitution x = r · cosφ · cosλ y = r · cosφ · sinλ z = r · sinφ and the boundary condition r = 1 for the common unit radius leads to the equation

The solution of this equation system leads to

mit c1 = λ' – Λji, c2 = λ' – Λj(i+1), c3 = ϕ' – Φji, c4 = ϕ' – Φ(j+1)i errechnet. Mit der vorherigen Koordinatentransformation (λ,ϕ) → (λ',ϕ') wurden somit die radiometrischen Werte für die Modellkoordinaten gefunden.In the next step, the pixels (j, i), (j, i + 1), (j + 1, i) and (j + 1, i + 1) adjacent to the determined value pair (λ ', φ') are within the correction matrix ) according to the regulation Λ ji <λ '≤ Λ j (i + 1) to find. From these four pixels of the original recording, a mean value for the intermediate coordinates is now calculated in operator I by weighted interpolation, eg according to the equation

With c 1 = λ '- Λ ji , c 2 = λ '- Λ j (i + 1) , c3 = φ '- Φ ji , c 4 = φ '- Φ (J + 1) i calculated. With the previous coordinate transformation (λ, φ) → (λ ', φ'), the radiometric values for the model coordinates were thus found.

The proposed radiometric calculation is also valid if a larger number of pixels in the model coordinates arises between discrete pixels in the original image, the geometric resolution So at least partially increased. If, in the other case, fewer pixels are set, so that gaps are created between the original coordinates, further adjacent pixels must be included in the calculation in order to avoid aliasing effects. For the special case that the camera is a line scan camera, the deduced formula work is drastically simplified. With the resulting condition λ'≡0 simply λ = α and thus φ '= φ - β.

Would with the camera only punctiform mechanically scanned, would be furthermore φ'≡0 and therefore φ = β, which is the universality of the model coordinate system.

In the manner described let any camera images that arise under the conditions mentioned, in the spherical model coordinate system take over. Especially when the frames are taken by area cameras the edges of the picture because of the spherical Distortion curved, so that when joining together the pictures an overlap area arises. Besides the possibility the edges along the degree network in the model system to crop and on the then putting straight edges together can make a soft fading within the overlap area be beneficial. For this purpose, they are normalized according to distance criteria weighted radiometric values of one image with the other Picture combined, which also with more than two overlapping ones Pictures can be done.

The basic concept described allows another important option: if the mechanical pivoting and tilting are done with such small angles and precision that the resulting displacements in the image are smaller than the distance between two pixels, then the multiplication of the effective geometric resolution in the image possible. 8th explains the principle with an example. An image area A ₀₀ is supplemented by three further image areas A ₀₁ , A ₁₀ , A ₁₁ , which were each recorded offset by half a pixel pitch to the right, down and to the bottom right. The resulting image A has twice the linear resolution. Such artificial increases in the physical resolution of the sensor matrix are otherwise possible only by their mechanical displacement, for example with the aid of piezo actuators, and are referred to in the literature as microscanning. However, the variation of the angles of rotation proposed here not only saves the additional installation of such actuators, but also advantageously performs the subpixel displacement adequately to the spherical model coordinate system. In the case of using a line sensor, only tilting in the row direction is necessary.

Next the usability of a cheaper Sensor with a smaller number of sensor elements is another Advantage of the described increase in resolution therein, that in use cases with a smaller geometric resolution requirement this Subpixel displacement can be easily omitted. Would take place its intermediate pixels of a higher-resolution matrix omitted, would be these are no longer fully occupied. This would have been because of the sinking ratio the geometric sampling frequency for spatial integration Aliasing effect, which would be expressed by Moire pattern in the picture. To this would have to prevent neighboring pixels are averaged together, giving the time advantage partially re-leveled a smaller number of pixels. Furthermore have to all sensor arrays be read serially in CCD technology, so that anyway physically present sensor elements of a row or column would be to capture.

Also for the Calibration of the camera in spherical Coordinates, the described basic concept offers a simple and Thereby very accurate and effective solution. As a result of that both tilt and tilt angle of the camera accurately are positionable, is sufficient the establishment of a point mark at the level of the projection center in front of the camera and their continuous recording with changing mechanical Angles. about the image co-ordinates of the brand is the respectively set Angle, but with opposite sign, assign. Allows the rotary drive to the tilting axis even a complete rotation, so not even the height of the brand, because then as known from the theodolite ago measured on breakdown and half the vertical angle difference as Angle offset can be corrected. For the horizontal angle is no zero mark definition required. Here can easily from the well-known or suspected location of the main image point are assumed.

If, therefore, a spherical image exists completely or fragmentarily in the model coordinates, then all imaginable photogrammetric evaluations are optimally possible. Almost all transformations and coordinate transformations that are common and required in traditional photogrammetry are eliminated because of the universal coordinate system. As an example, the well-known topic of Messbildentzerrung of levels, eg in facades, called. Necessary and sufficient for the correct representation of a plane in the image is, as described below, the plane projection with a vir optical axis that intersects this plane at the nadir point. This can be controlled by the parallelism of the image of parallel lines in the plane of the original.

Also this is very practical and widespread in surveying technology Polar method has full validity in this model. To be with photogrammetric won angle coordinates to be able to work as well for example, with those of the theodolite, is already recommended when taking the calibration of the observation point and a careful leveling the cradle.

Around a photograph in natural Viewing representation is a central perspective view to demand, of course only makes sense if the viewing angle is less than 180 Degree recorded. Thus, only a limited section of the spherical stored overall image are projected - but this optional with any direction and with any angle within the limits mentioned. It will be a virtual camera in the observation point defined, the program can be tilted and tilted and thus an optional alignment of their virtual optical Axis allowed. The free angle selection appears as virtual Zoom with any focal length.

To project the angle-limited section from the model coordinate system onto a flat surface, a transformation to 9 required, which expires in two steps. In the first step, the coordinates (t, u, v) from the projection plane with a scale constant c are transformed into a spherical intermediate system (x ', y', z ') with the angle coordinates (λ', φ ') ( 10 ). The projection plane Pr is perpendicular to the x 'axis. The origin of (t, u, v) is on the intermediate coordinates (c, 0, 0). For the displacement transformation T ₁ is simple t = x '- c, u = y', v = z '

The intermediate system was adapted to the coordinates in the projection system (u, v) without considering the horizontal and vertical angles (α, β) of the virtual optical axis during projection. This is done now with a subsequent rotation transformation T ₂ in the model coordinate system according to the equation

With the polar coordinate substitution x = r · cosφ · cos (λ-α), y = r · cosφ · sin (λ-α), z = r · sinφ with the radius vector r = √ c² + u² + v² and with the plane condition t = 0, the equation becomes simpler

The solution of this system of equations leads to the general projection rule

The scale constant c has the same meaning as a camera constant or focal length. How out 10 can be seen, the relationships are valid

ein Proportionalitätsfaktor zwischen der Bildbreite und dem zugehörigen Bildwinkel-Tangens.So this is constant with

a proportionality factor between the image width and the associated image angle tangent.

(Pixeln) zugewiesen bekommen.If the image width is given as a number of pixels instead of in a length measure, then c also gets this dimension. For example, to display a picture angle of 80 degrees on a screen with a width of 1024 pixels, c would have to be the size of

(Pixels) get assigned.

mit c1 = λ – Λji, c2 = λ – Λj(i+1), c3 = ϕ – Φji, c4 = ϕ – Φ(j+1)i errechnet. Mit der vorherigen Koordinatentransformation (u, v) → (λ,ϕ) wurden somit die radiometrischen Werte für die Bilddarstellung gefunden. Ähnlich wie bei der Transformation in das Modellkoordinatensystem muss auch hier bei Reduzierung der geometrischen Auflösung die Anzahl einzubeziehender Pixel erhöht werden, um Aliasing-Effekte auszuschließen. Die moderne Rechentechnik erlaubt die Anwendung der gefundenen Transformation für die Ebenenentzerrung in Echtzeit, so dass ein scheinbares Schwenken, Neigen und Zoomen ohne nennenswerte Verzögerungen unmittelbar aus dem Modellkoordinatensystem erfolgen kann. Zur Beschleunigung der Rechnungen werden Terme, die für mehrere Argumente gültig sind, einmalig berechnet und wieder verwendet, wiederkehrende Zwischenergebnisse in Zuordnungstabellen abgelegt und Schleifen nach ihrem Gesamtaufwand optimiert.For the calculated coordinate pair (λ, φ) in the next step, the nearest discrete angles in the model coordinate system according to the rule Λ ji <λ ≤ Λ j (i + 1) Φ ji <φ ≤ Φ (J + 1), i to determine. From these four pixels of the original image, a mean value for the image projection by weighted interpolation, for example according to the equation, is now given in the operator I

With c 1 = λ - Λ ji , c 2 = λ - Λ j (i + 1) , c3 = φ - Φ ji , c 4 = φ - Φ (J + 1) i calculated. With the previous coordinate transformation (u, v) → (λ, φ), the radiometric values for the image representation were thus found. Similar to the transformation into the model coordinate system, the number of pixels to be included has to be increased in order to avoid aliasing effects when reducing the geometric resolution. The modern computer technology allows the application of the found transformation for the plane equalization in real time, so that an apparent panning, tilting and zooming without significant delays can be done directly from the model coordinate system. To speed up the calculations, terms that apply to multiple arguments are calculated and used once, recurring intermediate results are stored in allocation tables, and loops are optimized according to their total effort.

Many measurement tasks take place in a close range, which allows a direct distance measurement by means of point triangulation in the required tolerance limits in relation to technically feasible base distances within the recording arrangement. In this close range, a structured object illumination, for example with collimated laser light, is possible without problems. 11 shows the possible arrangement of a collimated light source Lq whose beam center is located at the base distance b from the projection center of the taking lens Ao and is at an angle γ to the recording axis. The light source Lq and the beam emanating from it are in the plane u = 0. The incident on the surface of the object Ob laser beam is imaged in the vertical angle φ of the recording camera on the image sensor B. The distance between the projection center and the object point is calculated according to the simple equation

Thus, if one or more sources of structured light in certain, precisely adjusted beam angles with the recording camera rigidly connected and pivoted or tilted together with this, so results for each image recording in a different camera position also a number of own light brands, de can be measured individually, from which point the object distance can be determined.

is the image sensor is a vertically oriented line sensor and are the Light sources collimated laser light sources, thus producing a gradual Panning in the resulting image as many spatial light curves as discrete Laser light sources are available. The result for each of these curves is in Appearance and evaluation with the well-known light-section method comparable.

Of course you can also flat Image sensors and at the same time with a horizontal and a vertical angle radiating structured light sources can be used advantageously. It may also be advantageous to use the object with image pattern projections, e.g. from a slide, an LCD projector or similar Arrangements to irradiate.

In addition, can it would be advantageous to have existing projection equipment with one Adjusting device to be provided so that a targeted change the angle of radiation opposite the optical camera axis takes place.

Around distinguish the projected structured light as well as possible from ambient light to be able to the use of color filters is advantageous. If e.g. red radiant Laser diodes used, so come narrow-band, adapted to the wavelength Red filter used.

Around the object projection of the structured light in the image easier find the light is modulated synchronously with the image capture, that shots from the same camera position alternatively once with and once without the light switched on and the difference is calculated from both images. Likewise, it can be proceeded to distinguish different light sources from each other.

to Differentiation of certain beam angles of the structured illumination is further their coding by different wavelengths, or Colors, possible. Also, the structured lighting or parts of it can be synchronous be modulated for field recording, so that in the overall picture e.g. Assign lines with specific dot-dash coding.

auf seine Ebenenzugehörigkeit untersucht werden. Der Grenzwert ∊ berücksichtigt tolerierbare Unebenheiten in realen Objektflächen, die einer Ebene zugeordnet werden, oder auch Messtoleranzen. Gehört ein Punkt nach diesem Kriterium zu einer Ebene, so wird er zur Präzisierung deren Lage mit herangezogen. In einer iterativen Rechnung wird die resultierende Quadratsumme der Abstände aller erfassten Ebenenpunkte minimiert. Somit werden auch Messtoleranzen und Objektunebenheiten bestmöglich ausgeglichen und eine optimale Ebenenapproximation erreicht. Wird ein Punkt erkannt, der nicht zu einer bereits bekannten Ebene gehört, so liegt dieser unter Umständen auf einer neuen Ebene, was mit weiteren Punkten getestet wird.Often, a measurement task is to perform measurements of objects that are mostly composed of regular geometric bodies or surfaces, eg, planes. For this purpose, found object points whose spatial coordinates were determined including their distance to the projection center, examined for their affiliation to common areas. With three known point coordinates is after the equation system ax i + by i + cz i = 1 Already an area determinable. Each additional point can be over its distance to the plane

be examined for his level affiliation. The limit value ε takes into account tolerable unevennesses in real object surfaces, which are assigned to a plane, or also measurement tolerances. If a point according to this criterion belongs to a plane, it is used to specify its position. In an iterative calculation, the resulting sum of squares of the distances of all detected plane points is minimized. Thus, measurement tolerances and object unevenness are compensated in the best possible way and an optimal level approximation is achieved. If a point is detected that does not belong to a known level, it may be at a new level, which will be tested with additional points.

bestimmbar. Über einen beliebigen Punkt, der beide Ebenengleichungen erfüllt, ist damit die Lage der Schnittgerade im Raum definiert.After all known points have been processed and multiple levels detected, all intersection lines between these levels are determined. The direction vector of this intersection line is derived from the normal vectors of both planes

determinable. By means of an arbitrary point that satisfies both plane equations, the position of the intersection line in space is defined.

vertices can as intersections found cutting lines are defined by adjacent surfaces.

With the approaches shown In particular, an automated measurement of interior spaces, e.g. for exact inventory in Facility Management, feasible. This is generally enough only one location of the cradle in each room, in one local or global coordinate network can be integrated.

embodiment

Based an advantageous embodiment the invention will be explained below.

12 shows the basic basic mechanical structure of a receiving arrangement according to the invention. In a stable frame ( 1 ) as Alhidade is a camera assembly ( 2 ) by means of precision drive ( 3 ) rotatably mounted so that an inclination can take place about the horizontal tilting axis HH. The frame in turn is on its underside with another precision drive ( 4 ), which allows a pivoting movement about the vertical standing axis VV. This rotary actuator rests on a standardized plug ( 5 ) for surveying equipment for mounting on a tripod, which together forms the limbus. Rotary actuator and pin have a central bore through which an overlying laser diode with collimator ( 6 ) projects its collimated beam vertically downwards and thus performs an optical solder function. This makes it possible to wobble the receiving arrangement onto a measuring mark. On the frame is still an adjusted tube level ( 7 ) for leveling the recording arrangement. A desk-shaped housing ( 8th ) takes up a part of the control electronics, in particular the user interface with keyboard and display.

13 explains the exemplary construction of the camera assembly according to 12 ( 2 ). The objective ( 20 ) is located with its object-side main point as the center of projection exactly in the center of rotation, in the horizontal axis of rotation VV according to 12 lies. In the projection plane of the lens, the image sensor ( 21 ), which here is a line sensor with three parallel individual lines for the three primary colors red, green and blue. A UV filter ( 22 ) in the beam path prevents the falsification of image acquisition by ultraviolet radiation. Symmetrical to the camera axis are arranged two groups of three laser diodes with preset collimators, which are all in the same plane with the projection center and the line sensor. These laser diodes emit light in six different vertical angles, so that six light curves are simultaneously generated in the object.

In 14 the functional structure of the embodiment is shown in the form of a block diagram. A microprocessor system (μP) is provided with a program memory (ROM), a random access memory (RAM), a counter / timer (CTC), a user interface (BI) consisting of a display and an input keyboard, and an external data interface (DI). connected. The microprocessor system controls in each case via an interface circuit (I) the precision drive (M) in the vertical and horizontal axis of rotation. An encoder (E) monitors the mechanical rotation angle achieved by the microprocessor system and adapts it to the setpoint. The camera interface (KI) with the image sensor control (B) and the integrated analog-to-digital converter for the image sensor signal controls the image acquisition and transmits all image information to the processor system. A diaphragm actuator (BM) can be used to change the lens aperture. The laser diodes (LD) are also connected via an interface as a modulator (IM) to the microprocessor system, from which they are switched on and off depending on the program.

Claims (48)

Method for photogrammetric image acquisition with an opto-electronic camera with a or two-dimensional matrix-shaped, arranged according to the rules of a linear rectangular coordinate system structure of the image sensor, characterized in that the storage of image information obtained takes place in a spherical coordinate system, the information technical description of the optical image projection through the central observation point in space on a virtual sphere surface with a For this purpose, a coordinate transformation of the image elements obtained in the image acquisition by central projection in discrete horizontal and vertical angle steps is performed, both a camera-specific assignment matrix between the discrete picture elements in the image space and the corresponding angles of incidence in the object space, as well as any pivoting, tilting or tilting of the camera, according to the rules known from analytic geometry become. Method according to claim 1, characterized in that that the coordinate transformation did not happen immediately during the Recording, but later based on the cached frames. Method according to claim 1, characterized in that that the generation of the camera-specific assignment matrix during a one or repeated calibration process is done so that the camera located on a fixed observation point fixed target point and gradually swung and / or is inclined for that each pixel onto which the target point is mapped, the associated instantaneous one Assignment angle is assigned, or the mapping between pixel coordinates and angle of incidence in another suitable manner, e.g. as approximated mathematical function, takes place. Method according to claim 1, characterized in that that the mapping of the gray or color values into the discrete coordinates of the spherical Target system by interpolation of the nearest adjacent angle coordinates considering distance-based Weighting factors takes place. Method according to claim 1, characterized in that that by pivoting and / or tilting in defined angular steps around a central observation point, repeated image acquisition in each one of these discrete camera positions and supplementing those obtained as partial images new image information to the common spherical coordinate system the Object field gradually scanned around the observation point becomes. Method according to claim 5, characterized in that that for the purpose of enlarging the total captured angle of view the angle increments are chosen to be so large that the captured frames merge or overlap at their edges, making a gapless Projection of the entire field of view on the whole sphere is possible. Method according to claim 5, characterized in that that in overlapping areas between the partial images a gradual transition from a partial image on the other hand by weighted overlay the image information is produced. Method according to claim 5, characterized in that that the central observation point before taking pictures in a local or global coordinate system is measured. Method according to claim 5, characterized in that that to avoid tilting a leveling of the recording device is made, in which the pivoting and tilting axes in vertical, horizontal position. Method according to claim 5, characterized in that that for the purpose of raising the effective image sensor resolution the angle steps chosen so small that they are smaller than the discrete picture elements of the Imagine angle differences are assigned to image sensors, so that newly captured picture elements are stored between already existing ones become. Method according to claim 1, characterized in that that at least one directional projection device, consisting from a light source with structured or collimated light, with a constant angle to the optical axis of the camera and with respect to the possibly pivotable, tiltable or cantilevered one Camera fixed position a detectable by the camera light pattern projected on the object and by evaluation of the image coordinates selected Points of this light pattern with knowledge of parallax and direction angles the distance to the examined object points according to the known ones Rules of triangulation is determined. Method according to claim 11, characterized in that that for filtering the projected and imaged light pattern from the rest Extraneous light a second image recording in the same camera position without projected light pattern, and then a difference image two recordings is calculated. Method according to claim 12, characterized in that that the difference image calculation also for images with partially turned on Light pattern projection is done to different angles of emission clearly assigned. Method according to claim 13, characterized in that that the partial switching on and off of the light pattern projection according to the rules of the binary Coding is done. Process according to claims 5 and 11, characterized that the projection devices synchronized during recording to change the camera position can be modulated in their intensities, so that from the figure a code assignment to the corresponding beam angle can be done. Process according to claims 5 and 11, characterized that the pattern of existing electrically controllable projection original the projection devices during recording in sync with the change changed the camera position so that from the figure a pattern mapping to the corresponding Beam angle can be done. Method according to claim 11, characterized in that that the projection devices light different, from the beam angle dependent Wavelength ranges send out, so that from the picture a color assignment to the corresponding Beam angle can be done. Method according to claim 11, characterized in that that shots from the same camera position with both, as well without, turned on projection devices and performed stored separately, on the one hand evaluable in spatial coordinates Measurements, on the other hand, but also images without superimposed To obtain light marks with the same imaging geometries, so that the measured and converted into a 3D model object profiles without more geometric corrections with textures of the natural Views overdrawn can be. Method for photogrammetric image processing, characterized in that in the form of digitized gray or Color values organized in a horizontal and vertical angle Matrix stored image after optional specification of a horizontal and vertical angle for the optical axis of a virtual camera, as well as its field angle, a transformation into the associated one central perspective illustration in the form of a right-angled two-dimensional Target coordinate system is made and this target coordinate system either comes to the direct image display or as additional Image file is saved individually. Method according to claim 19, characterized that for the production of land equalized measurement images, in particular for floor plans and facade measurements, the optical axis of the virtual camera is chosen so that it is vertical to the area to be rectified what is additionally about the pictured parallelism controlled by original parallel lines within this area can be. Method for photogrammetric image processing, characterized in that all points with detected complete space coordinates within a project-specific tolerance limit on their group affiliation be checked to common areas from the totality of all recorded points within each Group the associated one Surface defining equation according to the rules of statistical averages, or according to the criterion the least squares are formed, from the thus known surfaces whose Line of intersection and the intersections formed between the intersection lines and the surfaces are calculated as node or corner points, so that in particular for shorter distances an automatic determination of simple body geometries, e.g. as an indoor space allowance, is made possible. Arrangement for photogrammetric measurement image recording, consisting of an optoelectronic camera with matrix-shaped, arranged according to the rules of a linear rectangular coordinate system structure of the image sensor gimbal stored with preferably extending through their projection center axes of rotation and is provided with electrically individually controllable drives for the horizontal and vertical angle adjustment , characterized in that the camera output leading to the digitized image signal, the control input of the camera and the control inputs and outputs of the drives are connected to a central control, preferably in the form of a microcomputer, which further comprises a data memory for storing the image information obtained, as well as a program memory, said control computer also fulfilling the function of a synchronization of the drives with each other and with respect to the camera control, and that both the rotary bearings, as well as the drives with suitable mechanical elements, in particular with precision gears and Winkelencodern provided are that ensure a positioning accuracy that is better than the converted to the optical angle of incidence of the camera distance between two adjacent pixels of the image sensor in the respective direction. Arrangement according to claim 22, characterized that the cardanic storage is carried out in the manner of a theodolite, that a fixed limbus a vertical pivot bearing contains which carries a horizontal angle rotatable Alhidade, the In turn, has a horizontal pivot bearing on which the Camera is rotatably mounted in the vertical angle. Arrangement according to claim 22, characterized that the matrix of the image sensor is one-dimensional, so the camera a line camera is. Arrangement according to claim 22, characterized in that at least one structured or collimated projection device Light mechanically rigidly connected to the camera so that the Object illuminated within the region imaged by the camera and parallax between the beam paths of the camera and each projection device is available. Arrangement according to claims 24 and 25, characterized that the main beam path of the projection devices, the imaging center the line scan camera and the center line of their line sensor in one common plane. Arrangement according to claim 25, characterized that existing projection devices with single or common Adjusting devices are provided so that the horizontal and / or Vertical angle of the projection devices, or parts thereof, across from the projection axis of the line camera is variable. Arrangement according to claim 27, characterized that at least one adjusting device for the projection devices discrete preferential angle, between which a stepwise Switchover takes place. Arrangement according to claim 27, characterized that the adjusting devices for the projection devices with electrically controllable drives are provided, which are connected to the control computer. Arrangement according to claim 25, characterized that the projection devices from one or more laser light sources consist of collimators. Arrangement according to claim 30, characterized that at least 2 laser light sources a common collimator have. Arrangement according to claim 30, characterized in that the laser light sources are connected to modulators having a Control of light intensity enable. Arrangement according to claim 32, characterized that the modulators are connected to the control computer. Arrangement according to claim 30, characterized that existing laser light sources have different wavelengths. Arrangement according to claim 25, characterized that the projection devices each consist of a light source with beam shaping means, e.g. Condenser, a steady Projection original, for example in the form of a slide, as well as consist of a projection lens. Arrangement according to claim 35, characterized that the projection original is electrically controllable, in particular ie a transmitted light liquid crystal display, a micromirror array or a comparably acting element and that the control input of this projection original with the control computer connected is. Arrangement according to claim 35, characterized that the projection original optical band filter or beam splitter contains so that in the projection screen one dependent on the radiation angle Wavelength distribution takes place. Arrangement according to claims 34 or 37, characterized that in the beam path of the line camera at least one optical Filter constantly or temporarily interposed, that of the projection device radiated wavelength ranges is tuned, so that a better selectivity between the on the object projected structure or parts thereof and other extraneous light is reached. Arrangement according to claim 22, characterized that they are equipped with a leveling device, preferably one the surveying known Dreufuß or ball base, connected or a standardized mechanical adapter, preferably a spigot, having, with commercially available Horizontiereinrichtungen can be connected. Arrangement according to claim 22, characterized that they are for their leveling at least one control device, preferably a tube dragonfly, having. Arrangement according to claim 22, characterized that they are for their leveling an automatic leveling in having known design. Arrangement according to claim 22, characterized that they for staggering on a brand an optical solder in known execution having. Arrangement according to claim 22, characterized that they stagger down to a mark one vertically having aligned laser projection device. Arrangement according to claim 22, characterized that on their mechanical support and movement elements at least one passive or active brand for self-calibration exists at certain vertical or horizontal angles can be detected by the beam path of the camera. Arrangement according to claim 22, characterized that the camera with a target optical device, preferably a diopter or a finderscope. Arrangement according to claim 22, characterized that these in any position, in particular lying, mounted is, so that the standing axis preferably assumes a horizontal position. Arrangement according to claim 22, characterized that this is mounted on a tripod to aerial view similar To be able to produce recordings. Arrangement according to claim 22, characterized that this is mounted on a tethered balloon, similar to aerial photography To be able to produce recordings.