CA2078638C - Distance measuring method and a distance measuring apparatus - Google Patents
Distance measuring method and a distance measuring apparatus Download PDFInfo
- Publication number
- CA2078638C CA2078638C CA002078638A CA2078638A CA2078638C CA 2078638 C CA2078638 C CA 2078638C CA 002078638 A CA002078638 A CA 002078638A CA 2078638 A CA2078638 A CA 2078638A CA 2078638 C CA2078638 C CA 2078638C
- Authority
- CA
- Canada
- Prior art keywords
- distance
- image
- lens
- receiving plane
- observed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
- G02B7/28—Systems for automatic generation of focusing signals
- G02B7/36—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
- G02B7/365—Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals by analysis of the spatial frequency components of the image
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
- H04N23/673—Focus control based on electronic image sensor signals based on contrast or high frequency components of image signals, e.g. hill climbing method
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60G—VEHICLE SUSPENSION ARRANGEMENTS
- B60G2400/00—Indexing codes relating to detected, measured or calculated conditions or factors
- B60G2400/80—Exterior conditions
- B60G2400/82—Ground surface
- B60G2400/823—Obstacle sensing
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60G—VEHICLE SUSPENSION ARRANGEMENTS
- B60G2401/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60G2401/14—Photo or light sensitive means, e.g. Infrared
- B60G2401/142—Visual Display Camera, e.g. LCD
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60G—VEHICLE SUSPENSION ARRANGEMENTS
- B60G2800/00—Indexing codes relating to the type of movement or to the condition of the vehicle and to the end result to be achieved by the control action
- B60G2800/22—Braking, stopping
- B60G2800/222—Braking, stopping during collision
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60G—VEHICLE SUSPENSION ARRANGEMENTS
- B60G2800/00—Indexing codes relating to the type of movement or to the condition of the vehicle and to the end result to be achieved by the control action
- B60G2800/24—Steering, cornering
- B60G2800/242—Obstacle avoidance manoeuvre
Abstract
A distance data is obtained from an object to a lens by obtaining an image of the object by an image input unit for forming an image of the object through the lens on an image receiving plane. The present method inputs two observed images Is and Is' with different blurs from the image input unit by making a position of the lens, a position of the image receiving plane with regard to the article different by a minute distance .DELTA.z and performs a filtering process by applying filter to the two input observed images Is and Is', thereby providing two output images Fs and Fs' and calculates the distance a up to the object by using two output images Fs and Fs' obtained by the filtering process based on the relation between the two output images Fs and Fs' and radius s of the blur and the distance a from the lens to the object.
Description
20 786 38 y A DISTANCE MEASURING METHOD AND A DISTANCE MEASURING
APPARATUS
Background of the Invention The present invention relates to a distance measuring method and a distance measuring apparatus.
An input portion of the apparatus forms a three dimensional image of an object and processes it, thereby obtaining distance data from the object to the lens. The input unit uses, for example, a television camera to form a two dimensional image of the object through a lens on a image focusing plane. More particularly this distance measuring method and distance measuring apparatus obtains distance data with ease using monocular based on two observed images with different blurs provided at positions which are different by a minute distance from each other along the same optical axis.
Several attempts for providing a distance measuring method have been made but a method similar to a human vision system has not been provided although needs for such a computer vision system are very great in various industries. At present, the distance measuring methods used for an automatic apparatus require complicated techniques and structure. This is especially true when the distance measuring method is used as a computer vision system for an autonomous vehicle. It is necessary for the vehicle to instantaneously and automatically judge its outside situation in order to autonomously operate to a destination and to stop or avoid an obstacle in its way. The distance measuring method to be used for such an application needs to instantaneously judge with accuracy the distance up to an obstacle that has been detected on its path.
However, a method for obtaining complete distance image data has not been available. It is apparent that an visual sensor with distance sensitivity in three dimensional space like a human vision system will be required by various industries.
Recently, a method of obtaining distance image data by performing computer processing of an image input from an image input apparatus such as a television camera has been utilized for visual inspection or control in the field of factory automation.
A method of measuring distance up to an object based on a principle of triangular measurement includes an image processing method of searching corresponding points between two input images of binocular by using plurality of television cameras and measuring distances between the television cameras and the corresponding points, and parallax between the corresponding points.
On the other hand, a distance measuring method which does not use image processing, but measures distance up to an object by using a pointing device equipped with a laser has been widely used in the industrial field of public engineering works and architecture. A range sensor and a range finder for obtaining distance data based on two dimensional images by using laser scanning have been on the market.
fihe former method using image processing searches corresponding picture elements in each image based on a calculation by using an image input from at least two television cameras. For this method of utilizing a stereo view, a relative method or a dynamic programming method has been proposed for searching corresponding points.
However, although this method can recognize a situation in three dimensional space, it requires a large amount of data as it scans the whole screen and thus it takes time to calculate distance data. It especially takes much time to obtain the distance of a 20 786 ~8 part such as at an edge at which the light and shade variation of the image is large and is easily affected by noise, thereby having the defect in that the complete distance image data cannot be obtained.
The latter method not using image processing can.
extract two dimensional distance data but has the defects in that a high resolution of data is difficult and that it takes time to obtain three dim~~ensional distance data.
Summary of the Invention The present invention aims at obtaining image data by processing input images from an image input unit and calculating distance data based on two different observed images along the same optical axis with different blur by using monocular.
The present invention applies respective filters (Labor function) to two input observed images Is and Is' with different blurs and obtains respective output images Fs and Fs' for respective predetermined frequencies k in a frequency range. The present invention utilizes an expression representing a relation between the output images Fs and Fs' and their radius s of blur and an expression representing a relation between a radius s of the blur and distance a from a lens to an object and substitutes numerical values into the respective variables of the expressions.
Further, the present invention refers to a look-up-table between the observed quantity and the distance up to the 5 object, the table being previously prepared through an experimental calibration. Thus the present invention can calculate the distance data up to the object.
The invention may be summarized, according to a first broad aspect, as a method of obtaining distance data from an object to a lens by obtaining an image of the object by an image input unit for forming an image of the object through the lens on an image receiving plane and then performing a process, comprising the steps of: inputting two observed images Is and Is' with different blurs from the said image input unit by making a position of the lens, a position of the image receiving plane or positions of both of them with regard to the object different by a minute distance Oz on the same optical axis; performing a filtering process by applying filtering to the said two input observed images Is and Is', thereby providing two output images Fs and Fs'; and calculating the distance a up to the object by using two output images Fs and Fs' obtained by the filtering process based on the relation between said two output images Fs and Fs' and a radius s of the blur and the relation between the radius s of the blur and the distance a from the lens up to the object.
According to a second broad aspect, the invention provides a distance measuring apparatus comprising: an image "~' 28151-79 207863$
5a input means comprising a lens and an image receiving plane for obtaining the observed image of the object on the image receiving plane and outputting the observed image; a first operating means for applying a filtering process to the first image input from the said image input means thereby performing a Fourier transformation for a respective predetermined frequency k; a second operation means for applying the same operation as that of the first operation means to the second observed image which is obtained from the image input means when at least one of the lens position, the position of the image receiving plane, and the position of the image input means is moved; and a third operation means for obtaining the distance up to the object based on the results of the first operating means for the first observed image, the result of the second operation means for the second observed image and the distance in which the said at least one of the lens, the image receiving plane and the image input means are moved.
Brief Description of the Drawings One skilled in the art can easily understand additional features and objects of this invention from the description of the preferred embodiments and some of the attached drawings. In the drawings:
Figure 1 shows a basic optical system using the principle of the distance measuring method according to the present invention;
Figure 2 shows a flowchart of the principle of the distance measuring method according to the present invention;
Figure 3 shows a block diagram of the principle of 5b the present invention;
Figure 4 shows a block diagram of the structure of a first embodiment of a distance measuring apparatus according to the present invention;
2o7ss38 Figure 5 shows a general flowchart of the process of measuring distance according to the present invention;
Figure 6 shows a configuration of a block diagram of a second embodiment using a look-up-table defined by the present invention;
Figure 7 shows a diagram of the first example of an image input unit of the distance measuring.
apparatus according to the present invention;
Figure 8 shows a diagram of the second example of the image input unit of the distance measuring apparatus according to the present invention;
Figure 9 shows the third example of the image input unit of the distance measuring apparatus according to the present invention;
Figure 10 shows the forth example of the image input unit of the distance measuring apparatus according to the present invention;
Figure 11 shows a block diagram of the third embodiment of the distance measuring apparatus for processing two images obtained at different points in time; and Figure 12 shows the fourth example of the distance measuring apparatus for receiving two image data at a single point in time period.
Preferred Embodiments First, the principle of distance measurement according to the present invention will be explained.
Figure 1 shows the principle of the present invention wherein the in-focus distance of lens L . f, the radius of lens L . r, the distance from the center of lens L up to the object M . a and the distance from the center of lens L to an image P . b. A lens formula is provided as follows;
1/a+1/b = 1/f ... (1) therefore, the object M located at a preceding position with distance a from the center of lens L
focuses its image P at a succeeding position with a distance b = af/(a-f) from the center of lens L. The distance from the lens L to the object is to be determined. For example, when a focus of a television camera is matched with an infinity point, either the lens L or the image receiving plane S is moved. Then, the image of the object located at the succeeding position with distance a from the center of the lens provides the following blur;
r(b-f)/b = rf/a ... (2) The radius s of the blur is provided based on the geometrical resemblance relation as follows.
s = rf/a ... (3) Where the radius s of the blur is defined as a radius s on the image receiving plane S when the object, namely, a point light source, is located at distance a from the center of the lens L.
For the sake of simplicity, a one dimensional image located on the optical axis is assumed. In this case, the x axis and y axis are located on the image receiving plane S and the z axis is in the direction of the optical axis, thereby providing three dimensional coordinates. Where an image which is in focus (unknown image):IO(x) and an image which is out of focus with blur of radius s (of an observed image):Is (x) an image with blur can be obtained by applying a filtering process to the image which is in focus.
Is(x)= p(x-x')Io(x')dx' ... (4) 2p78638 The function p (x-x') represented by the following expression is a filter function.
p(x-x')=1/2s if ~x-x'~Ss =0 if ~x-x'~>s ... (5) The filter function corresponds to a blur filter, it gives a value of 0 outside the diameter 2s of the blur and gives the value of the function inside the diameter 2s of the blur. If the inverse transformation is performed in the expression (4), an image in focus can be obtained from an image with a blur.
Therefore, a space filter operation ( Gabor function) is applied to the observed image Is (x).
Namely, a filtering process is applied to the blurred image, thereby eliminating blur to provide the image which is in focus.
Fs= exp(-x2/a2)exp(ikx)Is(x)dx ... (6) Where exp(-x2/a2) represents a Gaussian function and exp(ikx) represents a kernel of a Fourier transformation. This expression (6) represents a Gaussian filter applied to the observed image Is, thereby performing a Fourier transformation. k represents a predetermined frequency and Fs represents an output when the center of the filter is positioned at the origin of the coordinates formed by x, y, and z 5 axes. Namely, the output Fs is the data obtained by performing a Fourier transformation of the image for the corresponding predetermined frequency k with regard to the both sides of the origin of the coordinates when the optical axis is provided through 10 the origin.
The following expression is obtained from expression (4).
Fs= exp(-x2/a2)exp(ikx)p(x-x')Io(x')dxdx' ... (7) An integral operation is performed with regard to x.
exp(-x2/Q2)exp(ikx)p(x-x')dx x: ; s =(1/2s) exp(-x2/a2)exp(ikx)dx Q » s x'-s x+s ''=.(1/2s)exp(-x'2/a2) exp(ikx)dx x-s =(1/2s)exp(-x'2/Q2)2sin(ks)exp(ikx')/k =(sin(ks)/ks)exp(-x'2/Q2)exp(ikx') ... (8) Therefore, the following expression is obtained.
Fs=(sin(ks)/ks) exp(-x'2/a2)exp(ikx')Io(x')dx' ... (9) Next, the television camera is moved towards the object M by ~Z along the optical axis, thereby taking.
a photograph of an observed image Is' (x). Then, the radius of the blur is changed to s' and is expressed by the following expression.
s' =rf/ ( a-,oz )'--.( rf/a ) ( 1+~z/a )=s+~z/a=s+s2pz/rf ... (10) When the same filter is applied to the second observed image Is'(x), the following expression (11) is obtained in the same manner as in expression (9).
Fs'=(sin(ks')/ks') exp(-x'2/Q2)exp(ikx') Io(x')dx' ... (11) In order to delete the integral term including the unknown quantity, the expression (9) is subtracted from expression (11) and is further divided by expression (9).
sin(ks')/ks'-sin(ks)/ks (Fs'-Fs)/Fs=
sin(ks)/ks ... (12) The left side of expression (12) represents a definition of a differential equation, assuming that;
ks'-ks=ks2az/rf << 1 ... (13) namely, the amount of the movement of the camera is made sufficiently small. Both sides of expression (12) are divided by pz, thus providing expression (14);
(Fs'-Fs) ks cos(ks)-sinks) s (~z)Fs sinks) fr ... (14) The expression (14) represents a relation between the observed quantity (the left side of expression (14)) and the radius s of the blur. In expression (14), k is a predetermined frequency, namely, a constant number, f and t are known, Fs, Fs' and ~z are values obtained by processing an image when the camera is moved.
When the television camera is moved along the optical axis, images at two points are input to the television camera and a filtering process represented by expression (6) is applied to the respective 13 ~ ~ 7 8 6 3 8 images. A calculation of expression (14) is also performed, thereby obtaining the radius s of the blur. When the expression (3) is substituted into the expression (14), the expression representing the relation between the observed quantity (the left side of the expression) and the distance a up to the object is derived. The respective numerical values of Fs, Fs', Oz, k, f, and r are substituted into the expression and the calculation is performed. Thus, the distance a up to the object is obtained by the calculation.
As is explained above, a distance a can be obtained by carrying out a calculation using the expression which represents the relation between the observed quantity (the left side) and the distance a up to the object and is obtained by manipulating expression (14). The distance a can be also obtained by referring to a look-up-table. The look-up-table is previously prepared for corresponding predetermined frequencies k with regard to the calculation value of the left side of the expression (14) and the distance a up to the object which is actually measured upon an operation of the calculation. The look-up-table is referred to using the input of discretionary image data, thereby obtaining as output the distance a from the input value obtained by calculating the observed quantity on the left side of expression (14).
The distance measuring method of the present invention comprises the following processes. Figure 2 shows a flowchart of the principle of operation of the present invention.
Step ST1 shows an image input process of receiving two observed images Is and Is' having different quantities of blur from the image input unit bY changing the position of the lens, the position of the image receiving plane or the positions of both the lens and the image receiving plane by a minute distance pz from the object on the same axis.
Step ST2 shows filtering process for obtaining two output images Fs and Fs' by applying the filtering process shown in expressions (6) and (7) to the input observed images Is and Is'.
Step ST3 shows a process of calculating distance data for obtaining the distance a up to the object by using two output images obtained by the filtering process based on the expression (3) representing the relation between the radius s of the blur and the distance a from the lens up to the object.
When the distance measuring method of the present invention starts, the process carries out step ST1 for inputting from the image input unit 6 two observed images- which are obtained at positions different from each other by a minute distance on the same optical axis and have different quantities of blur.
5 The process carries out filtering step ST2 for applying a Gabor function to two input observed images and for obtaining two output images in focus with regard to the predetermined frequency. The step ST2 is performed by referring to the calculation shown by 10 the expression (6).
Next, the process carries out step ST3 for calculating the distance data based on the expression representing the relation between the observed quantity and the distance a up to the object, the 15 expression being obtained by substituting the expression (3) into the expression (14) and by substituting respective numerical values to the expression. The distance data can be also obtained by referring to a previously prepared look-up-table between the observed quantity and the distance a to the object with regard to the predetermined frequency k. Therefore, the distance data is calculated by using the expression based on the two output image out of focus which are obtained by the above filtering step ST2.
16 2p 786 38 As described above, the distance measurement operation can be performed at high speed effectively and easily by using monocular.
Figure 3 shows a principle configuration of the present invention. The image input unit 1 includes a lens and a physical image receiving plane. An observed image of the object is obtained on the physical image receiving plane through a lens thereby outputting the observed image.
The calculation unit 2 includes the calculation control unit 2-1 and a memory unit 2-2 and performs a calculation of an output image from the observed image input from the image input unit 1 using the expression (6) and (7) and obtains the distance a up to the object from the output image using expressions (14) and (3).
When the image input unit 1 outputs an analog signal, the calculation processing unit 2 should be able to deal with an analog signal and thus the calculation control unit 2-1 and memory unit 2-2 should process or memorize the analog signal. When an A/D converter is provided between the image input unit 1 and the calculation processing unit 2, the calculation processing unit 2 can deal with the digital signal.
1~ 2o~as38 ~~
The expressions (6) and (7) represent basic expressions for providing the digital signals. The physical meaning and the possible expansion of the expressions (6) and (7) will be explained as follows.
The expressions (6) and (7) represent the Gabor function for the input signal Is (x) and Is' (x).
Namely, a Gaussian function is obtained for an input signal, namely, a center of information of a field and, in other words, a weight according to a Gaussian distribution is applied to the input signal and a Fourier transformation is applied to the field with regard to a frequency k in a positive or negative direction of the center of the information of the field.
In the video signal, namely, the information of the field is generally two dimensional. In this invention, however, a distance from an object to the image receiving apparatus (camera) is obtained by using only one dimension of information along a direct line in a direction perpendicular to the direction of the distance (i.e. the z axis), namely, in the directions of the x axis and y axis. In the present embodiment, the distance is obtained by using only information in the direction of the x axis, but the information in the direction of the y axis may be also used. Further, the information in the direction of both the x axis and the y axis may be used to obtain the Gabor function and the information with a larger signal energy may be utilized.
v is a parameter for representing a function of a weighting operation, namely a distribution function for providing the size of a filter. The o is determined such that a calculation for obtaining a limit of the image size is used for obtaining the distance. Especially, when a predetermined image element is considered, an inter-relation is strong in the vicinity of the image element and becomes weak in a location remote from the image element. Therefore the a is controlled such that the process is performed by considering only the vicinity of the image element.
Q may be infinity (~) and in this case a calculation is conducted under the assumption that exp(-x2/Q2)=1, namely, the calculation is conducted with the weight made constant.
Figure 4 shows a block diagram of a schematic structure of the distance measuring apparatus for carrying out the distance measuring operation of the present invention. The distance measuring apparatus obtains the distance data by using monocular based on the detailed description above. The symbols and codes used for the explanation of the principle in Figure 1 are also used in the following explanation for the same parts and the same contents.
In Figure 4, the image input unit 6 comprises a television camera including lens L and image receiving plane S, for example. The observed image Is of the object, measured and with a blur at a certain position on the same optical axis, and the observed image Is' at a position moved from the certain position by a minute distance ~z are provided through the lens L on the image taking plane S.
Two observed images Is and Is' have blurs whose quantities are different. The two observed images obtained on the image receiving plane S are converted to digital form through a transducing unit 7 comprising A/D converter, for example, thereby being output to the calculation processing unit 3.
The calculation processing unit 3 comprises well known CPU3a, ROM3b and RAM3c. A program for controlling the CPU3a is written into ROM3b. The CPU3a receives necessary data from the observed image input unit 6 in accordance with the program stored in ROM3b and performs an calculation process by exchanging the data between CPU3a and RAM3c. CPU3a stores the result of the processing, namely, the distance data, in RAM3c, if necessary, or outputs it to an external device. CPU3a performs the following operation in accordance with the capacity of the system.
When the processing speed of CPU3a is higher than the conversion speed of the A/D converter provided in the transducing unit 7, the resulting data obtained by the A/D conversion is stored in the image memory region 3c1 of RAM3c sequentially in units of an image element and performs the above described calculation, 10 thereby providing Fs and Fs'. Then it obtains the distance based on the values of Fs, Fs' and pz .
When the processing speed of CPU3a is slower than the conversion speed of the A/D converter provided in the transducing unit 7, the data resulting 15 from the A/D converter is stored in the image memory region 3c1 in RAM3c sequentially in units of image element. For example, when the data transmission speed of CPU3a is further slowed and CPU3a cannot store the data in RAM3c, the DMAC may be provided.
20 Thereafter, Fs and Fs' are obtained as previously.
The distance a can be obtained by using these values Fs, Fs' and dz.
The case of the high processing speed of the CPU
will be explained as follows. CPU3a performs the calculation represented by expression (6) with regard 2078638 y to the observed image input from the image input unit 6 and obtains the output images Fs and Fs', thereby storing them in image memory 3c1 in RAM3c.
Further, CPU3a substitute the respective numerical value of the predetermined frequency k, the focus distance f of lens L and the radius r of lens L
and the minute movement quantity Az of the camera and the output images Fs and Fs' into the expression (14) representing the relation between the above distance images Fs and Fs' and the blur radius s and the expression (3) representing the blur radius s and the distance a from the lens L to the object M and calculates the distance a up to the object based on the expression representing the relation between the observed quantity and the distance a which is obtained from the expression (14) and (3). Then it stores the distance data in RAM3c and further outputs it to the external device.
The distance data is obtained from the look-up-table, representing the relation between the value obtained by calculating the left side of the expression (14) using the values (Fs, Fs' and ~z) obtained from the observation, and the value obtained by an actual measurement of the distance up to the 2 5 ob j ect .
The distance data is obtained by using the look-up-table 3c2 which is stored in RAM3c. CPU3a calculates the left side of expression (14) from the value of the output image Fs and Fs' obtained by applying the calculation to the discretionary observed images Is and Is' and the value of the minute movement quantity pz and obtains the distance data by referring to the look-up-table based on these values.
Figure 5 shows a general flowchart of the process of the distance measuring method of the present invention which is performed by CPU3a in the distance measuring apparatus shown in Figure 4. The concrete operation of the distance measuring apparatus will be explained as follows.
Firstly, the process performs step S1 for obtaining the observed image Is of the object M at a certain position on the optical axis on the image receiving plane S through lens L by using the image input unit 6 such as a television camera and for inputting the observed image Is in the operation process unit 3 via the transducing unit 7. The observed image Is with a blur having a radius s is considered as obtained by applying a blur filter to the image Io in focus and can be represented by expression (4). Conversely, expression (4) can 23 2 O 7 ~ 6 3 8 determine the condition of the blur filter p(x-x'), namely, determines that the observed image can be obtained on the optical axis, thereby performing an inverse conversion. Then, the image Io in focus can be obtained from the observed image Is.
Next, the process performs step S2 for applying the filter (Labor function) defined by expression (6) to the observed image Is, obtaining the output image Fs with regard to the predetermined frequency k and for storing the output image Fs in the image memory 3cl in RAM 3c.
The image output Fs can be represented by the expression (6) as described above and is the output obtained when the center of the filter is matched with the origin of the filter.
Further, the process performs step S3 for moving the television camera on the same optical axis by a minute distance Oz and for obtaining the observed image Is' at a position apart from the original position by the minute distance Az, thereby receiving the observed image Is'. The observed image Is' is the image with the blur whose radius is changed from s to s', and which is different from that of the image observed Is.
Then the process performs step S4 for applying 20~8s38 the filter, (Labor function) to the observed image Is' as in step S2 and for obtaining the output image Fs' for the predetermined frequency k and storing it in the image memory 3c1 in RAM3c. Thus, the image output Fs' can be represented by expression (11).
Next, when the distance data a relating to the distance up to the object is calculated, the process advances to step S5. The process substitutes respective numerical values of the output image Fs and Fs', minute movement quantity ~z, a focus distance f of lens L, predetermined frequency k and radius r of lens L into the expression derived by substituting the expression (3) into expression (14), namely, the expression giving the relation between the quantity obtained by the observed and the distance up to the ob j ect .
On the other hand, when the distance data up to the object is obtained by referring to the look-up-table, step S4 advances to step S6. The process refers to the look-up-table using the output image Fs and Fs' of the left side of expression (14) and the numerical value of the minute movement quantity ~z and the actual distance data a up to the object.
With regard to the look-up-table used in step S6, a method of preparing it through a calculation or a method of preparing it by using an actual experimental calibration are considered. Either method can provide the distance data relating to the distance a up to the object.
5 The experimental calibration is used in the conventional distance measuring method of utilizing stereo vision, as the optical members such as lens of the camera have different properties and thus is not intrinsic to the present invention.
10 As described above, in the present embodiment, the observed image obtained at a given position and the observed image obtained at a small distance from the given position on the same optical axis have different blur quantities at the same point on the 15 image receiving plane. The filter (Labor function) designated by expression (6) is applied to two input observed images with different blurs. Further, respective output images are obtained for the predetermined frequencies in a frequency range and the 20 expression (14) relating to the relation between the observed quantity and the radius of the blur is obtained. The process calculates the data relating to the distance up to the object by using a method of calculating the distance by substituting respective 25 values in expressions (14) and (3) or by referring to the previously prepared look-up-table between the observed quantity and the distance up to the ob,~ect.
Figure 6 shows a configuration of the second embodiment of the present invention in which the look-up-table is used. The relation between the observed value of the left side of the expression (6) for corresponding predetermined frequencies k and the actual value is measured upon an execution of the calibration and is stored in the look-up-table. Then, when the discretional image data is input, the look-up-table obtained by calculating the left side of the expression based on the results Fs and Fs' of the filtering and the movement quantity ~z is referred to, thereby providing the distance data. Namely, as shown in Figure 6, the image data GA and GB are input to the filter operation units FXA and FXB, thereby providing the output Fs and Fs'. The filter operation units FXA and FXB comprise the same circuit but their input are different. As a result, the input data is stored in the table by using the Fs and Fs' and the distance pz, by which the lens or camera are moved upon inputting these image data as the address of the table . Therefore, the table THL outputs the distance data representing the actual distance previously obtained upon an execution of the calibration.
This embodiment does not require large quantities of calculations for finding the corresponding points between images input two television cameras which is required by the conventional distance measuring method of using stereo vision and thus measures the distance better than a human being, who makes defective measurements. It performs rapidly with a simple construction using monocular.
The above embodiment was described using a one dimensional image on an optical axis. Three dimensioned distance image can be also attained by applying the filtering shown in expression (6) to the entire image and calculating the related expression relating to the observed quantity and the distance up to the object for each respective image element in accordance with the expression (14) and (3) or by calculating the distance data by referring to the look-up-table. The situation in external space can be determined based on this distance image. For example, in the research field of computer vision, extensive research for enabling computers to understand an object as a human being does and for determining the position in relation to the objects has been extensively performed. However, there is a problem in the means of obtaining the distance image in three dimensional space and thus the positional relation of the objects and the structure in the three dimensional space can not be easily understood. The present invention can obtain the distance data and the three dimensional environment with ease and thus enables such an environment to be understood. It is possible to derive the distance image with a.high rate such as a video rate. A special solution of the distance image is about its distribution.
The basic principle of the present invention resides in measuring distance based on two images through a filtering process. The filtering process limits the degree of the distribution of the image by determining the filter size to be a. Accordingly, the sigma affects on the obtained distance image data and the accuracy of the distance image data depends on the distribution a. In other words, the actual distance differs from the distance obtained according to the present invention by the distribution Q in terms of accuracy. when the method of the present invention is compared with the sensitivity of the vision of the human beings, the human being cannot measure the distance with high accuracy but, can understand the environment of the works better than the present method does. Therefore, it can be said that the 2p7863$
present invention can achieve a sufficient accuracy as the distance measured by the present method differs from the actual distance nearly by the distribution a.
In the above description, the television camera is moved on the same optical axis. Instead of moving the television camera, the lens or the image receiving plane within the television camera may be moved on the same optical axis or a plurality of moving planes are positioned on different points within the television camera, thereby calculating the distance data using the same process as described above.
Figure 7 shows a schematic configuration of a first example of the image input unit 6 of the distance measuring apparatus shown in Figure 4.
In Figure 7, the image input unit 11 is provided with a lens moving structure for sliding the lens 12 provided in the lens cylinder 13 along the optical axis 12a of the lens 12 within the lens cylinder 13 and for changing the distance between the image receiving plane 15 provided by the lens 12 and the image receiving plane 15 provided in the camera body 14.
By controlling the lens moving structure, respective observed images Is and Is' with different blurs can be obtained on the image receiving plane 15 by moving the lens from its position shown by the solid line to the position shown by a chained line.
Two observed images are input to the calculation processing unit 3 not shown and the previously described series of processes is carried out by the calculation processing unit 3, thereby enabling the distance data to be obtained with ease by using monocular.
Figure 8 shows the distance measuring apparatus shown in Figure 4 according to the present invention and designates a schematic drawing of a structure of the second example of the image input unit 6.
As shown in Figure 8, the image input unit 21 is provided with a moving structure for the image receiving plane for moving the image receiving plane provided in the camera body 24 along an optical axis 22a of the lens 22 provided in the lens cylinder 23 in the camera body 24, thereby changing the distance between the image receiving plane 25 and lens 20 22.
By controlling the image receiving plane movement structure, respective observed images Is and Is' lwith different blurs are obtained on the image receiving plane 25 with regard to a position shown by a solid 25 line before the image receiving plane 25 is moved and a position shown by a chained line after the image receiving plane 25 is moved. These two observed images are input to the calculation processing unit 3 not shown and the above described process is carried out by the calculation processing unit 3, thereby enabling the distance data to be obtained using monocular with ease.
Figure 9 shows the distance measuring apparatus shown in Figure 4 and a schematic diagram of a structure of the third example of the image input unit 6.
As shown in Figure 9, the image input unit 31 is provided with a structure 36 to move the camera body.
This fixes the distance between the image receiving plane 35 provided in the camera body 34 and the lens 32 provided in the lens cylinder 33 and moves the camera body 34 along an optical axis 32a of the lens 32.
By controlling the moving structure 36 for the camera body, respective observed images Is and Is' with different blurs are obtained on the image receiving plane 35 relative to the position before the camera body is moved and the position after the camera body 34 is moved. These two observed images are input to the calculation processing unit 3 not shown and the 32 2 0 ~ a s 3 8 above described process is carried out, thereby enabling the distance data to be obtained with ease using monocular.
Figure 10 shows a schematic diagram of a structure of the fourth example of the image input unit 6 in the distance measuring apparatus shown in Figure 4.
As shown in Figure 10, the image input unit 41 is provided with a spectroscope 46 such as a prism and respective image receiving plane 45b and 45c are provided positions with different distances from the spectroscope within the camera body 44 and the optical axis 42a of the lens 42 provided in the lens cylinder 43 is divided into two optical axes 42b and 42c by the spectroscope 46.
It is not necessary to move the lens 42 in the lens cylinder 43 and also not necessary to move the image receiving planes 45b and 45c in the camera body 44. Further, it is not necessary to move the camera body 44. The lens 42 and the image receiving planes 44b and 44c are in fixed positions and the distance between the spectroscope 46 and two image receiving planes 45b and 45c are made different and respective observed images Is and Is' with different blurs are obtained on the respective image receiving planes 45b and 45c. These two observed images are input to the calculation processing unit 3 not shown and the above described process is carried out, thereby enabling the distance data to be obtained with ease using a single eye sight. Further, in this embodiment, moving structures such as a lens and image receiving plane are not needed and thus in the present example the two observed images are simultaneously obtained by using the same optical system and thus the deterioration of accuracy which is caused movement structures can be prevented.
The method of moving the lens, image receiving plane and the camera body along an optical axis without changing the focus distance f of the lens and the radius r of the lens can satisfy the principle expression (3) where either the distance a between the lens and the object, or the distance f between the lens and the image receiving plane or both are changed. Accordingly, the object measuring method of the present invention can be applied in respective cases where the lens of the camera is moved, the image receiving plane is moved in the camera body, the entire camera is moved and a plurality of image receiving planes are provided in different position in the camera body. If the method satisfies the 34 2 ~ 7 ~ 6 3 8 principle expression (3), then it also satisfies expression (14) and thus the distance measuring method of the present invention can be applied to the respective methods of movement.
Figure 11 shows a block diagram of the third embodiment of the distance measuring apparatus according to the present invention in which an operation of taking an image is performed twice. The image input unit in this embodiment uses the structures shown in Figures 7, 8 and 9.
The analog image taken at the first point in time is converted to a digital signal by the A/D conversion circuit 51. The digital image is input to the filter operation circuit 52 and the result Fs is stored in the image memory 53.
Sequentially, after the movement of the camera, lens, or image receiving plane is conducted by Oz, the analog image taken at the point in second time is converted to the digital image by the A/D converter 51 and then the result is input to the filter operation circuit 52, thereby-providing Fs'. The result Fs' is stored in the image memory 54. The Fs and Fs' obtained by these sequential operations and the movement distance ~z are input to the table TBL, thereby providing the distance data from the table 35 2 ~ ~ 8 6 3 8 TBL.
Figure 12 shows the fourth embodiment of the distance measuring apparatus according to the present invention in which the images are received and processed simultaneously at a single point in time.
In the third embodiment, the movement by the distance Oz is required but in the fourth embodiment two analog images obtained by two image receiving planes 45b and 45c as shown in Figure 10 are received and processed at a single point in time, thereby providing the distance.
The analog images obtained from two image receiving planes 45b and 45c are input to the A/D
conversion circuits 61 and 62. The digital image converted to the digital data by the A/D conversion circuits 61 and 62 are input to the filter operation circuits 63 and 64, thereby providing Fs and Fs'.
These results are stored in the image memory 65 and 66 to provide an address of the table THL. In this structure, ~z is fixed in the above operation, and the analog image of the image receiving planes which are different by a distance ~z can be taken at the single point in time. The A/D conversion operation and the filtering operation can be also simultaneously Performed with regard to the two images, thereby providing the distance in half the time required for the third embodiment.
APPARATUS
Background of the Invention The present invention relates to a distance measuring method and a distance measuring apparatus.
An input portion of the apparatus forms a three dimensional image of an object and processes it, thereby obtaining distance data from the object to the lens. The input unit uses, for example, a television camera to form a two dimensional image of the object through a lens on a image focusing plane. More particularly this distance measuring method and distance measuring apparatus obtains distance data with ease using monocular based on two observed images with different blurs provided at positions which are different by a minute distance from each other along the same optical axis.
Several attempts for providing a distance measuring method have been made but a method similar to a human vision system has not been provided although needs for such a computer vision system are very great in various industries. At present, the distance measuring methods used for an automatic apparatus require complicated techniques and structure. This is especially true when the distance measuring method is used as a computer vision system for an autonomous vehicle. It is necessary for the vehicle to instantaneously and automatically judge its outside situation in order to autonomously operate to a destination and to stop or avoid an obstacle in its way. The distance measuring method to be used for such an application needs to instantaneously judge with accuracy the distance up to an obstacle that has been detected on its path.
However, a method for obtaining complete distance image data has not been available. It is apparent that an visual sensor with distance sensitivity in three dimensional space like a human vision system will be required by various industries.
Recently, a method of obtaining distance image data by performing computer processing of an image input from an image input apparatus such as a television camera has been utilized for visual inspection or control in the field of factory automation.
A method of measuring distance up to an object based on a principle of triangular measurement includes an image processing method of searching corresponding points between two input images of binocular by using plurality of television cameras and measuring distances between the television cameras and the corresponding points, and parallax between the corresponding points.
On the other hand, a distance measuring method which does not use image processing, but measures distance up to an object by using a pointing device equipped with a laser has been widely used in the industrial field of public engineering works and architecture. A range sensor and a range finder for obtaining distance data based on two dimensional images by using laser scanning have been on the market.
fihe former method using image processing searches corresponding picture elements in each image based on a calculation by using an image input from at least two television cameras. For this method of utilizing a stereo view, a relative method or a dynamic programming method has been proposed for searching corresponding points.
However, although this method can recognize a situation in three dimensional space, it requires a large amount of data as it scans the whole screen and thus it takes time to calculate distance data. It especially takes much time to obtain the distance of a 20 786 ~8 part such as at an edge at which the light and shade variation of the image is large and is easily affected by noise, thereby having the defect in that the complete distance image data cannot be obtained.
The latter method not using image processing can.
extract two dimensional distance data but has the defects in that a high resolution of data is difficult and that it takes time to obtain three dim~~ensional distance data.
Summary of the Invention The present invention aims at obtaining image data by processing input images from an image input unit and calculating distance data based on two different observed images along the same optical axis with different blur by using monocular.
The present invention applies respective filters (Labor function) to two input observed images Is and Is' with different blurs and obtains respective output images Fs and Fs' for respective predetermined frequencies k in a frequency range. The present invention utilizes an expression representing a relation between the output images Fs and Fs' and their radius s of blur and an expression representing a relation between a radius s of the blur and distance a from a lens to an object and substitutes numerical values into the respective variables of the expressions.
Further, the present invention refers to a look-up-table between the observed quantity and the distance up to the 5 object, the table being previously prepared through an experimental calibration. Thus the present invention can calculate the distance data up to the object.
The invention may be summarized, according to a first broad aspect, as a method of obtaining distance data from an object to a lens by obtaining an image of the object by an image input unit for forming an image of the object through the lens on an image receiving plane and then performing a process, comprising the steps of: inputting two observed images Is and Is' with different blurs from the said image input unit by making a position of the lens, a position of the image receiving plane or positions of both of them with regard to the object different by a minute distance Oz on the same optical axis; performing a filtering process by applying filtering to the said two input observed images Is and Is', thereby providing two output images Fs and Fs'; and calculating the distance a up to the object by using two output images Fs and Fs' obtained by the filtering process based on the relation between said two output images Fs and Fs' and a radius s of the blur and the relation between the radius s of the blur and the distance a from the lens up to the object.
According to a second broad aspect, the invention provides a distance measuring apparatus comprising: an image "~' 28151-79 207863$
5a input means comprising a lens and an image receiving plane for obtaining the observed image of the object on the image receiving plane and outputting the observed image; a first operating means for applying a filtering process to the first image input from the said image input means thereby performing a Fourier transformation for a respective predetermined frequency k; a second operation means for applying the same operation as that of the first operation means to the second observed image which is obtained from the image input means when at least one of the lens position, the position of the image receiving plane, and the position of the image input means is moved; and a third operation means for obtaining the distance up to the object based on the results of the first operating means for the first observed image, the result of the second operation means for the second observed image and the distance in which the said at least one of the lens, the image receiving plane and the image input means are moved.
Brief Description of the Drawings One skilled in the art can easily understand additional features and objects of this invention from the description of the preferred embodiments and some of the attached drawings. In the drawings:
Figure 1 shows a basic optical system using the principle of the distance measuring method according to the present invention;
Figure 2 shows a flowchart of the principle of the distance measuring method according to the present invention;
Figure 3 shows a block diagram of the principle of 5b the present invention;
Figure 4 shows a block diagram of the structure of a first embodiment of a distance measuring apparatus according to the present invention;
2o7ss38 Figure 5 shows a general flowchart of the process of measuring distance according to the present invention;
Figure 6 shows a configuration of a block diagram of a second embodiment using a look-up-table defined by the present invention;
Figure 7 shows a diagram of the first example of an image input unit of the distance measuring.
apparatus according to the present invention;
Figure 8 shows a diagram of the second example of the image input unit of the distance measuring apparatus according to the present invention;
Figure 9 shows the third example of the image input unit of the distance measuring apparatus according to the present invention;
Figure 10 shows the forth example of the image input unit of the distance measuring apparatus according to the present invention;
Figure 11 shows a block diagram of the third embodiment of the distance measuring apparatus for processing two images obtained at different points in time; and Figure 12 shows the fourth example of the distance measuring apparatus for receiving two image data at a single point in time period.
Preferred Embodiments First, the principle of distance measurement according to the present invention will be explained.
Figure 1 shows the principle of the present invention wherein the in-focus distance of lens L . f, the radius of lens L . r, the distance from the center of lens L up to the object M . a and the distance from the center of lens L to an image P . b. A lens formula is provided as follows;
1/a+1/b = 1/f ... (1) therefore, the object M located at a preceding position with distance a from the center of lens L
focuses its image P at a succeeding position with a distance b = af/(a-f) from the center of lens L. The distance from the lens L to the object is to be determined. For example, when a focus of a television camera is matched with an infinity point, either the lens L or the image receiving plane S is moved. Then, the image of the object located at the succeeding position with distance a from the center of the lens provides the following blur;
r(b-f)/b = rf/a ... (2) The radius s of the blur is provided based on the geometrical resemblance relation as follows.
s = rf/a ... (3) Where the radius s of the blur is defined as a radius s on the image receiving plane S when the object, namely, a point light source, is located at distance a from the center of the lens L.
For the sake of simplicity, a one dimensional image located on the optical axis is assumed. In this case, the x axis and y axis are located on the image receiving plane S and the z axis is in the direction of the optical axis, thereby providing three dimensional coordinates. Where an image which is in focus (unknown image):IO(x) and an image which is out of focus with blur of radius s (of an observed image):Is (x) an image with blur can be obtained by applying a filtering process to the image which is in focus.
Is(x)= p(x-x')Io(x')dx' ... (4) 2p78638 The function p (x-x') represented by the following expression is a filter function.
p(x-x')=1/2s if ~x-x'~Ss =0 if ~x-x'~>s ... (5) The filter function corresponds to a blur filter, it gives a value of 0 outside the diameter 2s of the blur and gives the value of the function inside the diameter 2s of the blur. If the inverse transformation is performed in the expression (4), an image in focus can be obtained from an image with a blur.
Therefore, a space filter operation ( Gabor function) is applied to the observed image Is (x).
Namely, a filtering process is applied to the blurred image, thereby eliminating blur to provide the image which is in focus.
Fs= exp(-x2/a2)exp(ikx)Is(x)dx ... (6) Where exp(-x2/a2) represents a Gaussian function and exp(ikx) represents a kernel of a Fourier transformation. This expression (6) represents a Gaussian filter applied to the observed image Is, thereby performing a Fourier transformation. k represents a predetermined frequency and Fs represents an output when the center of the filter is positioned at the origin of the coordinates formed by x, y, and z 5 axes. Namely, the output Fs is the data obtained by performing a Fourier transformation of the image for the corresponding predetermined frequency k with regard to the both sides of the origin of the coordinates when the optical axis is provided through 10 the origin.
The following expression is obtained from expression (4).
Fs= exp(-x2/a2)exp(ikx)p(x-x')Io(x')dxdx' ... (7) An integral operation is performed with regard to x.
exp(-x2/Q2)exp(ikx)p(x-x')dx x: ; s =(1/2s) exp(-x2/a2)exp(ikx)dx Q » s x'-s x+s ''=.(1/2s)exp(-x'2/a2) exp(ikx)dx x-s =(1/2s)exp(-x'2/Q2)2sin(ks)exp(ikx')/k =(sin(ks)/ks)exp(-x'2/Q2)exp(ikx') ... (8) Therefore, the following expression is obtained.
Fs=(sin(ks)/ks) exp(-x'2/a2)exp(ikx')Io(x')dx' ... (9) Next, the television camera is moved towards the object M by ~Z along the optical axis, thereby taking.
a photograph of an observed image Is' (x). Then, the radius of the blur is changed to s' and is expressed by the following expression.
s' =rf/ ( a-,oz )'--.( rf/a ) ( 1+~z/a )=s+~z/a=s+s2pz/rf ... (10) When the same filter is applied to the second observed image Is'(x), the following expression (11) is obtained in the same manner as in expression (9).
Fs'=(sin(ks')/ks') exp(-x'2/Q2)exp(ikx') Io(x')dx' ... (11) In order to delete the integral term including the unknown quantity, the expression (9) is subtracted from expression (11) and is further divided by expression (9).
sin(ks')/ks'-sin(ks)/ks (Fs'-Fs)/Fs=
sin(ks)/ks ... (12) The left side of expression (12) represents a definition of a differential equation, assuming that;
ks'-ks=ks2az/rf << 1 ... (13) namely, the amount of the movement of the camera is made sufficiently small. Both sides of expression (12) are divided by pz, thus providing expression (14);
(Fs'-Fs) ks cos(ks)-sinks) s (~z)Fs sinks) fr ... (14) The expression (14) represents a relation between the observed quantity (the left side of expression (14)) and the radius s of the blur. In expression (14), k is a predetermined frequency, namely, a constant number, f and t are known, Fs, Fs' and ~z are values obtained by processing an image when the camera is moved.
When the television camera is moved along the optical axis, images at two points are input to the television camera and a filtering process represented by expression (6) is applied to the respective 13 ~ ~ 7 8 6 3 8 images. A calculation of expression (14) is also performed, thereby obtaining the radius s of the blur. When the expression (3) is substituted into the expression (14), the expression representing the relation between the observed quantity (the left side of the expression) and the distance a up to the object is derived. The respective numerical values of Fs, Fs', Oz, k, f, and r are substituted into the expression and the calculation is performed. Thus, the distance a up to the object is obtained by the calculation.
As is explained above, a distance a can be obtained by carrying out a calculation using the expression which represents the relation between the observed quantity (the left side) and the distance a up to the object and is obtained by manipulating expression (14). The distance a can be also obtained by referring to a look-up-table. The look-up-table is previously prepared for corresponding predetermined frequencies k with regard to the calculation value of the left side of the expression (14) and the distance a up to the object which is actually measured upon an operation of the calculation. The look-up-table is referred to using the input of discretionary image data, thereby obtaining as output the distance a from the input value obtained by calculating the observed quantity on the left side of expression (14).
The distance measuring method of the present invention comprises the following processes. Figure 2 shows a flowchart of the principle of operation of the present invention.
Step ST1 shows an image input process of receiving two observed images Is and Is' having different quantities of blur from the image input unit bY changing the position of the lens, the position of the image receiving plane or the positions of both the lens and the image receiving plane by a minute distance pz from the object on the same axis.
Step ST2 shows filtering process for obtaining two output images Fs and Fs' by applying the filtering process shown in expressions (6) and (7) to the input observed images Is and Is'.
Step ST3 shows a process of calculating distance data for obtaining the distance a up to the object by using two output images obtained by the filtering process based on the expression (3) representing the relation between the radius s of the blur and the distance a from the lens up to the object.
When the distance measuring method of the present invention starts, the process carries out step ST1 for inputting from the image input unit 6 two observed images- which are obtained at positions different from each other by a minute distance on the same optical axis and have different quantities of blur.
5 The process carries out filtering step ST2 for applying a Gabor function to two input observed images and for obtaining two output images in focus with regard to the predetermined frequency. The step ST2 is performed by referring to the calculation shown by 10 the expression (6).
Next, the process carries out step ST3 for calculating the distance data based on the expression representing the relation between the observed quantity and the distance a up to the object, the 15 expression being obtained by substituting the expression (3) into the expression (14) and by substituting respective numerical values to the expression. The distance data can be also obtained by referring to a previously prepared look-up-table between the observed quantity and the distance a to the object with regard to the predetermined frequency k. Therefore, the distance data is calculated by using the expression based on the two output image out of focus which are obtained by the above filtering step ST2.
16 2p 786 38 As described above, the distance measurement operation can be performed at high speed effectively and easily by using monocular.
Figure 3 shows a principle configuration of the present invention. The image input unit 1 includes a lens and a physical image receiving plane. An observed image of the object is obtained on the physical image receiving plane through a lens thereby outputting the observed image.
The calculation unit 2 includes the calculation control unit 2-1 and a memory unit 2-2 and performs a calculation of an output image from the observed image input from the image input unit 1 using the expression (6) and (7) and obtains the distance a up to the object from the output image using expressions (14) and (3).
When the image input unit 1 outputs an analog signal, the calculation processing unit 2 should be able to deal with an analog signal and thus the calculation control unit 2-1 and memory unit 2-2 should process or memorize the analog signal. When an A/D converter is provided between the image input unit 1 and the calculation processing unit 2, the calculation processing unit 2 can deal with the digital signal.
1~ 2o~as38 ~~
The expressions (6) and (7) represent basic expressions for providing the digital signals. The physical meaning and the possible expansion of the expressions (6) and (7) will be explained as follows.
The expressions (6) and (7) represent the Gabor function for the input signal Is (x) and Is' (x).
Namely, a Gaussian function is obtained for an input signal, namely, a center of information of a field and, in other words, a weight according to a Gaussian distribution is applied to the input signal and a Fourier transformation is applied to the field with regard to a frequency k in a positive or negative direction of the center of the information of the field.
In the video signal, namely, the information of the field is generally two dimensional. In this invention, however, a distance from an object to the image receiving apparatus (camera) is obtained by using only one dimension of information along a direct line in a direction perpendicular to the direction of the distance (i.e. the z axis), namely, in the directions of the x axis and y axis. In the present embodiment, the distance is obtained by using only information in the direction of the x axis, but the information in the direction of the y axis may be also used. Further, the information in the direction of both the x axis and the y axis may be used to obtain the Gabor function and the information with a larger signal energy may be utilized.
v is a parameter for representing a function of a weighting operation, namely a distribution function for providing the size of a filter. The o is determined such that a calculation for obtaining a limit of the image size is used for obtaining the distance. Especially, when a predetermined image element is considered, an inter-relation is strong in the vicinity of the image element and becomes weak in a location remote from the image element. Therefore the a is controlled such that the process is performed by considering only the vicinity of the image element.
Q may be infinity (~) and in this case a calculation is conducted under the assumption that exp(-x2/Q2)=1, namely, the calculation is conducted with the weight made constant.
Figure 4 shows a block diagram of a schematic structure of the distance measuring apparatus for carrying out the distance measuring operation of the present invention. The distance measuring apparatus obtains the distance data by using monocular based on the detailed description above. The symbols and codes used for the explanation of the principle in Figure 1 are also used in the following explanation for the same parts and the same contents.
In Figure 4, the image input unit 6 comprises a television camera including lens L and image receiving plane S, for example. The observed image Is of the object, measured and with a blur at a certain position on the same optical axis, and the observed image Is' at a position moved from the certain position by a minute distance ~z are provided through the lens L on the image taking plane S.
Two observed images Is and Is' have blurs whose quantities are different. The two observed images obtained on the image receiving plane S are converted to digital form through a transducing unit 7 comprising A/D converter, for example, thereby being output to the calculation processing unit 3.
The calculation processing unit 3 comprises well known CPU3a, ROM3b and RAM3c. A program for controlling the CPU3a is written into ROM3b. The CPU3a receives necessary data from the observed image input unit 6 in accordance with the program stored in ROM3b and performs an calculation process by exchanging the data between CPU3a and RAM3c. CPU3a stores the result of the processing, namely, the distance data, in RAM3c, if necessary, or outputs it to an external device. CPU3a performs the following operation in accordance with the capacity of the system.
When the processing speed of CPU3a is higher than the conversion speed of the A/D converter provided in the transducing unit 7, the resulting data obtained by the A/D conversion is stored in the image memory region 3c1 of RAM3c sequentially in units of an image element and performs the above described calculation, 10 thereby providing Fs and Fs'. Then it obtains the distance based on the values of Fs, Fs' and pz .
When the processing speed of CPU3a is slower than the conversion speed of the A/D converter provided in the transducing unit 7, the data resulting 15 from the A/D converter is stored in the image memory region 3c1 in RAM3c sequentially in units of image element. For example, when the data transmission speed of CPU3a is further slowed and CPU3a cannot store the data in RAM3c, the DMAC may be provided.
20 Thereafter, Fs and Fs' are obtained as previously.
The distance a can be obtained by using these values Fs, Fs' and dz.
The case of the high processing speed of the CPU
will be explained as follows. CPU3a performs the calculation represented by expression (6) with regard 2078638 y to the observed image input from the image input unit 6 and obtains the output images Fs and Fs', thereby storing them in image memory 3c1 in RAM3c.
Further, CPU3a substitute the respective numerical value of the predetermined frequency k, the focus distance f of lens L and the radius r of lens L
and the minute movement quantity Az of the camera and the output images Fs and Fs' into the expression (14) representing the relation between the above distance images Fs and Fs' and the blur radius s and the expression (3) representing the blur radius s and the distance a from the lens L to the object M and calculates the distance a up to the object based on the expression representing the relation between the observed quantity and the distance a which is obtained from the expression (14) and (3). Then it stores the distance data in RAM3c and further outputs it to the external device.
The distance data is obtained from the look-up-table, representing the relation between the value obtained by calculating the left side of the expression (14) using the values (Fs, Fs' and ~z) obtained from the observation, and the value obtained by an actual measurement of the distance up to the 2 5 ob j ect .
The distance data is obtained by using the look-up-table 3c2 which is stored in RAM3c. CPU3a calculates the left side of expression (14) from the value of the output image Fs and Fs' obtained by applying the calculation to the discretionary observed images Is and Is' and the value of the minute movement quantity pz and obtains the distance data by referring to the look-up-table based on these values.
Figure 5 shows a general flowchart of the process of the distance measuring method of the present invention which is performed by CPU3a in the distance measuring apparatus shown in Figure 4. The concrete operation of the distance measuring apparatus will be explained as follows.
Firstly, the process performs step S1 for obtaining the observed image Is of the object M at a certain position on the optical axis on the image receiving plane S through lens L by using the image input unit 6 such as a television camera and for inputting the observed image Is in the operation process unit 3 via the transducing unit 7. The observed image Is with a blur having a radius s is considered as obtained by applying a blur filter to the image Io in focus and can be represented by expression (4). Conversely, expression (4) can 23 2 O 7 ~ 6 3 8 determine the condition of the blur filter p(x-x'), namely, determines that the observed image can be obtained on the optical axis, thereby performing an inverse conversion. Then, the image Io in focus can be obtained from the observed image Is.
Next, the process performs step S2 for applying the filter (Labor function) defined by expression (6) to the observed image Is, obtaining the output image Fs with regard to the predetermined frequency k and for storing the output image Fs in the image memory 3cl in RAM 3c.
The image output Fs can be represented by the expression (6) as described above and is the output obtained when the center of the filter is matched with the origin of the filter.
Further, the process performs step S3 for moving the television camera on the same optical axis by a minute distance Oz and for obtaining the observed image Is' at a position apart from the original position by the minute distance Az, thereby receiving the observed image Is'. The observed image Is' is the image with the blur whose radius is changed from s to s', and which is different from that of the image observed Is.
Then the process performs step S4 for applying 20~8s38 the filter, (Labor function) to the observed image Is' as in step S2 and for obtaining the output image Fs' for the predetermined frequency k and storing it in the image memory 3c1 in RAM3c. Thus, the image output Fs' can be represented by expression (11).
Next, when the distance data a relating to the distance up to the object is calculated, the process advances to step S5. The process substitutes respective numerical values of the output image Fs and Fs', minute movement quantity ~z, a focus distance f of lens L, predetermined frequency k and radius r of lens L into the expression derived by substituting the expression (3) into expression (14), namely, the expression giving the relation between the quantity obtained by the observed and the distance up to the ob j ect .
On the other hand, when the distance data up to the object is obtained by referring to the look-up-table, step S4 advances to step S6. The process refers to the look-up-table using the output image Fs and Fs' of the left side of expression (14) and the numerical value of the minute movement quantity ~z and the actual distance data a up to the object.
With regard to the look-up-table used in step S6, a method of preparing it through a calculation or a method of preparing it by using an actual experimental calibration are considered. Either method can provide the distance data relating to the distance a up to the object.
5 The experimental calibration is used in the conventional distance measuring method of utilizing stereo vision, as the optical members such as lens of the camera have different properties and thus is not intrinsic to the present invention.
10 As described above, in the present embodiment, the observed image obtained at a given position and the observed image obtained at a small distance from the given position on the same optical axis have different blur quantities at the same point on the 15 image receiving plane. The filter (Labor function) designated by expression (6) is applied to two input observed images with different blurs. Further, respective output images are obtained for the predetermined frequencies in a frequency range and the 20 expression (14) relating to the relation between the observed quantity and the radius of the blur is obtained. The process calculates the data relating to the distance up to the object by using a method of calculating the distance by substituting respective 25 values in expressions (14) and (3) or by referring to the previously prepared look-up-table between the observed quantity and the distance up to the ob,~ect.
Figure 6 shows a configuration of the second embodiment of the present invention in which the look-up-table is used. The relation between the observed value of the left side of the expression (6) for corresponding predetermined frequencies k and the actual value is measured upon an execution of the calibration and is stored in the look-up-table. Then, when the discretional image data is input, the look-up-table obtained by calculating the left side of the expression based on the results Fs and Fs' of the filtering and the movement quantity ~z is referred to, thereby providing the distance data. Namely, as shown in Figure 6, the image data GA and GB are input to the filter operation units FXA and FXB, thereby providing the output Fs and Fs'. The filter operation units FXA and FXB comprise the same circuit but their input are different. As a result, the input data is stored in the table by using the Fs and Fs' and the distance pz, by which the lens or camera are moved upon inputting these image data as the address of the table . Therefore, the table THL outputs the distance data representing the actual distance previously obtained upon an execution of the calibration.
This embodiment does not require large quantities of calculations for finding the corresponding points between images input two television cameras which is required by the conventional distance measuring method of using stereo vision and thus measures the distance better than a human being, who makes defective measurements. It performs rapidly with a simple construction using monocular.
The above embodiment was described using a one dimensional image on an optical axis. Three dimensioned distance image can be also attained by applying the filtering shown in expression (6) to the entire image and calculating the related expression relating to the observed quantity and the distance up to the object for each respective image element in accordance with the expression (14) and (3) or by calculating the distance data by referring to the look-up-table. The situation in external space can be determined based on this distance image. For example, in the research field of computer vision, extensive research for enabling computers to understand an object as a human being does and for determining the position in relation to the objects has been extensively performed. However, there is a problem in the means of obtaining the distance image in three dimensional space and thus the positional relation of the objects and the structure in the three dimensional space can not be easily understood. The present invention can obtain the distance data and the three dimensional environment with ease and thus enables such an environment to be understood. It is possible to derive the distance image with a.high rate such as a video rate. A special solution of the distance image is about its distribution.
The basic principle of the present invention resides in measuring distance based on two images through a filtering process. The filtering process limits the degree of the distribution of the image by determining the filter size to be a. Accordingly, the sigma affects on the obtained distance image data and the accuracy of the distance image data depends on the distribution a. In other words, the actual distance differs from the distance obtained according to the present invention by the distribution Q in terms of accuracy. when the method of the present invention is compared with the sensitivity of the vision of the human beings, the human being cannot measure the distance with high accuracy but, can understand the environment of the works better than the present method does. Therefore, it can be said that the 2p7863$
present invention can achieve a sufficient accuracy as the distance measured by the present method differs from the actual distance nearly by the distribution a.
In the above description, the television camera is moved on the same optical axis. Instead of moving the television camera, the lens or the image receiving plane within the television camera may be moved on the same optical axis or a plurality of moving planes are positioned on different points within the television camera, thereby calculating the distance data using the same process as described above.
Figure 7 shows a schematic configuration of a first example of the image input unit 6 of the distance measuring apparatus shown in Figure 4.
In Figure 7, the image input unit 11 is provided with a lens moving structure for sliding the lens 12 provided in the lens cylinder 13 along the optical axis 12a of the lens 12 within the lens cylinder 13 and for changing the distance between the image receiving plane 15 provided by the lens 12 and the image receiving plane 15 provided in the camera body 14.
By controlling the lens moving structure, respective observed images Is and Is' with different blurs can be obtained on the image receiving plane 15 by moving the lens from its position shown by the solid line to the position shown by a chained line.
Two observed images are input to the calculation processing unit 3 not shown and the previously described series of processes is carried out by the calculation processing unit 3, thereby enabling the distance data to be obtained with ease by using monocular.
Figure 8 shows the distance measuring apparatus shown in Figure 4 according to the present invention and designates a schematic drawing of a structure of the second example of the image input unit 6.
As shown in Figure 8, the image input unit 21 is provided with a moving structure for the image receiving plane for moving the image receiving plane provided in the camera body 24 along an optical axis 22a of the lens 22 provided in the lens cylinder 23 in the camera body 24, thereby changing the distance between the image receiving plane 25 and lens 20 22.
By controlling the image receiving plane movement structure, respective observed images Is and Is' lwith different blurs are obtained on the image receiving plane 25 with regard to a position shown by a solid 25 line before the image receiving plane 25 is moved and a position shown by a chained line after the image receiving plane 25 is moved. These two observed images are input to the calculation processing unit 3 not shown and the above described process is carried out by the calculation processing unit 3, thereby enabling the distance data to be obtained using monocular with ease.
Figure 9 shows the distance measuring apparatus shown in Figure 4 and a schematic diagram of a structure of the third example of the image input unit 6.
As shown in Figure 9, the image input unit 31 is provided with a structure 36 to move the camera body.
This fixes the distance between the image receiving plane 35 provided in the camera body 34 and the lens 32 provided in the lens cylinder 33 and moves the camera body 34 along an optical axis 32a of the lens 32.
By controlling the moving structure 36 for the camera body, respective observed images Is and Is' with different blurs are obtained on the image receiving plane 35 relative to the position before the camera body is moved and the position after the camera body 34 is moved. These two observed images are input to the calculation processing unit 3 not shown and the 32 2 0 ~ a s 3 8 above described process is carried out, thereby enabling the distance data to be obtained with ease using monocular.
Figure 10 shows a schematic diagram of a structure of the fourth example of the image input unit 6 in the distance measuring apparatus shown in Figure 4.
As shown in Figure 10, the image input unit 41 is provided with a spectroscope 46 such as a prism and respective image receiving plane 45b and 45c are provided positions with different distances from the spectroscope within the camera body 44 and the optical axis 42a of the lens 42 provided in the lens cylinder 43 is divided into two optical axes 42b and 42c by the spectroscope 46.
It is not necessary to move the lens 42 in the lens cylinder 43 and also not necessary to move the image receiving planes 45b and 45c in the camera body 44. Further, it is not necessary to move the camera body 44. The lens 42 and the image receiving planes 44b and 44c are in fixed positions and the distance between the spectroscope 46 and two image receiving planes 45b and 45c are made different and respective observed images Is and Is' with different blurs are obtained on the respective image receiving planes 45b and 45c. These two observed images are input to the calculation processing unit 3 not shown and the above described process is carried out, thereby enabling the distance data to be obtained with ease using a single eye sight. Further, in this embodiment, moving structures such as a lens and image receiving plane are not needed and thus in the present example the two observed images are simultaneously obtained by using the same optical system and thus the deterioration of accuracy which is caused movement structures can be prevented.
The method of moving the lens, image receiving plane and the camera body along an optical axis without changing the focus distance f of the lens and the radius r of the lens can satisfy the principle expression (3) where either the distance a between the lens and the object, or the distance f between the lens and the image receiving plane or both are changed. Accordingly, the object measuring method of the present invention can be applied in respective cases where the lens of the camera is moved, the image receiving plane is moved in the camera body, the entire camera is moved and a plurality of image receiving planes are provided in different position in the camera body. If the method satisfies the 34 2 ~ 7 ~ 6 3 8 principle expression (3), then it also satisfies expression (14) and thus the distance measuring method of the present invention can be applied to the respective methods of movement.
Figure 11 shows a block diagram of the third embodiment of the distance measuring apparatus according to the present invention in which an operation of taking an image is performed twice. The image input unit in this embodiment uses the structures shown in Figures 7, 8 and 9.
The analog image taken at the first point in time is converted to a digital signal by the A/D conversion circuit 51. The digital image is input to the filter operation circuit 52 and the result Fs is stored in the image memory 53.
Sequentially, after the movement of the camera, lens, or image receiving plane is conducted by Oz, the analog image taken at the point in second time is converted to the digital image by the A/D converter 51 and then the result is input to the filter operation circuit 52, thereby-providing Fs'. The result Fs' is stored in the image memory 54. The Fs and Fs' obtained by these sequential operations and the movement distance ~z are input to the table TBL, thereby providing the distance data from the table 35 2 ~ ~ 8 6 3 8 TBL.
Figure 12 shows the fourth embodiment of the distance measuring apparatus according to the present invention in which the images are received and processed simultaneously at a single point in time.
In the third embodiment, the movement by the distance Oz is required but in the fourth embodiment two analog images obtained by two image receiving planes 45b and 45c as shown in Figure 10 are received and processed at a single point in time, thereby providing the distance.
The analog images obtained from two image receiving planes 45b and 45c are input to the A/D
conversion circuits 61 and 62. The digital image converted to the digital data by the A/D conversion circuits 61 and 62 are input to the filter operation circuits 63 and 64, thereby providing Fs and Fs'.
These results are stored in the image memory 65 and 66 to provide an address of the table THL. In this structure, ~z is fixed in the above operation, and the analog image of the image receiving planes which are different by a distance ~z can be taken at the single point in time. The A/D conversion operation and the filtering operation can be also simultaneously Performed with regard to the two images, thereby providing the distance in half the time required for the third embodiment.
Claims (12)
1. A method of obtaining distance data from an object to a lens by obtaining an image of the object by an image input unit for forming an image of the object through the lens on an image receiving plane and then performing a process, comprising steps of;
inputting two observed images Is and Is' with different blurs from the said image input unit by making a position of the lens, a position of the image receiving plane or positions of both of them with regard to the object different by a minute distance .DELTA.z on the same optical axis;
performing a filtering process by applying filtering to the said two input observed images Is and Is', thereby providing two output images Fs and Fs';
and calculating the distance a up to the object by using two output images Fs and Fs' obtained by the filtering process based on t:he relation between said two output images Fs and Fs' and a radius s of the blur and the relation between the radius s of the blur and the distance a from the lens up to the object.
inputting two observed images Is and Is' with different blurs from the said image input unit by making a position of the lens, a position of the image receiving plane or positions of both of them with regard to the object different by a minute distance .DELTA.z on the same optical axis;
performing a filtering process by applying filtering to the said two input observed images Is and Is', thereby providing two output images Fs and Fs';
and calculating the distance a up to the object by using two output images Fs and Fs' obtained by the filtering process based on t:he relation between said two output images Fs and Fs' and a radius s of the blur and the relation between the radius s of the blur and the distance a from the lens up to the object.
2 The method of measuring the distance according to Claim 1 wherein the said filtering processes are designated by the following expressions (A) and (B);
Fs= ~ exp(-x2/.sigma.2)exp(ikx)Is(x)dx ... (A) Fs'= ~ exp(-x2/.sigma.2)exp(ikx)Is'(x)dx ... (H) where k represents a certain predetermined frequency and .sigma. a distribution.
And the relation between the two output images Fs and Fs' and the radius s of the blur is represented by the following expression (C);
and the relation between the radius s of the blur and the distance a from the lens to the object is expressed by the following expression (D);
where .GAMMA. represents a radius of the lens and f a focus distance of the lens.
Fs= ~ exp(-x2/.sigma.2)exp(ikx)Is(x)dx ... (A) Fs'= ~ exp(-x2/.sigma.2)exp(ikx)Is'(x)dx ... (H) where k represents a certain predetermined frequency and .sigma. a distribution.
And the relation between the two output images Fs and Fs' and the radius s of the blur is represented by the following expression (C);
and the relation between the radius s of the blur and the distance a from the lens to the object is expressed by the following expression (D);
where .GAMMA. represents a radius of the lens and f a focus distance of the lens.
3. The method of measuring the distance according to Claim 2 wherein the said distance data calculating process calculates the distance data by referring to a look-up-table which is previously prepared to represent a relation between the calculation value of the said expression (C) and the distance a up to the object.
4. A distance measuring apparatus comprising:
an image input means comprising a lens and an image receiving plane for obtaining the observed image of the object on the image receiving plane and outputting the observed image;
a first operating means for applying a filtering process to the first image input from the said image input means thereby performing a Fourier transformation for a respective predetermined frequency k;
a second operation means for applying the same operation as that of the first operation means to the second observed image which is obtained from the image input means when at least one of the lens position, the position of the image receiving plane, and the position of the image input means is moved; and a third operation means for obtaining the distance up to the object based on the results of the first operating means for the first observed image, the result of the second operation means for the second observed image and the distance in which the said at least one of the lens, the image receiving plane and the image input means are moved.
an image input means comprising a lens and an image receiving plane for obtaining the observed image of the object on the image receiving plane and outputting the observed image;
a first operating means for applying a filtering process to the first image input from the said image input means thereby performing a Fourier transformation for a respective predetermined frequency k;
a second operation means for applying the same operation as that of the first operation means to the second observed image which is obtained from the image input means when at least one of the lens position, the position of the image receiving plane, and the position of the image input means is moved; and a third operation means for obtaining the distance up to the object based on the results of the first operating means for the first observed image, the result of the second operation means for the second observed image and the distance in which the said at least one of the lens, the image receiving plane and the image input means are moved.
5. The distance measuring apparatus according to Claim 4 wherein the said first and second operations comprises an integration of the result obtained by multiplying the observed image input from the said image input means with a Gaussian function and a Fourier function.
6. The distance measuring apparatus according to Claim 5 wherein the said third operation obtains the distance a from the following expressions;
wherein Fs represents the result of the first operation, Fs' represents the result of the second operation, .DELTA.z is the quantity of the movement, k is a predetermined frequency, .GAMMA. is the radius of the lens, f is the focus distance of the lens and a is the distance up to the object.
wherein Fs represents the result of the first operation, Fs' represents the result of the second operation, .DELTA.z is the quantity of the movement, k is a predetermined frequency, .GAMMA. is the radius of the lens, f is the focus distance of the lens and a is the distance up to the object.
7. The distance measuring apparatus according to Claim 5 further comprising a memory means for storing a look-up-table previously prepared for the result of the first operation, the result of the second operation and the distance up to the object and wherein the said third operation can be performed by referring to the said look-up-table.
8. The distance measuring apparatus according to Claim 7 wherein the said look-up-table is obtained by performing a calculation of the data associated with the result of the first operation and the result of the second operation obtained for the distance of the movement.
9. The distance measuring apparatus according to Claim 4 wherein the said image input means comprises a lens movement structure for moving the lens along the optical axis and outputs the first and second observed images obtained on the image receiving plane before the lens is moved and after the lens is moved, respectively.
10. The distance measuring apparatus according to Claim 4 wherein the said image input means comprises an image receiving plane structure for moving the image receiving plane along an optical axis and outputs the first and second observed images on the image receiving plane before the image receiving plane is moved and after the image receiving plane is moved, respectively.
11. The distance measuring apparatus according to Claim 4 wherein the said image input means comprises a structure for moving the camera body along the optical axis keeping the positions of the lens and the image receiving plane fixed and outputting the first and second observed images on the image receiving plane before the camera body is moved and after the camera body is moved, respectively.
12. The distance measuring apparatus according to Claim 4 wherein the said image input means is provided with a spectroscope and respective image receiving plane with different distances from the spectroscope and output the first and second observed images at the respective image receiving plane.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP03-241289 | 1991-09-20 | ||
| JP03241289A JP3110095B2 (en) | 1991-09-20 | 1991-09-20 | Distance measuring method and distance measuring device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2078638A1 CA2078638A1 (en) | 1993-03-21 |
| CA2078638C true CA2078638C (en) | 2000-01-25 |
Family
ID=17072055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002078638A Expired - Fee Related CA2078638C (en) | 1991-09-20 | 1992-09-18 | Distance measuring method and a distance measuring apparatus |
Country Status (6)
| Country | Link |
|---|---|
| US (1) | US5576975A (en) |
| EP (1) | EP0533590B1 (en) |
| JP (1) | JP3110095B2 (en) |
| KR (1) | KR970001140B1 (en) |
| CA (1) | CA2078638C (en) |
| DE (1) | DE69208005T2 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103063138A (en) * | 2013-01-24 | 2013-04-24 | 惠州Tcl移动通信有限公司 | Methods for measuring size and speed of object by camera of mobile terminal, and mobile terminal |
Families Citing this family (96)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5910854A (en) | 1993-02-26 | 1999-06-08 | Donnelly Corporation | Electrochromic polymeric solid films, manufacturing electrochromic devices using such solid films, and processes for making such solid films and devices |
| US5442167A (en) * | 1993-04-16 | 1995-08-15 | Intermec Corporation | Method and apparatus for automatic image focusing |
| DE4413368C1 (en) * | 1994-04-19 | 1995-09-28 | Wilfried Donner | Method of optical focussing of object image |
| US5668663A (en) | 1994-05-05 | 1997-09-16 | Donnelly Corporation | Electrochromic mirrors and devices |
| US5666569A (en) * | 1994-08-25 | 1997-09-09 | Flashpoint Technology, Inc. | System and method for detecting and indicating proper focal distance in a fixed lens camera |
| JPH0887600A (en) * | 1994-09-19 | 1996-04-02 | Topcon Corp | Edge detection device |
| US6891563B2 (en) | 1996-05-22 | 2005-05-10 | Donnelly Corporation | Vehicular vision system |
| US6326613B1 (en) | 1998-01-07 | 2001-12-04 | Donnelly Corporation | Vehicle interior mirror assembly adapted for containing a rain sensor |
| US6172613B1 (en) | 1998-02-18 | 2001-01-09 | Donnelly Corporation | Rearview mirror assembly incorporating vehicle information display |
| US6124886A (en) | 1997-08-25 | 2000-09-26 | Donnelly Corporation | Modular rearview mirror assembly |
| US8294975B2 (en) | 1997-08-25 | 2012-10-23 | Donnelly Corporation | Automotive rearview mirror assembly |
| JPH11102438A (en) * | 1997-09-26 | 1999-04-13 | Minolta Co Ltd | Distance image generation device and image display device |
| US6445287B1 (en) | 2000-02-28 | 2002-09-03 | Donnelly Corporation | Tire inflation assistance monitoring system |
| US8288711B2 (en) | 1998-01-07 | 2012-10-16 | Donnelly Corporation | Interior rearview mirror system with forwardly-viewing camera and a control |
| US6693517B2 (en) | 2000-04-21 | 2004-02-17 | Donnelly Corporation | Vehicle mirror assembly communicating wirelessly with vehicle accessories and occupants |
| US6477464B2 (en) | 2000-03-09 | 2002-11-05 | Donnelly Corporation | Complete mirror-based global-positioning system (GPS) navigation solution |
| US6329925B1 (en) | 1999-11-24 | 2001-12-11 | Donnelly Corporation | Rearview mirror assembly with added feature modular display |
| JP2000146523A (en) * | 1998-09-02 | 2000-05-26 | Sony Corp | Instrument and method for measuring distance |
| JP4160683B2 (en) * | 1999-03-02 | 2008-10-01 | 株式会社東京測器研究所 | Strain measurement system |
| US6683651B1 (en) | 1999-10-28 | 2004-01-27 | Hewlett-Packard Development Company, L.P. | Method of automatically adjusting focus in a shutterless digital camera |
| WO2007053710A2 (en) | 2005-11-01 | 2007-05-10 | Donnelly Corporation | Interior rearview mirror with display |
| US7370983B2 (en) | 2000-03-02 | 2008-05-13 | Donnelly Corporation | Interior mirror assembly with display |
| US7167796B2 (en) | 2000-03-09 | 2007-01-23 | Donnelly Corporation | Vehicle navigation system for use with a telematics system |
| EP1263626A2 (en) | 2000-03-02 | 2002-12-11 | Donnelly Corporation | Video mirror systems incorporating an accessory module |
| US7581859B2 (en) | 2005-09-14 | 2009-09-01 | Donnelly Corp. | Display device for exterior rearview mirror |
| US7255451B2 (en) | 2002-09-20 | 2007-08-14 | Donnelly Corporation | Electro-optic mirror cell |
| ATE363413T1 (en) | 2001-01-23 | 2007-06-15 | Donnelly Corp | IMPROVED VEHICLE LIGHTING SYSTEM |
| AU2002240415A1 (en) * | 2001-02-16 | 2002-09-04 | Canesta Inc. | Technique for removing blurring from a captured image |
| US6918674B2 (en) | 2002-05-03 | 2005-07-19 | Donnelly Corporation | Vehicle rearview mirror system |
| US7329013B2 (en) | 2002-06-06 | 2008-02-12 | Donnelly Corporation | Interior rearview mirror system with compass |
| WO2003105099A1 (en) | 2002-06-06 | 2003-12-18 | Donnelly Corporation | Interior rearview mirror system with compass |
| AU2003278863A1 (en) | 2002-09-20 | 2004-04-08 | Donnelly Corporation | Mirror reflective element assembly |
| WO2004103772A2 (en) | 2003-05-19 | 2004-12-02 | Donnelly Corporation | Mirror assembly for vehicle |
| US7310177B2 (en) | 2002-09-20 | 2007-12-18 | Donnelly Corporation | Electro-optic reflective element assembly |
| US7446924B2 (en) | 2003-10-02 | 2008-11-04 | Donnelly Corporation | Mirror reflective element assembly including electronic component |
| US7308341B2 (en) | 2003-10-14 | 2007-12-11 | Donnelly Corporation | Vehicle communication system |
| US7268006B2 (en) | 2004-12-30 | 2007-09-11 | E.I. Du Pont De Nemours And Company | Electronic device including a guest material within a layer and a process for forming the same |
| ATE517368T1 (en) | 2005-05-16 | 2011-08-15 | Donnelly Corp | VEHICLE MIRROR ARRANGEMENT WITH CHARACTER ON THE REFLECTIVE PART |
| EP2378350B1 (en) | 2006-03-09 | 2013-12-11 | Gentex Corporation | Vehicle rearview assembly including a high intensity display |
| US7397540B2 (en) * | 2006-08-21 | 2008-07-08 | The Boeing Company | Phase diversity ranging sensor |
| JP2009047734A (en) * | 2007-08-13 | 2009-03-05 | Olympus Corp | Imaging apparatus and image processing program |
| JP2009047735A (en) * | 2007-08-13 | 2009-03-05 | Olympus Corp | Imaging apparatus and image processing program |
| US8154418B2 (en) | 2008-03-31 | 2012-04-10 | Magna Mirrors Of America, Inc. | Interior rearview mirror system |
| JP5199771B2 (en) * | 2008-07-25 | 2013-05-15 | 株式会社豊田中央研究所 | Optical receiver and optical communication system |
| US9487144B2 (en) | 2008-10-16 | 2016-11-08 | Magna Mirrors Of America, Inc. | Interior mirror assembly with display |
| JP5430292B2 (en) * | 2009-03-26 | 2014-02-26 | 京セラ株式会社 | Subject distance measuring device |
| JP5459768B2 (en) * | 2009-10-28 | 2014-04-02 | 京セラ株式会社 | Subject distance estimation device |
| JP2012003233A (en) * | 2010-05-17 | 2012-01-05 | Sony Corp | Image processing device, image processing method and program |
| JP5576739B2 (en) | 2010-08-04 | 2014-08-20 | オリンパス株式会社 | Image processing apparatus, image processing method, imaging apparatus, and program |
| US8711215B2 (en) | 2010-08-06 | 2014-04-29 | Panasonic Corporation | Imaging device and imaging method |
| WO2012147245A1 (en) | 2011-04-27 | 2012-11-01 | パナソニック株式会社 | Image pick-up device, image pick-up system equipped with image pick-up device, and image pick-up method |
| WO2013080551A1 (en) | 2011-11-30 | 2013-06-06 | パナソニック株式会社 | Imaging device |
| WO2013080552A1 (en) | 2011-11-30 | 2013-06-06 | パナソニック株式会社 | Imaging device and imaging system |
| US9561022B2 (en) | 2012-02-27 | 2017-02-07 | Covidien Lp | Device and method for optical image correction in metrology systems |
| US9117277B2 (en) | 2012-04-04 | 2015-08-25 | Canon Kabushiki Kaisha | Determining a depth map from images of a scene |
| US8879139B2 (en) | 2012-04-24 | 2014-11-04 | Gentex Corporation | Display mirror assembly |
| JP6214183B2 (en) * | 2012-05-11 | 2017-10-18 | キヤノン株式会社 | Distance measuring device, imaging device, distance measuring method, and program |
| US9598018B2 (en) | 2013-03-15 | 2017-03-21 | Gentex Corporation | Display mirror assembly |
| EP3008485A1 (en) * | 2013-06-13 | 2016-04-20 | Basf Se | Detector for optically detecting at least one object |
| AU2014326772B2 (en) | 2013-09-24 | 2017-07-20 | Gentex Corporation | Display mirror assembly |
| US9511715B2 (en) | 2014-01-31 | 2016-12-06 | Gentex Corporation | Backlighting assembly for display for reducing cross-hatching |
| WO2015143333A1 (en) | 2014-03-21 | 2015-09-24 | Gentex Corporation | Tri-modal display mirror assembly |
| JP5895270B2 (en) * | 2014-03-28 | 2016-03-30 | パナソニックＩｐマネジメント株式会社 | Imaging device |
| KR101894262B1 (en) | 2014-04-01 | 2018-09-04 | 젠텍스 코포레이션 | Automatic display mirror assembly |
| US11041718B2 (en) | 2014-07-08 | 2021-06-22 | Basf Se | Detector for determining a position of at least one object |
| WO2016044746A1 (en) | 2014-09-19 | 2016-03-24 | Gentex Corporation | Rearview assembly |
| WO2016073848A1 (en) | 2014-11-07 | 2016-05-12 | Gentex Corporation | Full display mirror actuator |
| EP3218227B1 (en) | 2014-11-13 | 2018-10-24 | Gentex Corporation | Rearview mirror system with a display |
| KR101997815B1 (en) | 2014-12-03 | 2019-07-08 | 젠텍스 코포레이션 | Display mirror assembly |
| USD746744S1 (en) | 2014-12-05 | 2016-01-05 | Gentex Corporation | Rearview device |
| JP6637980B2 (en) | 2014-12-09 | 2020-01-29 | ビーエーエスエフ ソシエタス・ヨーロピアＢａｓｆ Ｓｅ | Optical detector |
| US9744907B2 (en) | 2014-12-29 | 2017-08-29 | Gentex Corporation | Vehicle vision system having adjustable displayed field of view |
| US9720278B2 (en) | 2015-01-22 | 2017-08-01 | Gentex Corporation | Low cost optical film stack |
| KR102496245B1 (en) | 2015-01-30 | 2023-02-06 | 트리나미엑스 게엠베하 | Detector for optical detection of one or more objects |
| JP2018513810A (en) | 2015-04-20 | 2018-05-31 | ジェンテックス コーポレイション | Rear view assembly with decoration |
| EP3297870B1 (en) | 2015-05-18 | 2020-02-05 | Gentex Corporation | Full display rearview device |
| WO2016209877A1 (en) | 2015-06-22 | 2016-12-29 | Gentex Corporation | System and method for processing streamed video images to correct for flicker of amplitude-modulated lights |
| US10955936B2 (en) | 2015-07-17 | 2021-03-23 | Trinamix Gmbh | Detector for optically detecting at least one object |
| CN108141579B (en) | 2015-09-14 | 2020-06-12 | 特里纳米克斯股份有限公司 | 3D camera |
| USD797627S1 (en) | 2015-10-30 | 2017-09-19 | Gentex Corporation | Rearview mirror device |
| EP3368375B1 (en) | 2015-10-30 | 2020-03-04 | Gentex Corporation | Rearview device |
| USD798207S1 (en) | 2015-10-30 | 2017-09-26 | Gentex Corporation | Rearview mirror assembly |
| CN108349435B (en) | 2015-10-30 | 2021-06-15 | 金泰克斯公司 | Switching board |
| USD800618S1 (en) | 2015-11-02 | 2017-10-24 | Gentex Corporation | Toggle paddle for a rear view device |
| USD845851S1 (en) | 2016-03-31 | 2019-04-16 | Gentex Corporation | Rearview device |
| USD817238S1 (en) | 2016-04-29 | 2018-05-08 | Gentex Corporation | Rearview device |
| US10025138B2 (en) | 2016-06-06 | 2018-07-17 | Gentex Corporation | Illuminating display with light gathering structure |
| JP2019523562A (en) | 2016-07-29 | 2019-08-22 | トリナミクス ゲゼルシャフト ミット ベシュレンクテル ハフツング | Optical sensor and detector for optical detection |
| US10890491B2 (en) | 2016-10-25 | 2021-01-12 | Trinamix Gmbh | Optical detector for an optical detection |
| US11428787B2 (en) | 2016-10-25 | 2022-08-30 | Trinamix Gmbh | Detector for an optical detection of at least one object |
| US11860292B2 (en) | 2016-11-17 | 2024-01-02 | Trinamix Gmbh | Detector and methods for authenticating at least one object |
| EP3571522B1 (en) | 2016-11-17 | 2023-05-10 | trinamiX GmbH | Detector for optically detecting at least one object |
| USD809984S1 (en) | 2016-12-07 | 2018-02-13 | Gentex Corporation | Rearview assembly |
| USD854473S1 (en) | 2016-12-16 | 2019-07-23 | Gentex Corporation | Rearview assembly |
| EP3562710A4 (en) | 2016-12-30 | 2019-11-13 | Gentex Corporation | Full display mirror with on-demand spotter view |
| CN110312641A (en) | 2017-03-17 | 2019-10-08 | 金泰克斯公司 | Dual display reversing camera system |
Family Cites Families (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US3788201A (en) * | 1970-12-14 | 1974-01-29 | F Abell | Method for establishing vehicle identification, speed and conditions of visibility |
| JPS5156627A (en) * | 1974-11-13 | 1976-05-18 | Canon Kk | Butsutaizosenmeidokenshutsuhoshiki |
| US4173010A (en) * | 1975-05-01 | 1979-10-30 | Hoffmann Anton R | Traffic sign and improved system for recording vehicle speed |
| US4135817A (en) * | 1978-03-09 | 1979-01-23 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Apparatus for measuring an aircraft's speed and height |
| US4257703A (en) * | 1979-03-15 | 1981-03-24 | The Bendix Corporation | Collision avoidance using optical pattern growth rate |
| DE2910875C2 (en) * | 1979-03-20 | 1985-11-14 | Kernforschungszentrum Karlsruhe Gmbh, 7500 Karlsruhe | Auto-focus procedure |
| US4433325A (en) * | 1980-09-30 | 1984-02-21 | Omron Tateisi Electronics, Co. | Optical vehicle detection system |
| US4495589A (en) * | 1982-09-20 | 1985-01-22 | Crane Co. | Aircraft ground velocity determination system |
| US4671650A (en) * | 1982-09-20 | 1987-06-09 | Crane Co. (Hydro-Aire Division) | Apparatus and method for determining aircraft position and velocity |
| DE3687760T2 (en) * | 1985-03-13 | 1993-06-09 | Topcon Corp | DEVICE AND METHOD FOR MEASURING COORDINATES. |
| DE3516664A1 (en) * | 1985-05-09 | 1986-11-13 | Honeywell Regelsysteme GmbH, 6050 Offenbach | OPTICAL ARRANGEMENT FOR PASSIVE DISTANCE DETERMINATION |
| FI74556C (en) * | 1986-04-11 | 1988-02-08 | Valtion Teknillinen | FOERFARANDE FOER TREDIMENSIONELL OEVERVAKNING AV ETT MAOLUTRYMME. |
| JP2543505B2 (en) * | 1986-07-22 | 1996-10-16 | 安川商事 株式会社 | Signal processing device and measuring device using space-time differential method |
| GB8701288D0 (en) * | 1987-01-21 | 1987-02-25 | Waldern J D | Perception of computer-generated imagery |
| US4847772A (en) * | 1987-02-17 | 1989-07-11 | Regents Of The University Of Minnesota | Vehicle detection through image processing for traffic surveillance and control |
| JP2570315B2 (en) * | 1987-09-01 | 1997-01-08 | アイシン精機株式会社 | On-vehicle distance detection device |
| US5034904A (en) * | 1988-01-27 | 1991-07-23 | Storage Technology Corporation | Vision system illumination calibration apparatus |
| IL86202A (en) * | 1988-04-27 | 1992-01-15 | Driver Safety Systems Ltd | Traffic safety monitoring apparatus |
| US4969735A (en) * | 1989-03-07 | 1990-11-13 | Sperry Marine Inc. | Passive range finding apparatus utilizing television sensors |
| JPH03269315A (en) * | 1990-03-20 | 1991-11-29 | Mitsubishi Electric Corp | Inter-vehicle distance detector for tracking leading vehicle |
| JP2646146B2 (en) * | 1990-03-28 | 1997-08-25 | 三菱電機株式会社 | Inter-vehicle distance control device |
| JP2583641B2 (en) * | 1990-05-18 | 1997-02-19 | 日産自動車株式会社 | Travel control method |
| JPH0812072B2 (en) * | 1990-06-13 | 1996-02-07 | 三菱電機株式会社 | Distance measuring device |
| DE69124726T2 (en) * | 1990-10-25 | 1997-07-03 | Mitsubishi Electric Corp | Distance detection device for a motor vehicle |
| US5161107A (en) * | 1990-10-25 | 1992-11-03 | Mestech Creation Corporation | Traffic surveillance system |
| US5231470A (en) * | 1991-09-06 | 1993-07-27 | Koch Stephen K | Scanning system for three-dimensional object digitizing |
| JP3167752B2 (en) * | 1991-10-22 | 2001-05-21 | 富士重工業株式会社 | Vehicle distance detection device |
| JP2917661B2 (en) * | 1992-04-28 | 1999-07-12 | 住友電気工業株式会社 | Traffic flow measurement processing method and device |
| FR2695742B1 (en) * | 1992-09-15 | 1994-10-21 | Thomson Csf | System for calculating at least one vehicle traffic control parameter. |
-
1991
- 1991-09-20 JP JP03241289A patent/JP3110095B2/en not_active Expired - Fee Related
-
1992
- 1992-09-18 CA CA002078638A patent/CA2078638C/en not_active Expired - Fee Related
- 1992-09-21 DE DE69208005T patent/DE69208005T2/en not_active Expired - Fee Related
- 1992-09-21 KR KR1019920017149A patent/KR970001140B1/en not_active IP Right Cessation
- 1992-09-21 EP EP92402588A patent/EP0533590B1/en not_active Expired - Lifetime
-
1995
- 1995-03-07 US US08/400,526 patent/US5576975A/en not_active Expired - Lifetime
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103063138A (en) * | 2013-01-24 | 2013-04-24 | 惠州Tcl移动通信有限公司 | Methods for measuring size and speed of object by camera of mobile terminal, and mobile terminal |
| CN103063138B (en) * | 2013-01-24 | 2016-08-10 | 惠州Tcl移动通信有限公司 | The method of mobile terminal camera Measuring Object size and mobile terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| JPH0579838A (en) | 1993-03-30 |
| JP3110095B2 (en) | 2000-11-20 |
| EP0533590A2 (en) | 1993-03-24 |
| KR930006434A (en) | 1993-04-21 |
| US5576975A (en) | 1996-11-19 |
| EP0533590A3 (en) | 1993-05-12 |
| DE69208005T2 (en) | 1996-06-13 |
| KR970001140B1 (en) | 1997-01-29 |
| DE69208005D1 (en) | 1996-03-14 |
| EP0533590B1 (en) | 1996-01-31 |
| CA2078638A1 (en) | 1993-03-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2078638C (en) | Distance measuring method and a distance measuring apparatus | |
| EP0466277A1 (en) | Passive ranging and rapid autofocusing | |
| US20070189750A1 (en) | Method of and apparatus for simultaneously capturing and generating multiple blurred images | |
| US5193124A (en) | Computational methods and electronic camera apparatus for determining distance of objects, rapid autofocusing, and obtaining improved focus images | |
| Lavest et al. | Dry camera calibration for underwater applications | |
| EP0408224B1 (en) | Computational methods and electronic camera apparatus for determining distance of objects, rapid autofocusing and obtaining improved focus images | |
| EP3676802B1 (en) | Image processing device that utilizes built-in dynamic camera models to support rapid determination of camera intrinsics and method of operating same | |
| Yamaguti et al. | A method of distance measurement by using monocular camera | |
| JPH0252204A (en) | Measuring instrument for three-dimensional coordinate | |
| JP3655065B2 (en) | Position / attitude detection device, position / attitude detection method, three-dimensional shape restoration device, and three-dimensional shape restoration method | |
| JP2809348B2 (en) | 3D position measuring device | |
| EP0140972A1 (en) | Three-dimensional distance-measuring device | |
| Lippiello et al. | Robust visual tracking using a fixed multi-camera system | |
| JPH11223516A (en) | Three dimensional image pickup device | |
| JP3060114B2 (en) | Corresponding point search method | |
| JPS62203485A (en) | Focal point detector | |
| JPH09325019A (en) | Three-dimensional measuring device | |
| Egami et al. | Three dimensional measurement using color image and movable CCD system | |
| Mishima et al. | Absolute Scale from Varifocal Monocular Camera through SfM and Defocus Combined. | |
| JP2005147959A (en) | Distance-measuring method and distance-measuring device | |
| Subbarao et al. | Root-mean square error in passive autofocusing and 3D shape recovery | |
| KR930011688B1 (en) | Method for measuring position and angle by image | |
| Sasiadek et al. | Disparity Error in Advanced Vision Sensors | |
| Uffenkamp | Digital camera with motorized focus for high-precision measurements | |
| Yamguti et al. | A distance measurement method available to texture surface by using complex-log mapping |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKLA | Lapsed |