US20080043113A1 - Image processor and visual field support device - Google Patents
Image processor and visual field support device Download PDFInfo
- Publication number
- US20080043113A1 US20080043113A1 US11/842,685 US84268507A US2008043113A1 US 20080043113 A1 US20080043113 A1 US 20080043113A1 US 84268507 A US84268507 A US 84268507A US 2008043113 A1 US2008043113 A1 US 2008043113A1
- Authority
- US
- United States
- Prior art keywords
- image
- bird
- eye view
- region
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30261—Obstacle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/16—Image acquisition using multiple overlapping images; Image stitching
Abstract
Description
- This nonprovisional application claims priority under 35 U.S.C. § 119(a) on Patent Application No. 2006-224429 filed in Japan on Aug. 21, 2006 and Patent Application No. 2006-224002 filed in Japan on Aug. 21, 2006, the entire contents of which are hereby incorporated by reference.
- 1. Field of the Invention
- The present invention relates to a visual field support device and a visual field support method, and particularly relates to technology of generating bird's-eye views from images taken by a plurality of cameras installed at a vehicle and then synthesizing the bird's-eye views. The present invention also relates to an image processor for use in the visual field support device.
- 2. Description of Related Art
- A driver of an automobile or the like, upon running backward, faces difficulty in checking rearview due to the presence of a blind spot. Thus, a system has been already developed which is equipped with an in-car camera for monitoring the rear of a vehicle that is likely to be a blind spot for the driver and which displays a photographed image of the rear on a screen for car navigation or the like.
- Moreover, studies have been conducted for, instead of simply displaying a picture with a camera, presenting a more human-friendly picture by use of image processing technology. One of such studies is performing coordinate conversion of a photographed image to thereby generate and display such a bird's-eye view image as is viewed from above the ground. Displaying this bird's-eye view image makes it easier for the driver to recognize condition of the rear of the vehicle.
- Furthermore, a visual field support device has been developed which converts images obtained from a plurality of cameras into an all-around bird's-eye view image through geometric conversion and then displays this image on a display section. This visual field support device has an advantage of capable of presenting the driver with condition of all the periphery of the vehicle as a picture viewed from the above, thus covering the surrounding of the vehicle through 360 degrees without any blind spots.
- Now, the visual field support device of this type will be described as a conventional example.
FIG. 16 is a plan view from above thevehicle 100, showing installation condition of cameras at thevehicle 100.FIG. 17 is a view from the diagonally left front of thevehicle 100.FIG. 17 schematically shows visual fields (photographing regions) of the respective cameras. Thevehicle 100 is a truck composed of a driver's cabin and a luggage compartment having a height larger than that of the driver's cabin. - As shown in
FIG. 16 , thevehicle 100 has cameras (photographing apparatuses) 1F, 1B, 1L, and 1R respectively fitted at the front, rear, left, and right thereof. Thecameras - The visual field support device generates bird's-eye view images from images taken by the respective cameras and then synthesizes these bird's-eye view images to thereby display on a display device an all-around bird's-eye view image as shown in
FIG. 18 . On the display screen of the display device, the vehicle is displayed at the center and the bird's-eye view images respectively obtained from thecameras - There is a portion where visual fields (photographing regions) of the different cameras overlap each other, as shown in
FIG. 17 . For example, at a given region located diagonally left front of thevehicle 100, the visual fields of thecameras numeral 101 ofFIG. 18 . Typically displayed on theregion 101 is a picture based on the image photographed by thecamera 1F or the image photographed by thecamera 1L, or a picture obtained by averaging these images. - The visual field support device of this type performs synthesis processing so that an image has continuity on the ground surface, and thus displays parking slot lines, signs, letters, and the like drawn on the ground surface without any problems. This also applies to the
region 101 ofFIG. 18 . However, a solid object placed on the ground surface is viewed differently at different viewpoints of the camera, thus making it difficult, in principle, to accurately and continuously draw this object in an all-round bird's-eye view image. - For example, consider a case where a person as a
solid object 102 is present in a portion (space) where the visual field of thecamera 1F and the visual field of thecamera 1L overlap each other, as shown inFIG. 19 . In this case, when a bird's-eye view image is generated from an image photographed by thecamera 1F, thesolid object 102 on this bird's-eye view image appears as an image tilted leftward, as shown inFIG. 20 . On the other hand, when a bird's-eye view image is generated from an image photographed by thecamera 1L, thesolid object 102 on this bird's-eye view image appears as an image tilted forward, as shown inFIG. 21 . - To synthesize the bird's-eye view image obtained from the
camera 1F and the bird's-eye view image obtained from thecamera 1L, defining asynthesis boundary 103 as shown inFIGS. 20 and 21 and then simply attaching together the both bird's-eye view images at thesynthesis boundary 103 causes a problem that thesolid object 102 disappears in an all-round bird's-eye view image obtained through this synthesis. - A possible technique for solving such a problem is generating an image of a common region, where the bird's-eye view image obtained from the
camera 1F and the bird's-eye view image obtained from thecamera 1L overlap each other, by way of averaging the both bird's-eye view images. However, adopting such an averaging technique results in a solid object appearing as a double image in the all-round bird's-eye view image. Moreover, each image included in the double image is averaged with a background image, which makes it very hard to see the solid object depending on the colors of the solid object and the background. - A method as provided below is a conventional method for separately generating an all-round bird's-eye view image adopting only a bird's-eye view image obtained from the cameral 1F as an image of a common region and an all-round bird's-eye view image adopting only a bird's-eye view image obtained from the cameral 1L as an image of a common region and then displaying these two types of all-around bird's-eye view images on the right and left. A display image in this case appears as a
display image 200 ofFIG. 22 . With this method, however, a plurality of images to be checked by the driver are displayed simultaneously, thus adversely causing confusion, which possibly compromises the security. - There is also a possible method for switching, through, manual or machine operation between adopting an image of a bird's-eye view obtained from the
camera 1F as an image of a common region and adopting an image of a bird's-eye view obtained from thecamera 1L as an image of a common region. In this case, adisplay image 201 and adisplay image 202 shown inFIG. 23 are switched for display. However, switching by the manual operation is complicated, and a picture generated only based on the machine operation is not necessarily a picture in which a solid object can be easily recognized. Further, the switching based on manual or machine operation is equivalent to moving the synthesis boundary to the end of the common region, thus leaving possibility that a display of the solid object around the boundary disappears. That is, it is difficult to avoid, by simple switching, the problem that the solid object disappears. - A first image processor according to the present invention includes: a visual point conversion section respectively converting images photographed by an n-number of photographing apparatuses (where n is an integer of 2 or larger) into bird's-eye view images as viewed from a virtual visual point; and an image synthesis section synthesizing the obtained bird's-eye view images to generate a synthetic bird's-eye view image, and generates a video signal for displaying the synthetic bird's-eye view image on a display device. In the image processor, the image synthesis section compares, between a plurality of the bird's-eye view images, images of a common region where the plurality of the bird's-eye view images overlap upon the synthesis to thereby specify a difference region between the plurality of the bird's-eye view images in the common region, selects, in accordance with position of the difference region in the common region, one bird's-eye view image from among the plurality of the bird's-eye view images as an image to be adopted, and adopts, as an image of the common region in the synthetic bird's-eye view image, the image of the common region in the image to be adopted.
- Specifically, for example, weight values corresponding to respective pixel positions of the image in the common region may be set for each of the plurality of the bird's-eye view images, part or all of the set weight values may differ between the plurality of the bird's-eye view images, the image synthesis section may sum up the weight values corresponding to pixels included in the difference region to thereby calculate a summed value for each of the plurality of the bird's-eye view images, and may select, based on the summed values, the image to be adopted.
- More specifically, for example, the image synthesis section may compare the summed values between the plurality of the bird's-eye view images to thereby select the image to be adopted.
- More specifically, for example, the n-number of photographing apparatuses may be installed at a vehicle and may photograph surrounding of the vehicle, and the image synthesis section, based on driving condition of the vehicle in addition to the summed values, may select the image to be adopted.
- For example, the weight values may be set based on installation positions of a plurality of the photographing apparatuses corresponding to the plurality of the bird's-eye view images.
- A visual field support device may be formed by including the image processor described above, and at least one of the n-number of photographing apparatuses and the display device.
- A first visual support method according to the invention for respectively converting images photographed by an n-number of photographing apparatuses (where n is an integer of 2 or larger) into bird's-eye view images as viewed from a virtual visual point and displaying on a display device a synthetic bird's-eye view image obtained by synthesizing the obtained bird's-eye view images: compares, between a plurality of the bird's-eye view images, images of a common region where the plurality of the bird's-eye view images overlap upon the synthesis to thereby specify a difference region between the plurality of the bird's-eye view images in the common region; selects, in accordance with position of the difference region in the common region, one bird's-eye view image from among the plurality of the bird's-eye view images as an image to be adopted; and adopts, as an image of the common region in the synthetic bird's-eye view image, the image of the common region in the image to be adopted.
- A second image processor according to the invention includes: a visual point conversion section respectively converting images photographed by an n-number of photographing apparatuses (where n is an integer of 2 or larger) into bird's-eye view images as viewed from a virtual visual point; and an image synthesis section synthesizing the obtained bird's-eye view images to generate a synthetic bird's-eye view image, and generates a video signal for displaying the synthetic bird's-eye view image on a display device. In the image processor, the image synthesis section, when a solid object having some height is present in a common visual field photographed in common by a plurality of the photographing apparatuses, detects a solid object reflecting region corresponding to a synthetic region of regions where the solid object is drawn in a plurality of the bird's-eye view images obtained from the plurality of the photographing apparatuses, and adopts, as an image of the solid object reflecting region in the synthetic bird's-eye view image, an image of the solid object reflecting region in one of the plurality of the bird's-eye view images.
- Specifically, for example, the image synthesis section may compare, between the plurality of the bird's-eye view images, images of a common region where the plurality of the bird's-eye view images overlap upon the synthesis to thereby detect the solid object reflecting region.
- More specifically, for example, the image synthesis section may compare, between the plurality of the bird's-eye view images, images of a common region where the plurality of the bird's-eye view images overlap upon the synthesis to thereby specify a difference region between the plurality of the bird's-eye view images in the common region, and may detect the difference region as the solid object reflecting region.
- A visual field support device may be formed by including: the image processor described above; and at least one of the n-number of photographing apparatuses and the display device.
- A second visual support method according to the invention for respectively converting images photographed by an n-number of photographing apparatuses (where n is an integer of 2 or larger) into bird's-eye view images as viewed from a virtual visual point and displaying on a display device a synthetic bird's-eye view image obtained by synthesizing the obtained bird's-eye view images, when a solid object having some height is present in a common visual field photographed in common by a plurality of the photographing apparatuses, detects a solid object reflecting region corresponding to a synthetic region of regions where the solid object is drawn in a plurality of the bird's-eye view images obtained from the plurality of the photographing apparatuses, and adopts, as an image of the solid object reflecting region in the synthetic bird's-eye view image, an image of the solid object reflecting region in one of the plurality of the bird's-eye view images.
-
FIG. 1 is a plan view of a vehicle equipped with one camera as viewed from the side, explaining a method for generating a bird's-eye view image from an image photographed by this camera; -
FIG. 2 is a diagram showing relationship between a camera coordinate system XYZ, a coordinate system XbuYbu of a camera image-sensing surface and a worldwide coordinate system XwYwZw including a two-dimensional ground coordinate system XwYw; -
FIG. 3 is a diagram of the vehicle according to embodiments of the present invention as viewed from diagonally front left thereof, -
FIG. 4 is a diagram of the vehicle according to the embodiments of the present invention as viewed from diagonally front left thereof, -
FIG. 5 is a diagram showing different bird's-eye view images according to the embodiments of the invention; -
FIG. 6 is a diagram of the bird's-eye view images ofFIG. 5 converted onto an all-round bird's-eye view image coordinates; -
FIG. 7A is a diagram showing a bird's-eye view image generated from an image photographed by a left side camera ofFIG. 3 ; -
FIG. 7B is a diagram showing a bird's-eye view image generated from an image photographed by a front camera ofFIG. 3 ; -
FIG. 7C is a diagram clarifying a common region between the two bird's-eye views images; -
FIG. 8 is an overall configuration diagram of a visual field support device according to the embodiments of the invention; -
FIG. 9 is a flowchart showing procedures of processing performed by the visual field support device ofFIG. 8 according to Example 1 of the invention; -
FIG. 10 is a diagram showing a difference image and a difference region in the common region ofFIG. 7C ; -
FIG. 11 is a diagram for explaining the processing of step S4 ofFIG. 9 , showing the common region ofFIG. 7C divided into three regions; -
FIG. 12 is a diagram for explaining the processing of step S4 ofFIG. 9 , showing weight values for the front camera and weight values for the left side camera for the common region ofFIG. 7C ; -
FIG. 13 is a diagram for explaining the processing of step S4 ofFIG. 9 ; -
FIG. 14 is a diagram for explaining the processing of step S4 ofFIG. 9 , illustrating the number of pixels belonging to the respective regions ofFIG. 11 ; -
FIG. 15 is a diagram showing an example of an all-round bird's-eye view image generated by an image processing section ofFIG. 8 according to Example 1 of the invention; -
FIG. 16 is a plan view showing a condition that a plurality of cameras are installed at the vehicle; -
FIG. 17 is a diagram of the vehicle ofFIG. 16 as viewed from diagonally front left according to a conventional method; -
FIG. 18 is a diagram showing an all-round bird's-eye view image generated from photographed images obtained by the respective cameras ofFIG. 16 according to the conventional method; -
FIG. 19 is a diagram of the vehicle ofFIG. 16 as viewed from diagonally left front according to the conventional method; -
FIG. 20 is a diagram for explaining the conventional method for generating the all-round bird's-eye view image ofFIG. 18 ; -
FIG. 21 is a diagram for explaining the conventional method for generating the all-round bird's-eye view image ofFIG. 18 ; -
FIG. 22 is a diagram showing an example of a display image when the conventional method is employed; -
FIG. 23 is a diagram showing another example of a display image when the conventional method is employed; -
FIG. 24 is a flowchart showing procedures of processing performed by a visual field support device ofFIG. 8 according to Example 2 of the invention; -
FIG. 25A is a diagram showing a difference region for an image obtained by the left side camera; -
FIG. 25B is a diagram showing a difference region for an image obtained by the front camera; and -
FIG. 26 is a diagram showing an example of an all-round bird's-eye view image generated by the image processing section ofFIG. 8 according to Example 2 of the invention. - Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the figures referenced, the same portions are numbered with the same numerals and thus overlapping explanation for the same portions will be basically omitted. Examples 1 to 3 will be described later, and items common between these Examples or items referenced in these Examples will be described first.
- First, the method for generating a bird's-eye view image from a photographed image taken by one camera will be described. The description below is based on the assumption that the ground is located on the horizontal plane and that “height” represents height with respect to the ground.
- As shown in
FIG. 1 , consider a case where acamera 1 is installed at the back of avehicle 100 diagonally downward thereof. Thevehicle 100 is, for example, a truck. The angle formed by the horizontal plane and the optical axis of thecamera 1 includes two types of angles expressed by θ and θ2, respectively, shown inFIG. 1 . The angle θ2 is typically called a look-down angle or an angle of depression. Now, on the assumption that the angle θ is a tilt angle of thecamera 1 with respect to the horizontal plane, 90°<0<180° holds. -
FIG. 2 shows relationship between a camera coordinate system XYZ, a coordinate system XbuYbu of an image-sensing surface S of thecamera 1, and a worldwide coordinate system XwYwZw including a two-dimensional ground coordinate system XwYw. The camera coordinate system XYZ is a three-dimensional coordinate system with X, Y, and Z axes provided as coordinates axes. The coordinate system XbuYbu is a two-dimensional coordinate system with Xbu and Ybu axes provided as coordinate axes. The two-dimensional ground coordinate system XwYw is a two-dimensional coordinate system with Xw and Yw axes provided as coordinates axes. The worldwide coordinate system XwYwZw is a three-dimensional coordinate system with Xw, Yw, and Zw axes provided as coordinate axes. - In the camera coordinate system XYZ, with the optical center of the
camera 1 provided as an origin O, the Z-axis is provided along the optical axis, the X-axis is provided in a direction orthogonal to the Z-axis and parallel to the ground, and the Y-axis is provided in a direction orthogonal to the Z- and X-axes. In the coordinate system XbuYbu of the image-sensing surface S, the center of the image-sensing surface S is provided as an origin, the Xbu-axis is provided in the lateral direction of the image-sensing surface S, and the Ybu-axis is provided in the longitudinal direction of the image-sensing surface S. - In the worldwide coordinate system XwYwZw, an intersection between a vertical line passing through the origin O of the camera coordinate system XYZ and the ground is provided as an origin Ow, the Yw-axis is provided in a direction perpendicular to the ground, the Xw-axis is provided in a direction parallel to the X-axis of the camera coordinate system XYZ, and the Zw-axis is provided in a direction orthogonal to the Xw- and Yw-axes.
- The amount of parallel movement between the Xw- and X-axes is “h” and the direction of this parallel movement is vertical to the ground. An obtuse angle formed by the Zw- and Xw-axes agrees with the tilt angle θ.
- Coordinates in the camera coordinate system XYZ are expressed as (x, y, z). Symbols x, y, and z are an X-axis component, a Y-axis component, and a Z-axis component, respectively, in the camera coordinate system XYZ. Coordinates in the worldwide coordinate system XwYwZw are expressed as (xw, yw, zw). Symbols xw, yw, and z w are an Xw-axis component, an Yw-axis component, and a Zw-axis component, respectively, in the worldwide coordinate system XwYwZw. Coordinates in the two-dimensional ground coordinate system XwZw are expressed as (xw, zw). Symbols xw and Zw are an Xw-axis component and a Zw-axis component, respectively, in the two-dimensional ground coordinate system XwZw, and they agree with the Xw-axis component and the Zw-axis component in the worldwide coordinate system XwYwZw. Coordinates in the coordinate system XbuYbu of the image sensing surface S are expressed as (Xbu, Ybu). Symbols xbu and ybu are an Xbu-axis component and an Ybu-axis component, respectively, in the coordinate system XbuYbu of the image sensing surface S.
- Equation for conversion between the coordinates (x, y, z) of the camera coordinate system XYZ and the coordinates (xw, yw, zw) of the worldwide coordinate system XwYwZw is expressed by equation (1) below:
-
- Provided that the focal length of the
camera 1 is “f”, equation for conversion between the coordinates (xbu, ybu) of the coordinate system XbuYbu of the image sensing surface S and the coordinates (x, y, z) of the camera coordinate system XYZ is expressed by equation (2) below: -
- Obtained based on the above equations (1) and (2) is an equation (3) below for conversion between the coordinates (xbu, ybu) of the coordinate system XbuYbu of the image sensing surface S and the coordinates (xw, zw) of the two-dimensional ground coordinate system XwZw:
-
- Although not shown in
FIG. 2 , a bird's-eye view coordinate system XauYau as a coordinate system for a bird's-eye view image is now defined. The bird's-eye view coordinate system XauYau is a two-dimensional coordinate system with a Xau-axis and a Yau-axis provided as coordinate axes. The coordinates in the bird's-eye view coordinate system XauYau are expressed as (xau, yau). The bird's-eye view image is expressed by a pixel signal of a plurality of pixels arrayed two-dimensionally. The position of each pixel on the bird's-eye view image is expressed by the coordinates (xau, yau). The symbols xau and yau are a xau-axis component and a Yau-axis component, respectively, in the bird's-eye view coordinate system XauYau. - The bird's-eye view image is obtained by converting an image photographed by an actual camera into an image as viewed from a visual point of a virtual camera (hereinafter referred to as a virtual visual point). More specifically, the bird's-eye view image is obtained by converting an image photographed by the actual camera into an image as the ground surface is looked down vertically. The conversion of the visual point at the time of generating a bird's-eye view image from a photographed image is typically called visual point conversion.
- Projection from the two dimensional ground coordinate system XwZw to the bird's-eye view coordinate system XauYau of the virtual camera is performed through parallel projection. Where the height of the virtual camera (that is, height of the virtual visual point) is H, an equation for conversion between the coordinates (xw, zw) of the two dimensional ground coordinate system XwZw and the coordinates (xau, yau) of the bird's-eye view coordinate system XauYau is expressed by equation (4) below. The height H of the virtual camera is previously set. Further, equation (5) below is obtained by modifying the equation (4):
-
- Substituting the equation (5) obtained into the equation (3) provides equation (6) below:
-
- From the equation (6), equation (7) below for converting the coordinates (xbu, ybu) of the coordinate system XbuYbu of the image-sensing surface S into the coordinates (xau, yau) of the bird's-eye view coordinate system XauYau is obtained:
-
- The coordinates (xbu, ybu) of the coordinate system XbuYbu of the image-sensing surface S express the coordinates on the image photographed by the
camera 1. Thus, the image photographed by thecamera 1 is converted into a bird's-eye view image by employing the equation (7). In practice, image processing such as lens distortion correction or the like is performed as appropriate on the image photographed by thecamera 1 and then the photographed image already subjected to the image processing is converted into the bird's-eye view image by employing the equation (7). - In this embodiment, a plan view showing a condition that the camera is installed at the
vehicle 100 is the same asFIG. 16 described above, and thus overlapping illustration will be omitted.FIGS. 3 and 4 are diagrams of thevehicle 100 as viewed diagonally from front left. - As shown in
FIG. 16 , cameras (photographing apparatuses) 1F, 1B, 1L, and 1R are respectively fitted at the front, rear, left side, and right side of thevehicle 100. In the description below, thecameras front camera 1F, arear camera 1B, aleft side camera 1L, and aright side camera 1R, respectively. - Moreover, as shown in
FIGS. 3 and 4 , thecamera 1F is installed, for example, at the upper part of the front mirror of thevehicle 100. Thecamera 1L is installed, for example, at the uppermost part of the left side surface of thevehicle 100. Although not shown inFIGS. 3 and 4 , thecamera 1B is installed, for example, at the rear uppermost part of thevehicle 100, and thecamera 1R is installed, for example, at the uppermost part of the right side surface of thevehicle 100. - The
camera 1F, thecamera 1B, thecamera 1L, and thecamera 1R are installed at thevehicle 100 so that the optical axis of thecamera 1F is directed diagonally downward to the front of thevehicle 100, so that the optical axis of thecamera 1B is directed diagonally downward to the rear of thevehicle 100, so that the optical axis of thecamera 1L is directed diagonally downward to the left of thevehicle 100, and so that the optical axis of thecamera 1R is directed diagonally downward to the right of thevehicle 100. - The heights of installation positions of the
cameras camera 1F. Thevehicle 100 is located on the ground. -
FIGS. 3 and 4 show visual fields, that is, regions photographed by the respective cameras. Each of the cameras generates a photographed image of a subject taken within its own visual field. The visual fields of thecameras visual fields FIGS. 3 and 4 . - The
visual field 12F of thecamera 1F includes a solid object located in a given range in front of thevehicle 100 and the ground in front of thevehicle 100, with reference to the installation position of thecamera 1F. Thevisual field 12B of thecamera 1B includes a solid object located in a given range behind thevehicle 100 and the ground behind thevehicle 100, with reference to the installation position of thecamera 1B. Thevisual field 12L of thecamera 1L includes a solid object located in a given range to the left of thevehicle 100 and the ground to the left of thevehicle 100, with reference to the installation position of thecamera 1L. Thevisual field 12R of thecamera 1R includes a solid object located in a given range to the right of thevehicle 100 and the ground to the right of thevehicle 100, with reference to the installation position of thecamera 1R. - As described above, the different cameras have different visual points and thus different subjects fit within the visual fields (angles of views) of the respective cameras. The solid object (in other words, three-dimensional object) is an object, such as a person, which has some height. A road surface forming the ground or the like is not a solid object since it has no height.
- The
cameras vehicle 100, in common with each other. That is, thevisual fields vehicle 100. This overlapping portion is called a common visual field (common photographing space). - Similarly, the
visual fields vehicle 100, thereby forming their common visual field. Thevisual fields vehicle 100, thereby forming their common visual field. Thevisual fields vehicle 100, thereby forming their common visual field. - The common visual field between the
visual fields visual field 13. Hereinafter, description is given focusing on this commonvisual field 13. The same processing applies to the common visual fields other than the commonvisual field 13. -
FIG. 4 shows the commonvisual field 13 with a solid line. The commonvisual field 13 is a space similar to a cone having, as a bottom surface, the ground located diagonally leftward and forward of thevehicle 100. As shown inFIG. 4 , asolid object 14 is assumed to be present within the commonvisual field 13. - In this embodiment, as shown in
FIG. 5 , from photographed images obtained from thecameras eye view images eye view image 10B corresponding to thecamera 1B, the other bird's-eye view images eye view image 10B by rotating and/or parallely moving them (bird's-eye view images -
FIG. 6 shows the bird's-eye view images FIG. 6 . - In
FIG. 6 , a diagonally shaded region provided with CFL is a portion where the bird's-eye view images eye view image 10F, the image of the subject within the common visual field 13 (seeFIG. 4 ) as viewed from thecamera 1F appears in the common region CFL. In the bird's-eye view image 10L, the image of the subject within the commonvisual field 13 as viewed from thecamera 1L appears in the common region CFL. The common region can also be called an overlapping region where a plurality of bird's-eye view images overlap. - In addition to the common region CFL, there are a common region CFR where the bird's-
eye view images eye view images eye view images visual field 13. - In
FIGS. 5 and 6 , an XF-axis and an YF-axis are coordinate axes of the coordinate system of the bird's-eye view image 10F, and they correspond to a Xau-axis and a Yau-axis. Similarly, an XR-axis and a YR-axis are coordinate axes of the coordinate system of the bird's-eye view image 10R, and they correspond to a Xau-axis and a Yau-axis. Similarly, an XL-axis and an YL-axis are coordinate axes of the coordinate system of the bird's-eye view image 10L, and they correspond to a Xau-axis and a Yau-axis. Similarly, an XB-axis and an YB-axis are coordinate axes of the coordinate system of the bird's-eye view image 10B, and they correspond to a Xau-axis and a Yau-axis. - In
FIG. 6 , the common region CFL is rectangular for simplified illustration purposes, but the actual common region CFL is not rectangular. In addition, each of the bird's-eye view images is not necessarily rectangular.FIGS. 7A , 7B, and 7C show in more detail the regions where the respective bird's-eye view images appear and the common region CFL. -
FIGS. 7A and 7B respectively show the bird's-eye view images FIG. 7C shows the common region CFL thereof indicated by a diagonally shaded region. Note that illustration of an image near the rear of the vehicle is omitted inFIGS. 7A and 7C . The characteristic of this embodiment lies in how two bird's-eye view images are synthesized in this common region CFL. -
FIG. 8 shows an overall block diagram of a visual field support device (vehicle surrounding visual field support device) according to this embodiment. The visual field support device according to this embodiment includes:cameras vehicle 100 as described above; animage processing section 2 which generates an all-round bird's-eye view image from photographed images obtained by the respective cameras; and a display section (display device) 3 which displays the all-round bird's-eye view image generated by theimage processing section 2. The all-round bird's-eye view image is basically the same as that shown inFIG. 18 . Note that this embodiment is characterized in a technique of synthesis in the common region described above. - Used as the
cameras image processing section 2 is composed of, for example, an integrated circuit. Thedisplay section 3 is composed of a liquid crystal display panel or the like. A display device included in a car navigation system or the like may be used as thedisplay section 3 in the visual field support device. - Operation and the like of the visual field support device shown in
FIG. 8 will be described in detail. Examples 1 to 3 of the invention will be described below. - First, Example 1 will be described.
FIG. 9 shows a flowchart representing procedures of processing performed by the visual field support device ofFIG. 8 according to Example 1. The processing in steps S2 to S6 shown inFIG. 9 is performed by theimage processing section 2, the processing in step S1 is performed by the cameras and theimage processing section 2, and the processing in step S7 is performed by theimage processing section 2 and thedisplay section 3. - First, the
image processing section 2 reads images photographed by therespective cameras image processing section 2 converts each of the photographed images into a bird's-eye view image by using conversion table data or the like (step S2), whereby the bird's-eye view images - Subsequently, in step S3, the
image processing section 2 generates a difference image in the common region between the two bird's-eye view images, and then based on this difference image, detects a difference region in the common region between the two bird's-eye view images. Although there are four common regions as described referring toFIG. 6 , the description below will be focused on the common region CFL. The same processing is performed on the other common regions. - The difference image and the difference region for the common region CFL will be described. As described above, it is assumed that the
solid object 14 is present in the commonvisual field 13 corresponding to the common region CFL(seeFIG. 4 ). - Referring to
FIG. 10 , numeral 111 represents an image of the common region CFL within the bird's-eye view image 10L corresponding to theleft side camera 1L, and numeral 112 represents an image of the common region CFL within the bird's-eye view image 10F corresponding to thefront camera 1L. Aregion 121 shaded in theimage 111 is a region where thesolid object 14 is drawn in theimage 111. Aregion 122 shaded in theimage 112 is a region where thesolid object 14 is drawn in theimage 112. Now, for simplified description purposes, it is assumed that the even ground (for example, road surface) is drawn in a portion other than theregion 121 in theimage 111 and that the even ground (for example, road surface) is drawn in a portion other than theregion 122 in theimage 112. - In step S3, the
image processing section 2 generates adifference image 113 by obtaining a difference between theimages image processing section 2 perceives theimages difference image 113 of theimages difference image 113 are expressed by difference (or an absolute value of difference) between pixel values of respective pixels in theimage 111 and pixel values of respective pixels in theimage 112. Then, theimage processing section 2 identifies adifference region 123 from thedifference image 113. InFIG. 10 , thedifference region 123 is a region shaded inside thedifference image 113. For example, in thedifference image 113, thedifference region 123 is defined as a region where the values of the pixel signals of the pixels forming thedifference image 113 are equal to or larger than a predetermined threshold. The pixel signal is a brightness signal representing the brightness of the pixel or a color signal (color difference signal or the like) representing the color of the pixel. - The
difference region 123 is a region combining together theregions difference image 113, a difference value is obtained which is relatively large for only a region related to the presence of the solid object. Thus, thedifference region 123 can be detected by the processing described above. Alternatively, edge detection may be performed on thedifference image 113 to detect the contour surrounding thedifference region 123 and then the inside of the detected contour may be specified as thedifference region 123. Alternatively, edge detection may be performed individually on theimages images difference region 123 from an image portion having the difference. - The processing in step S4 following step S3 will be described, referring to the common region CFL as an example. The same processing applies to the other common regions.
- In step S4, weight value table data set for each camera is referenced. For the common region CFL, the weight value table data corresponding to the
front camera 1F and the weight value table data corresponding to theleft side camera 1L are previously set. - In each weight value table data, weight values corresponding to respective pixel positions of the image in the common region CFL are defined. For example, the common region CFL is divided into three regions AR1, AR2, and AR3 as show in
FIG. 11 . The weight value of the pixel belonging to the region AR1, the weight value of the pixel belonging to the region AR2, and the weight value of the pixel belonging to the region AR3 are stored into each weight value table data. - The weight values defined in relation to the pixel position of the image in the common region CFL are set differently among the different weight value table data.
FIG. 12 shows an example of setting the weight values corresponding to the front camera F and the weight values corresponding to theleft side camera 1L. For example, in the weight value table data corresponding to thefront camera 1F, the weight value WF1 for the pixel belonging to the region AR1 is 1, the weight value WF2 for the pixel belonging to the region AR2 is 2, and the weight value WF3 for the pixel belonging to the region AR3 is 3. On the other hand, in the weight value table data corresponding to theleft side camera 1L, the weight value WL1 for the pixel belonging to the region AR1 is 2, the weight value WL2 for the pixel belonging to the region AR2 is 3, and the weight value WL3 for the pixel belonging to the region AR3 is 2. - In step S4, the weight values corresponding to the pixels belonging to the
difference region 123 are summed up for each of the cameras (in other word, for each of the bird's-eye view images), and this summed value is provided as an evaluated weight value. For explanatory purposes, as shown inFIG. 13 , thedifference region 123 is considered as being divided into apartial difference region 123 a corresponding to the region 121 (seeFIG. 10 ) and apartial difference region 123 b corresponding to theregion 122. -
FIG. 14 shows the number of pixels in thepartial difference region 123 a and the number of pixels in thepartial difference region 123 b, respectively belonging to the regions AR1 to AR3. Assume that the number of pixels in thepartial difference region 123 a belonging to the regions AR1 to AR3 are 15, 18, and 2, respectively, and the number of pixels in thepartial difference region 123 b belonging to the regions AR1 to AR3 are 12, 5, and 0, respectively - In this case, the evaluated weight value for the
front camera 1F is expressed by: -
“WF1×(15+12)+WF2×(18+5)+WF3×(2+0)”. -
“1×(15+12)+2×(18+5)+3×(2+0)=79” is obtained. - On the other hand, the evaluated weight value for the
left side camera 1L is expressed by: -
“WL1×(15+12)+WL2×(18+5)+WL3×(2+0)”. -
“2×(15+12)+3×(18+5)+2×(2+0)=127” is obtained. - In step S5 following step S4, based on the evaluated weight values calculated in step S4, a priority camera is selected which provides the bird's eye view image including the image to be adopted as the image of the common region. As described above, when, concerning the common region CFL, the evaluated weight value for the
front camera 1F is 79 and the evaluated weight value for theleft side camera 1L is 127, theleft side camera 1L corresponding to the larger evaluated weight value is selected as the priority camera. In other word, the bird's-eye view image 10L corresponding to theleft side camera 1L is selected as the bird's-eye view image to be adopted (adopted image). The priority cameras are selected in the same manner for the other common regions. - Then, in step S6, the
image processing section 2, based on the results of selection of the priority camera, generates an all-round bird's-eye view image. That is, adopting, as an image of a common region in the all-round bird's-eye view image, the image of the common region in the bird's-eye view image based on the image photographed by the priority camera, theimage processing section 2 generates the all-round bird's-eye view image. For example, when theleft side camera 1L is selected as the priority camera for the common region CFL, the image of the common region CFL in the bird's-eye view image 10L is provided as the image of the common region CFL in the all-round bird's-eye view image. The same applies to the other common regions. -
FIG. 15 shows an example of the all-round bird's-eye view image obtained. In this all-round bird's-eye view image, an entire image of the solid object appears. For images of the regions other than the common region in the all-round bird's-eye view image, the bird's-eye view images FIG. 6 . That is, on the all-round bird's-eye view image, the images based on the bird's-eye view images - In step S7 following step S6, the
image processing section 2 generates a required video signal and outputs it to thedisplay section 3 so that the all-round bird's-eye view image generated in step S6 is displayed on thedisplay section 3. Consequently, the all-round bird's-eye view image as shown inFIG. 15 is displayed on thedisplay section 3. When step S7 ends, the processing returns to step S1, so that the processing of steps S1 to S7 is repeated so as to periodically update the all-round bird's-eye view image. - The weight values described above are set in accordance with the installation positions of the respective cameras with respect to the
vehicle 100, upon which the shape of thevehicle 100 and the like are also taken into consideration. This embodiment is, as shown inFIG. 3 , based on the assumption that thevehicle 100 is a truck composed of a driver's cabin and a luggage compartment having a height larger than that of the driver's cabin and that thefront camera 1F is installed at the top of the driver's cabin and theleft side camera 1L is installed at the top of the luggage compartment located higher than thefront camera 1F. - The
front camera 1F can precisely capture a solid object located close thereto but has difficulty in capturing an entire image of a solid object located distant therefrom due to its low installation position. On the other hand, theleft side camera 1L can easily capture an entire image of a solid object by viewing the solid object from higher position, but has difficulty in capturing an entire image of a solid object located close to thefront camera 1F. - Considering these facts, in order that the
front camera 1F is likely to be selected as a priority camera when a solid object is present at position relatively close to thefront camera 1F, the weight values for thefront camera 1F are set relatively large in a region close to thefront camera 1F, as shown inFIG. 12 . On the other hand, in order that theleft side camera 1L is likely to be selected as a priority camera when a solid object is present at position distant from thefront camera 1F, the weight values for theleft side camera 1L are set relatively large in a region distant form thefront camera 1F. - Setting the weight values in accordance with the installation position of each of the cameras in this manner makes it easy to display the entire image of the solid object as shown in
FIG. 15 . Specifically, the camera is automatically selected which is expected to be capable of obtaining an image from which the driver can easily recognize the condition of the surrounding of the vehicle and then the image of the common region in the all-round bird's-eye view image is formed, which permits display of a video more suitable for condition recognition than a conventional one. - A difference in the position and shape of the difference region in the common region may result in a difference in the priority camera to be selected. Even with the same shape of the difference region, a difference in the position of the difference region in the common region can result in a difference in the priority camera to be selected. That is, in this embodiment, the priority camera is selected in accordance with the position of the difference region in the common region (in other word, in accordance with the position of the solid object in relation to the vehicle 100). An advantage in adopting this selection method can be understood from the above description. Specifically, for example, for the common region CFL, when the solid object is present at position relatively close to the
front camera 1F, thefront camera 1F is likely to be selected as the priority camera, and when the solid object is present at position relatively distant from thefront camera 1F, theleft side camera 1L which views the solid object from higher position is likely to be selected as the priority camera. As a result, a video more suitable for condition recognition than a conventional one can be displayed. - In step S5, each of the evaluated weight values described above may be corrected in accordance with the driving condition of the
vehicle 100, and the priority camera (priority bird's-eye view image) may be selected based on each of the evaluated weight values corrected. - The driving condition includes: condition of the gear of the
vehicle 100, direction of handle operation (travel direction of the vehicle 100), speed of thevehicle 100, and the like. Information concerning the driving condition is given from a portion (for example, portion included in the vehicle 100) detecting this information to theimage processing section 2. - Consider a case where the evaluated weight values for the
front camera 1F and theleft side camera 1L before this correction are 79 and 127, respectively, as described above. For example, when thevehicle 100 takes a left turn, the evaluated weight value for thefront camera 1F is multiplied by acoefficient 1 and the evaluated weight value for theleft side camera 1L is multiplied by acoefficient 2. In this case, the evaluated weight values for thefront camera 1F and theleft side camera 1L after correction are 79 (=79×1) and 254 (=127×2), respectively. Since 79 is smaller than 254, theleft side camera 1L is selected as the priority camera. In addition, for example, when thevehicle 100 moves forward, the evaluated weight value for thefront camera 1F is multiplied by acoefficient 2 and the evaluated weight value for theleft side camera 1L is multiplied by acoefficient 1. In this case, the evaluated weight values for thefront camera 1F and theleft side camera 1L after correction are 158 (=79×2) and 127 (=127×1), respectively. Since 158 is larger than 127, thefront camera 1F is selected as the priority camera. - When the
vehicle 100 takes a left turn, the need for a video of the left side of thevehicle 100 is relatively great while the need for a video of the front of thevehicle 100 is relatively small. When thevehicle 100 moves forward, the need for the video of the left side of thevehicle 100 is relatively small while the need for the video of the front of thevehicle 100 is relatively great. Taking this into consideration, the coefficients are set to correct the evaluated weight values as described above. As a result, a suitable video also considering the driving operation can be displayed. - The illustration above refers to the left turn and the forward movement. Note that the evaluated weight values are corrected in accordance with the driving condition such as a right turn, backward movement, or the like, based on the same idea. Moreover, the degree of correction may be changed in accordance with the condition of the gear of the
vehicle 100, speed of thevehicle 100, or the like. - Next, Example 2 of the invention will be described.
FIG. 24 shows a flowchart representing procedures of processing performed by the visual field support device ofFIG. 8 according to Example 2. The processing in steps S12 to S15 shown inFIG. 24 is performed by theimage processing section 2, the processing in step S11 is performed by the cameras and theimage processing section 2, and the processing in step S16 is performed by theimage processing section 2 and thedisplay section 3. - First, the
image processing section 2 reads images photographed by therespective cameras image processing section 2 converts each of the photographed images into a bird's-eye view image by using conversion table data or the like (step S12), whereby the bird's-eye view images - Subsequently, in step S13, the
image processing section 2 generates a difference image in a common region between the two bird's-eye view images, and then based on this difference image, detects a difference region in the common region between the two bird's-eye view images. Although there are four common regions as described referring toFIG. 6 , the description below will be focused on the common region CFL. The same processing is performed on the other common regions. - The difference image and the difference region for the common region CFL are the same as those described in Example 1 with reference to
FIG. 10 . As described above, it is assumed that thesolid object 14 is present in the commonvisual field 13 corresponding to the common region CFL(seeFIG. 4 ). - Description of this Example also refers to
FIG. 10 . The processing in step S113 is the same as the processing in step S3 (seeFIG. 9 ) in Example 1. Specifically, in step S13, theimage processing section 2 generates adifference image 113 by obtaining a difference betweenimages image processing section 2 specifies adifference region 123 from thedifference image 113. As described above, inFIG. 10 , thedifference region 123 is a region shaded inside thedifference image 113. The way of specifying thedifference region 123 is the same as that described in Example 1. That is, for example, in thedifference image 113, thedifference region 123 is defined as a region where values of pixel signals of pixels forming thedifference image 113 are equal to or larger than a predetermined threshold. - The
difference region 123 is a region (solid object reflecting region) synthesizing aregion 121 where thesolid object 14 is drawn in theimage 111 and aregion 122 where thesolid object 14 is drawn in theimage 112. Upon the conversion into the bird's-eye view images, predetermined conversion is performed so that each of the bird's-eye view images has continuity on the ground. Thus, for thedifference image 113, a difference value is obtained which is relatively large for only a region related to the presence of the solid object. Thus, thedifference region 123 can be detected by the processing described above. - Now, consider the
difference region 123 in each of the bird's-eye view images FIG. 25A , for the bird's-eye view image 10L corresponding to theleft side camera 1L, thedifference region 123 is formed of a region where thesolid object 14 photographed by theleft side camera 1L is drawn (that is, a region 121) and a blockingregion 121 a blocked by thesolid object 14 as viewed from thefront camera 1F. In the bird's-eye view image 10L, the ground appears in this blockingregion 121 a. On the other hand, as shown inFIG. 25B , for the bird's-eye view image 10F corresponding to thefront camera 1F, thedifference region 123 is formed of a region where thesolid object 14 photographed by thefront camera 1F is drawn (that is, a region 122) and a blockingregion 122 a blocked by thesolid object 14 as viewed from theleft side camera 1L. In the bird's-eye view image 10F, the ground appears in this blockingregion 122 a. - Considering these characteristics, upon the synthesis of the bird's-
eye view images eye view images solid object 14 can be appropriately displayed without being lost. - In step S14 following step S13, a priority camera is selected which provides a bird's-eye view image including an image to be adopted as an image of a difference region (for example, difference region 123). For the
difference region 123, either one of thefront camera 1F and theleft side camera 1L is selected as the priority camera. For example, theleft side camera 1L installed at higher position is selected as the priority camera for thedifference region 123, because theleft side camera 1L installed at the higher position can capture the entire image of the solid object with relative ease since it views the solid object from a higher visual point. The priority cameras for the other difference regions are selected in the same manner. For the selection of the priority camera based on height of installation position, it is previously defined in accordance with height of each of the cameras which one of the cameras is selected as the priority camera. There are various other possible methods of selecting a priority camera, which will be described in Example 3 later. - In step S15 following step S14, the
image processing section 2, with reference to results of the processing in steps S13 and 14, generates an all-round bird's-eye view image. Specifically, theimage processing section 2 generates the all-round bird's-eye view image from the bird's-eye view images as described with reference toFIGS. 5 , 6, and the like. For the difference region in the common region, the image of the difference region based on the image photographed by the priority camera is adopted. For example, for the common region CFL, when thedifference region 123 is detected in step S13 and theleft side camera 1L is selected as the priority camera in step S14, an image of thedifference region 123 in the bird's-eye view image 10L (that is, image of a region combining together theregions FIG. 25A ) is adopted as an image of thedifference region 123 in the all-round bird's-eye view image. - For a region excluding the
difference region 123 in the common region CFL, an image of this region (region excluding thedifference region 123 in the common region CFL) in the all-round bird's-eye view image is generated by averaging pixel signals in the bird's-eye view images eye view images difference region 123 in the common region CFL, an image of either one of the bird's-eye view images difference region 123 in the common region CFL) in the all-round bird's-eye view image. The processing in step S15 has been described focusing on the common region CFL, but the same processing applies to the other common regions. -
FIG. 26 shows an example of the all-round bird's-eye view image obtained. In this example, in which theleft side camera 1L is a priority camera for thedifference region 123, an entire image of the solid object appears in all-round bird's-eye view image. For images of the regions other than the common region in the all-round bird's-eye view image, the bird's-eye view images FIG. 6 . That is, on the all-round bird's-eye view image, the images based on the bird's-eye view images - In step S16 following step S15, the
image processing section 2 generates a required video signal and outputs it to thedisplay section 3 so that the all-round bird's-eye view image generated in step S15 is displayed on thedisplay section 3. Consequently, the all-round bird's-eye view image as shown inFIG. 26 is displayed on thedisplay section 3. When step S16 ends, the processing returns to step S11, so that the processing of steps S1 to S16 is repeated so as to periodically update the all-round bird's-eye view image. - Performing the synthesis processing as in this Example can avoid a problem that the solid object disappears from the display screen and also can display the solid object which does not result in a double image thereof. Moreover, for the regions other than the region displaying the solid object, continuous images are synthesized, so that a video suitable for recognizing condition of the surrounding of the vehicle can be displayed.
- Next, Example 3 of the invention will be described. In Example 3, another example of the method for selecting a priority camera performed in step S14 of
FIG. 24 will be described. Example 3 is performed in combination with the Example 2. - As another example of the method for selecting a priority camera, a first, a second, and a third selection methods will be illustrated. Focusing on the
difference region 123 in the common region CFL, the first to third selection methods will be described. Needless to say, a priority camera is selected in the same manner for the other difference regions. The first to third selection methods can be combined arbitrarily as long as no inconsistency arises. - The first selection method will be described. In the first selection method, distances between the
solid object 14 and the cameras are referenced. Recognizing the position of thedifference region 123 on the image permits specifying, in an actual space, a distance between thesolid object 14 and thefront camera 1F and a distance between thesolid object 14 and theleft side camera 1L. The both distances are compared with each other, and the camera with the shorter distance is selected as the priority camera. This selection is based on the assumption that, when other conditions are identical, the camera with the shorter distance can capture thesolid object 14 more precisely. - The second selection method will be described. In the second selection method, the travel direction of the
vehicle 100 is referenced. Information on the travel direction is given from a portion for detecting this information (this portion is included, for example, in the vehicle 100) to theimage processing section 2. - For example, for the
difference region 123, the left side camera 1I, is selected as the priority camera when thevehicle 100 takes a left turn while thefront camera 1F is selected as the priority camera when thevehicle 100 moves forward. When thevehicle 100 takes a left turn, the need for a video of the left side of thevehicle 100 is relatively great, while the need for a video of the front of thevehicle 100 is relatively small. When thevehicle 100 moves forward, the need for the video of the left side of thevehicle 100 is relatively small, while the need for the video of the front of thevehicle 100 is relatively great. Taking this into consideration, the priority cameras are selected as described above. Consequently, a suitable video also considering the travel direction of thevehicle 100 can be displayed. - The illustration above refers to the left turn and the forward movement. Note that, based on the same idea, the priority camera is selected in accordance with driving condition such as a right turn, rearward movement, or the like.
- The third selection method will be described. In the third selection method, the priority camera is selected based on the position of the
difference region 123 in the common region CFL. The third selection method is basically identical to the method for selecting a priority camera in Example 1. The third selection method will be described, referring to details of description of Example 1 and the drawings also referenced in Example 1. - In the third selection method, weight value table data set for each camera is referenced. For the common region CFL, the weight value table data corresponding to the
front camera 1F and the weight value table data corresponding to theleft side camera 1L are previously set. - In each weight value table data, weight values corresponding to respective pixel positions of the image in the common region CFL are defined. For example, the common region CFL is divided into three regions AR1, AR2, and AR3 as show in
FIG. 11 . The weight value of the pixel belonging to the region AR1, the weight value of the pixel belonging to the region AR2, and the weight value of the pixel belonging to the region AR3 are stored into each weight value table data. - As in Example 1, the weight values defined in relation to the pixel positions of the image of the common region CFL are set differently among the different weight value table data. The example of setting the weight values corresponding to
FIG. 12 is also applied to the third selection method. For example, in the weight value table data corresponding to thefront camera 1F, the weight value WF1 for the pixel belonging to the region AR1 is 1, the weight value WF2 for the pixel belonging to the region AR2 is 2, and the weight value WF3 for the pixel belonging to the region AR3 is 3. On the other hand, in the weight value table data corresponding to theleft side camera 1L, the weight value WL1 for the pixel belonging to the region AR1 is 2, the weight value WL2 for the pixel belonging to the region AR2 is 3, and the weight value WL3 for the pixel belonging to the region AR3 is 2. - Then the weight values corresponding to the pixels belonging to the
difference region 123 are summed up for each of the cameras (in other word, for each of the bird's-eye view images), and this summed value is provided as an evaluated weight value. In this condition, as in Example 1, as shown inFIG. 13 , thedifference region 123 is considered to be divided into apartial difference region 123 a corresponding to the region 121 (seeFIG. 10 ) and apartial difference region 123 b corresponding to theregion 122. - Now, as is the case with the numerical example of Example 1 (see
FIG. 14 ), the number of pixels in thepartial difference region 123 a belonging to the regions AR1 to AR3 are 15, 18, and 2, respectively, and the number of pixels in thepartial difference region 123 b belonging to the regions AR1 to AR3 are 12, 5, and 0, respectively. - In this case, as in Example 1, the evaluated weight value for the
front camera 1F is expressed by: -
“WF1×(15+12)+WF2×(18+5)+WF3×(2+0)”. -
“1×(15+12)+2×(18+5)+3×(2+0)=79” is obtained. - On the other hand, as in Example 1, the evaluated weight value for the
left side camera 1L is expressed by: -
“WL1×(15+12)+WL2×(18+5)+WL3×(2+0)”. -
“2×(15+12)+3×(18+5)+2×(2+0)=127” is obtained. - Of the both evaluated weight values which have been calculated, the camera corresponding to the larger evaluated weight value is selected as the priority camera. In this example, since 79<127, the
left side camera 1L is selected as the priority camera. - The
front camera 1F can precisely capture a solid object located close thereto but has difficulty in capturing an entire image of a solid object located distant therefrom due to its low installation position. On the other hand, theleft side camera 1L can easily capture an entire image of a solid object by viewing the solid object from higher position, but has difficulty in capturing an entire image of a solid object located close to thefront camera 1F. - Considering these facts, in order that the
front camera 1F is likely to be selected as a priority camera when a solid object is present at position relatively close to thefront camera 1F, the weight values for thefront camera 1F are set relatively large in a region close to thefront camera 1F, as shown inFIG. 12 . On the other hand, in order that theleft side camera 1L is likely to be selected as a priority camera when a solid object is present at position distant from thefront camera 1F, the weight values for theleft side camera 1L are set relatively large in a region distant form thefront camera 1F. - Setting the weight values in accordance with the installation position of the respective cameras and the like in this manner makes it easy to display the entire image of the solid object. When the third selection method is adopted, a difference in the position and shape of the difference region in the common region may result in a difference in the priority camera to be selected. Even with the same shape of the difference region, a difference in the position of the difference region in the common region can result in a difference in the priority camera to be selected. That is, in the third selection method, the priority camera is selected in accordance with the position of the difference region in the common region (in other word, in accordance with the position of the solid object in relation to the vehicle 100). An advantage in adopting this selection method can be understood from the above description.
- The visual field support device described above is just an example of the embodiments of the invention, and the invention includes various Modified Examples (or other Examples). Hereinafter, as the Modified Examples (or other Examples) of the invention, a first to a fifth Modified Examples will be illustrated. Details described in the Modified Examples can be arbitrarily combined together as long as no inconsistency arises.
- The specific values shown in the above description is just illustrative, and it is needless to say that they can be changed to various values. The weight values described above are changed as appropriate in accordance with the installation position of the cameras, the shape of a vehicle where the cameras are installed, and the like.
- In Example 1 and the third selection method of Example 3, an example is illustrated in which the common region CFL is divided into three regions AR1 to AR3, although the number of divided regions may be other than 3. An example is provided in which the method for dividing the common region CFL into regions is the same between the
cameras eye view images - Note that such division of the region is not necessary, as long as the weight values are defined in correspondence with the position of each pixel in the image of the common region CFL, and in extreme cases, weight values corresponding to two different arbitrary pixel positions may all differ from each other.
- In the example of setting the weight values shown in
FIG. 12 , the weight values corresponding to the same pixel position all differ between thecameras eye view images - The
image processing section 2 ofFIG. 8 can be realized by hardware, software, or a combination of hardware and software. All or part of functions achieved by theimage processing section 2 may be described as a program, which may be executed on a computer to thereby achieve all or part of the functions. - The
image processing section 2 includes: a visual point conversion section which converts images photographed by the respective cameras into bird's-eye view images through visual point conversion; and an image synthesis section which synthesizes the bird's-eye view images obtained through the conversion to thereby generate a synthetic bird's-eye view image. As the synthetic bird's-eye view image, an all-round bird's-eye view image is illustrated in the above embodiment. - In the above embodiment, the
vehicle 100 is exemplified by a truck. However, the invention is also applicable to normal passenger cars and the like, regardless of the type of vehicle. - Further, a plurality of cameras (for example, the
cameras
Claims (12)
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2006-224429 | 2006-08-21 | ||
JP2006224002A JP4315968B2 (en) | 2006-08-21 | 2006-08-21 | Image processing apparatus and visibility support apparatus and method |
JP2006224429A JP4248570B2 (en) | 2006-08-21 | 2006-08-21 | Image processing apparatus and visibility support apparatus and method |
JP2006-224002 | 2006-08-21 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20080043113A1 true US20080043113A1 (en) | 2008-02-21 |
US7728879B2 US7728879B2 (en) | 2010-06-01 |
Family
ID=39101023
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/842,685 Active 2028-08-28 US7728879B2 (en) | 2006-08-21 | 2007-08-21 | Image processor and visual field support device |
Country Status (1)
Country | Link |
---|---|
US (1) | US7728879B2 (en) |
Cited By (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102008030104A1 (en) | 2008-06-25 | 2009-02-12 | Daimler Ag | Vehicle i.e. tractor-trailer, surrounding monitoring method, involves assigning positions of image capturing units and weighing factor to each pixel, and determining overall picture between vehicle elements by taking into inclination angle |
WO2010066332A1 (en) * | 2008-12-12 | 2010-06-17 | Daimler Ag | Device for monitoring an environment of a vehicle |
CN101976460A (en) * | 2010-10-18 | 2011-02-16 | 胡振程 | Generating method of virtual view image of surveying system of vehicular multi-lens camera |
US20110157361A1 (en) * | 2009-12-31 | 2011-06-30 | Industrial Technology Research Institute | Method and system for generating surrounding seamless bird-view image with distance interface |
US20110166782A1 (en) * | 2008-12-18 | 2011-07-07 | Aisin Seiki Kabushiki Kaisha | Display device |
US20110310245A1 (en) * | 2010-06-21 | 2011-12-22 | Nissan Motor Co., Ltd. | Travel distance detection device and travel distance detection method |
US20120026333A1 (en) * | 2009-07-29 | 2012-02-02 | Clarion Co., Ltd | Vehicle periphery monitoring device and vehicle periphery image display method |
US20120062745A1 (en) * | 2009-05-19 | 2012-03-15 | Imagenext Co., Ltd. | Lane departure sensing method and apparatus using images that surround a vehicle |
US20130169792A1 (en) * | 2010-08-12 | 2013-07-04 | Valeo Schalter Und Sensoren Gmbh | Method for assisting in a parking operation for a motor vehicle, driver assistance system and a motor vehicle |
US20140119597A1 (en) * | 2012-10-31 | 2014-05-01 | Hyundai Motor Company | Apparatus and method for tracking the position of a peripheral vehicle |
US20140347450A1 (en) * | 2011-11-30 | 2014-11-27 | Imagenext Co., Ltd. | Method and apparatus for creating 3d image of vehicle surroundings |
CN104683706A (en) * | 2013-11-28 | 2015-06-03 | 财团法人金属工业研究发展中心 | Image joint method |
US20150183370A1 (en) * | 2012-09-20 | 2015-07-02 | Komatsu Ltd. | Work vehicle periphery monitoring system and work vehicle |
US20150189170A1 (en) * | 2012-10-05 | 2015-07-02 | Fuji Xerox Co., Ltd. | Information processing apparatus, information processing system and non-transitory computer readable medium |
US20150217690A1 (en) * | 2012-09-21 | 2015-08-06 | Komatsu Ltd. | Working vehicle periphery monitoring system and working vehicle |
US20150341597A1 (en) * | 2014-05-22 | 2015-11-26 | Dr. Ing. H.C. F. Porsche Aktiengesellschaft | Method for presenting a vehicle's environment on a display apparatus; a display apparatus; a system comprising a plurality of image capturing units and a display apparatus; a computer program |
US20160090043A1 (en) * | 2014-09-26 | 2016-03-31 | Hyundai Motor Company | Driver customizable blind spot display method and apparatus |
EP3029601A1 (en) * | 2014-12-04 | 2016-06-08 | Conti Temic microelectronic GmbH | Method and apparatus for providing increased obstacle visibility |
EP2916540A4 (en) * | 2012-10-31 | 2016-10-19 | Clarion Co Ltd | Image processing system and image processing method |
US9767561B2 (en) * | 2013-11-18 | 2017-09-19 | Texas Instruments Incorporated | Method and apparatus for a optimal seam for surround view synthesis |
WO2018108215A1 (en) * | 2016-12-15 | 2018-06-21 | Conti Temic Microelectronic Gmbh | Device for providing improved obstacle detection |
US10157452B1 (en) * | 2015-09-28 | 2018-12-18 | Amazon Technologies, Inc. | Image processing system for image rectification |
US20190025854A1 (en) * | 2017-07-20 | 2019-01-24 | Mohsen Rohani | Method and system for vehicle localization |
US10339656B1 (en) * | 2016-09-29 | 2019-07-02 | Amazon Technologies, Inc. | Inferring count of items using image |
US10397544B2 (en) | 2010-08-19 | 2019-08-27 | Nissan Motor Co., Ltd. | Three-dimensional object detection device and three-dimensional object detection method |
US20190275970A1 (en) * | 2018-03-06 | 2019-09-12 | Aisin Seiki Kabushiki Kaisha | Surroundings monitoring apparatus |
US10417743B2 (en) | 2015-11-06 | 2019-09-17 | Mitsubishi Electric Corporation | Image processing device, image processing method and computer readable medium |
WO2020061794A1 (en) * | 2018-09-26 | 2020-04-02 | 深圳市大疆创新科技有限公司 | Vehicle driver assistance device, vehicle and information processing method |
US20200120371A1 (en) * | 2018-10-10 | 2020-04-16 | Rovi Guides, Inc. | Systems and methods for providing ar/vr content based on vehicle conditions |
US11403742B2 (en) * | 2018-03-28 | 2022-08-02 | Mitsubishi Electric Corporation | Image processing device, image processing method, and recording medium for generating bird's eye synthetic image |
US20220353316A1 (en) * | 2021-04-30 | 2022-11-03 | Woven Planet Holdings, Inc. | Communication device, communication method, and non-transitory computer-readable recording medium |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1696669B1 (en) * | 2005-02-24 | 2013-07-03 | Aisin Seiki Kabushiki Kaisha | Vehicle surrounding monitoring device |
JP2008312004A (en) * | 2007-06-15 | 2008-12-25 | Sanyo Electric Co Ltd | Camera system and mechanical apparatus |
JP5120880B2 (en) | 2007-10-15 | 2013-01-16 | アルパイン株式会社 | Image processing apparatus and image processing method |
JP5108605B2 (en) * | 2008-04-23 | 2012-12-26 | 三洋電機株式会社 | Driving support system and vehicle |
JP5068779B2 (en) * | 2009-02-27 | 2012-11-07 | 現代自動車株式会社 | Vehicle surroundings overhead image display apparatus and method |
JP5344227B2 (en) * | 2009-03-25 | 2013-11-20 | アイシン精機株式会社 | Vehicle periphery monitoring device |
JP5479956B2 (en) * | 2010-03-10 | 2014-04-23 | クラリオン株式会社 | Ambient monitoring device for vehicles |
JP5483120B2 (en) * | 2011-07-26 | 2014-05-07 | アイシン精機株式会社 | Vehicle perimeter monitoring system |
KR101371893B1 (en) * | 2012-07-05 | 2014-03-07 | 현대자동차주식회사 | Apparatus and method for detecting object using image around vehicle |
US10462442B2 (en) * | 2012-12-20 | 2019-10-29 | Brett I. Walker | Apparatus, systems and methods for monitoring vehicular activity |
JP6330383B2 (en) * | 2014-03-12 | 2018-05-30 | 株式会社デンソー | Composite image generation apparatus and composite image generation program |
US10486599B2 (en) * | 2015-07-17 | 2019-11-26 | Magna Mirrors Of America, Inc. | Rearview vision system for vehicle |
EP3832034B1 (en) * | 2018-07-31 | 2023-08-23 | Sumitomo (S.H.I.) Construction Machinery Co., Ltd. | Excavator |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020047901A1 (en) * | 2000-04-28 | 2002-04-25 | Kunio Nobori | Image processor and monitoring system |
US6593960B1 (en) * | 1999-08-18 | 2003-07-15 | Matsushita Electric Industrial Co., Ltd. | Multi-functional on-vehicle camera system and image display method for the same |
US7139412B2 (en) * | 2001-04-24 | 2006-11-21 | Matsushita Electric Industrial Co., Ltd. | Image synthesis display method and apparatus for vehicle camera |
US7266219B2 (en) * | 2000-07-19 | 2007-09-04 | Matsushita Electric Industrial Co., Ltd. | Monitoring system |
US7307655B1 (en) * | 1998-07-31 | 2007-12-11 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus for displaying a synthesized image viewed from a virtual point of view |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3372944B2 (en) | 2000-07-19 | 2003-02-04 | 松下電器産業株式会社 | Monitoring system |
JP2004235986A (en) | 2003-01-30 | 2004-08-19 | Matsushita Electric Ind Co Ltd | Monitoring system |
JP4830380B2 (en) | 2005-07-13 | 2011-12-07 | 日産自動車株式会社 | Vehicle periphery monitoring device and vehicle periphery monitoring method |
-
2007
- 2007-08-21 US US11/842,685 patent/US7728879B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7307655B1 (en) * | 1998-07-31 | 2007-12-11 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus for displaying a synthesized image viewed from a virtual point of view |
US6593960B1 (en) * | 1999-08-18 | 2003-07-15 | Matsushita Electric Industrial Co., Ltd. | Multi-functional on-vehicle camera system and image display method for the same |
US20020047901A1 (en) * | 2000-04-28 | 2002-04-25 | Kunio Nobori | Image processor and monitoring system |
US7266219B2 (en) * | 2000-07-19 | 2007-09-04 | Matsushita Electric Industrial Co., Ltd. | Monitoring system |
US7139412B2 (en) * | 2001-04-24 | 2006-11-21 | Matsushita Electric Industrial Co., Ltd. | Image synthesis display method and apparatus for vehicle camera |
Cited By (57)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102008030104A1 (en) | 2008-06-25 | 2009-02-12 | Daimler Ag | Vehicle i.e. tractor-trailer, surrounding monitoring method, involves assigning positions of image capturing units and weighing factor to each pixel, and determining overall picture between vehicle elements by taking into inclination angle |
WO2010066332A1 (en) * | 2008-12-12 | 2010-06-17 | Daimler Ag | Device for monitoring an environment of a vehicle |
DE102008061760A1 (en) | 2008-12-12 | 2010-06-17 | Daimler Ag | Device for monitoring an environment of a vehicle |
US20110228088A1 (en) * | 2008-12-12 | 2011-09-22 | Daimler Ag | Device for monitoring an environment of a vehicle |
CN102245433A (en) * | 2008-12-12 | 2011-11-16 | 戴姆勒股份公司 | Device for monitoring an environment of a vehicle |
US20110166782A1 (en) * | 2008-12-18 | 2011-07-07 | Aisin Seiki Kabushiki Kaisha | Display device |
CN102149573A (en) * | 2008-12-18 | 2011-08-10 | 爱信精机株式会社 | Display apparatus |
US8207868B2 (en) | 2008-12-18 | 2012-06-26 | Aisin Seiki Kabushiki Kaisha | Display device |
US20120062745A1 (en) * | 2009-05-19 | 2012-03-15 | Imagenext Co., Ltd. | Lane departure sensing method and apparatus using images that surround a vehicle |
US9056630B2 (en) * | 2009-05-19 | 2015-06-16 | Imagenext Co., Ltd. | Lane departure sensing method and apparatus using images that surround a vehicle |
US20120026333A1 (en) * | 2009-07-29 | 2012-02-02 | Clarion Co., Ltd | Vehicle periphery monitoring device and vehicle periphery image display method |
US9247217B2 (en) * | 2009-07-29 | 2016-01-26 | Clarion Co., Ltd. | Vehicle periphery monitoring device and vehicle periphery image display method |
US8446471B2 (en) * | 2009-12-31 | 2013-05-21 | Industrial Technology Research Institute | Method and system for generating surrounding seamless bird-view image with distance interface |
US20110157361A1 (en) * | 2009-12-31 | 2011-06-30 | Industrial Technology Research Institute | Method and system for generating surrounding seamless bird-view image with distance interface |
US20110310245A1 (en) * | 2010-06-21 | 2011-12-22 | Nissan Motor Co., Ltd. | Travel distance detection device and travel distance detection method |
US8854456B2 (en) * | 2010-06-21 | 2014-10-07 | Nissan Motor Co., Ltd. | Travel distance detection device and travel distance detection method |
US20130169792A1 (en) * | 2010-08-12 | 2013-07-04 | Valeo Schalter Und Sensoren Gmbh | Method for assisting in a parking operation for a motor vehicle, driver assistance system and a motor vehicle |
US10558221B2 (en) * | 2010-08-12 | 2020-02-11 | Valeo Schalter Und Sensoren Gmbh | Method for assisting in a parking operation for a motor vehicle, driver assistance system and a motor vehicle |
US10397544B2 (en) | 2010-08-19 | 2019-08-27 | Nissan Motor Co., Ltd. | Three-dimensional object detection device and three-dimensional object detection method |
CN101976460A (en) * | 2010-10-18 | 2011-02-16 | 胡振程 | Generating method of virtual view image of surveying system of vehicular multi-lens camera |
US20140347450A1 (en) * | 2011-11-30 | 2014-11-27 | Imagenext Co., Ltd. | Method and apparatus for creating 3d image of vehicle surroundings |
US9508189B2 (en) * | 2011-11-30 | 2016-11-29 | Kss-Imagenext Co., Ltd. | Method and apparatus for creating 3D image of vehicle surroundings |
US9333915B2 (en) * | 2012-09-20 | 2016-05-10 | Komatsu Ltd. | Work vehicle periphery monitoring system and work vehicle |
US20150183370A1 (en) * | 2012-09-20 | 2015-07-02 | Komatsu Ltd. | Work vehicle periphery monitoring system and work vehicle |
US9796330B2 (en) * | 2012-09-21 | 2017-10-24 | Komatsu Ltd. | Working vehicle periphery monitoring system and working vehicle |
US20150217690A1 (en) * | 2012-09-21 | 2015-08-06 | Komatsu Ltd. | Working vehicle periphery monitoring system and working vehicle |
US20150189170A1 (en) * | 2012-10-05 | 2015-07-02 | Fuji Xerox Co., Ltd. | Information processing apparatus, information processing system and non-transitory computer readable medium |
US9942471B2 (en) * | 2012-10-05 | 2018-04-10 | Fuji Xerox Co., Ltd. | Information processing apparatus, information processing system and non-transitory computer readable medium |
US20140119597A1 (en) * | 2012-10-31 | 2014-05-01 | Hyundai Motor Company | Apparatus and method for tracking the position of a peripheral vehicle |
EP2916540A4 (en) * | 2012-10-31 | 2016-10-19 | Clarion Co Ltd | Image processing system and image processing method |
US9485438B2 (en) | 2012-10-31 | 2016-11-01 | Clarion Co., Ltd. | Image processing system with image conversion unit that composites overhead view images and image processing method |
US9025819B2 (en) * | 2012-10-31 | 2015-05-05 | Hyundai Motor Company | Apparatus and method for tracking the position of a peripheral vehicle |
US9767561B2 (en) * | 2013-11-18 | 2017-09-19 | Texas Instruments Incorporated | Method and apparatus for a optimal seam for surround view synthesis |
CN104683706A (en) * | 2013-11-28 | 2015-06-03 | 财团法人金属工业研究发展中心 | Image joint method |
US20150341597A1 (en) * | 2014-05-22 | 2015-11-26 | Dr. Ing. H.C. F. Porsche Aktiengesellschaft | Method for presenting a vehicle's environment on a display apparatus; a display apparatus; a system comprising a plurality of image capturing units and a display apparatus; a computer program |
US9522633B2 (en) * | 2014-09-26 | 2016-12-20 | Hyundai Motor Company | Driver customizable blind spot display method and apparatus |
US20160090043A1 (en) * | 2014-09-26 | 2016-03-31 | Hyundai Motor Company | Driver customizable blind spot display method and apparatus |
EP3029601A1 (en) * | 2014-12-04 | 2016-06-08 | Conti Temic microelectronic GmbH | Method and apparatus for providing increased obstacle visibility |
US10311310B2 (en) * | 2014-12-04 | 2019-06-04 | Conti Temic Microelectronic Gmbh | Method and apparatus for providing increased obstacle visibility |
WO2016087173A1 (en) * | 2014-12-04 | 2016-06-09 | Conti Temic Microelectronic Gmbh | Method and apparatus for providing increased obstacle visibility |
US20170270371A1 (en) * | 2014-12-04 | 2017-09-21 | Continental Automotive Gmbh | Method and Apparatus for Providing Increased Obstacle Visibility |
US10157452B1 (en) * | 2015-09-28 | 2018-12-18 | Amazon Technologies, Inc. | Image processing system for image rectification |
US10909667B1 (en) | 2015-09-28 | 2021-02-02 | Amazon Technologies, Inc. | Image rectification using transformation data |
US10417743B2 (en) | 2015-11-06 | 2019-09-17 | Mitsubishi Electric Corporation | Image processing device, image processing method and computer readable medium |
US10339656B1 (en) * | 2016-09-29 | 2019-07-02 | Amazon Technologies, Inc. | Inferring count of items using image |
US10713803B1 (en) * | 2016-09-29 | 2020-07-14 | Amazon Technologies, Inc. | Testing validity of items using image data analysis |
US11373320B1 (en) * | 2016-09-29 | 2022-06-28 | Amazon Technologies, Inc. | Detecting inventory changes by comparing image data |
WO2018108215A1 (en) * | 2016-12-15 | 2018-06-21 | Conti Temic Microelectronic Gmbh | Device for providing improved obstacle detection |
US10824884B2 (en) | 2016-12-15 | 2020-11-03 | Conti Temic Microelectronic Gmbh | Device for providing improved obstacle identification |
US10579067B2 (en) * | 2017-07-20 | 2020-03-03 | Huawei Technologies Co., Ltd. | Method and system for vehicle localization |
US20190025854A1 (en) * | 2017-07-20 | 2019-01-24 | Mohsen Rohani | Method and system for vehicle localization |
CN110228468A (en) * | 2018-03-06 | 2019-09-13 | 爱信精机株式会社 | Ambient enviroment monitoring device |
US20190275970A1 (en) * | 2018-03-06 | 2019-09-12 | Aisin Seiki Kabushiki Kaisha | Surroundings monitoring apparatus |
US11403742B2 (en) * | 2018-03-28 | 2022-08-02 | Mitsubishi Electric Corporation | Image processing device, image processing method, and recording medium for generating bird's eye synthetic image |
WO2020061794A1 (en) * | 2018-09-26 | 2020-04-02 | 深圳市大疆创新科技有限公司 | Vehicle driver assistance device, vehicle and information processing method |
US20200120371A1 (en) * | 2018-10-10 | 2020-04-16 | Rovi Guides, Inc. | Systems and methods for providing ar/vr content based on vehicle conditions |
US20220353316A1 (en) * | 2021-04-30 | 2022-11-03 | Woven Planet Holdings, Inc. | Communication device, communication method, and non-transitory computer-readable recording medium |
Also Published As
Publication number | Publication date |
---|---|
US7728879B2 (en) | 2010-06-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7728879B2 (en) | Image processor and visual field support device | |
JP4248570B2 (en) | Image processing apparatus and visibility support apparatus and method | |
KR101295295B1 (en) | Image processing method and image processing apparatus | |
US8130270B2 (en) | Vehicle-mounted image capturing apparatus | |
KR101491170B1 (en) | Vehicle surrounding view display system | |
JP4861574B2 (en) | Driving assistance device | |
JP4934308B2 (en) | Driving support system | |
JP4596978B2 (en) | Driving support system | |
JP5729158B2 (en) | Parking assistance device and parking assistance method | |
JP4315968B2 (en) | Image processing apparatus and visibility support apparatus and method | |
JP5953824B2 (en) | Vehicle rear view support apparatus and vehicle rear view support method | |
US20090268027A1 (en) | Driving Assistance System And Vehicle | |
US20150042799A1 (en) | Object highlighting and sensing in vehicle image display systems | |
EP2254334A1 (en) | Image processing device and method, driving support system, and vehicle | |
US20090015675A1 (en) | Driving Support System And Vehicle | |
US8477191B2 (en) | On-vehicle image pickup apparatus | |
KR20160072190A (en) | Bowl-shaped imaging system | |
JP2009206747A (en) | Ambient condition monitoring system for vehicle, and video display method | |
JP2005311868A (en) | Vehicle periphery visually recognizing apparatus | |
JP2010016805A (en) | Image processing apparatus, driving support system, and image processing method | |
JP7263731B2 (en) | Image processing device | |
JP5083443B2 (en) | Driving support device and method, and arithmetic device | |
JP7000383B2 (en) | Image processing device and image processing method | |
US20180056870A1 (en) | Viewing device for vehicle and method of displaying viewed image for vehicle | |
JP7467402B2 (en) | IMAGE PROCESSING SYSTEM, MOBILE DEVICE, IMAGE PROCESSING METHOD, AND COMPUTER PROGRAM |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SANYO ELECTRIC CO., LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ISHII, YOHEI;REEL/FRAME:019726/0427 Effective date: 20070807 Owner name: SANYO ELECTRIC CO., LTD.,JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ISHII, YOHEI;REEL/FRAME:019726/0427 Effective date: 20070807 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552) Year of fee payment: 8 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 12 |