WO2006027563A1 - View enhancing system for a vehicle - Google Patents

View enhancing system for a vehicle Download PDF

Info

Publication number
WO2006027563A1
WO2006027563A1 PCT/GB2005/003423 GB2005003423W WO2006027563A1 WO 2006027563 A1 WO2006027563 A1 WO 2006027563A1 GB 2005003423 W GB2005003423 W GB 2005003423W WO 2006027563 A1 WO2006027563 A1 WO 2006027563A1
Authority
WO
WIPO (PCT)
Prior art keywords
view
vehicle
display
structural element
image
Prior art date
Application number
PCT/GB2005/003423
Other languages
French (fr)
Inventor
Robert Clive Mullins
Original Assignee
Mch Technology Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from GB0419747A external-priority patent/GB2417847A/en
Application filed by Mch Technology Limited filed Critical Mch Technology Limited
Publication of WO2006027563A1 publication Critical patent/WO2006027563A1/en

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/26Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the rear of the vehicle
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/20Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
    • B60R2300/202Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used displaying a blind spot scene on the vehicle part responsible for the blind spot
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/301Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/802Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8066Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring rearward traffic

Definitions

  • the present invention relates to a view enhancing system for a vehicle.
  • Vehicles in particular those with enclosed passenger / driver cabins (for example cars), typically have structural features such as pillars or uprights which obstruct the vehicle driver's view from the vehicle.
  • structural features such as pillars or uprights which obstruct the vehicle driver's view from the vehicle.
  • rear view mirrors are provided on most vehicles to provide an improved view towards the rear of the vehicle, these only provide a limited field of vision which is still affected by the presence of obstructive structural elements as well as by passengers and objects inside the vehicle.
  • the present invention seeks to alleviate some of these problems.
  • a view enhancing system for a vehicle having a structural element which in normal operation tends to obstruct a vehicle operator's view comprising: a camera adapted to form an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element; and display means connected with the camera and mounted on the structural element for displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present.
  • the vehicle operator's view can be enhanced, which can lead to greater visual and situational awareness and improved safety.
  • the structural element tends to obstruct the generally forward view of the vehicle operator.
  • the vehicle operator usually obtains a view of the vehicle surroundings in one of two ways. Firstly, during normal forward driving, by looking in a generally forward direction, and secondly, when manoeuvring, by looking in rear view mirrors or by physically turning around.
  • generally forward view is preferably meant the operator's direct view during normal forward driving.
  • peripheral vision or slight movement of the head this may encompass not only the view directly ahead of the vehicle but also a view to the sides and ahead of the vehicle, where the view may, for example, be obstructed by structural elements such as "A" pillars.
  • the generally forward view preferably lies within the forward 180 degree portion of the vehicle surroundings, and may lie within the forward 120 degree or forward 90 degree portion of the vehicle surroundings.
  • the vehicle operator has a field of view centred around a horizontal plane substantially at eye level, and the structural element obstructs a portion of the view in that plane.
  • the display means preferably covers substantially the entire operator-facing portion of the structural element in that plane, that is to say substantially the entire portion of the structural element in that plane which is visible to the operator. In this way, a substantially seamless or continuous view may be provided.
  • the structural element is preferably an element required for the structural integrity of the vehicle, and may be an outer or outward-facing element which preferably directly obstructs the operator's view.
  • the structural element preferably forms part of a support structure, such as the vehicle chassis or frame, and/or of an outer or covering structure, such as a shell or body.
  • the structural element need not be a single integral body, but could be of complex construction, for example, comprising a frame element covered by a shell, body or trim element.
  • the structural element is a pillar, in which case the display means preferably covers substantially the entire width of the pillar as seen by the operator.
  • the vehicle preferably has a plurality of pillars (e.g.
  • the system preferably comprises respective cameras and display means provided on at least two such pillars, preferably on two pillars provided either side of the vehicle's front windscreen. This can enhance the driver's main forward view.
  • the system may of course be applied to cabriolet-type cars or other vehicles which do not have a roof but which nevertheless have pillars or obstructive structures.
  • the display means preferably comprises a flexible display screen (such as a flexible OLEP (Organic Light Emitting Polymer) screen), and is preferably shaped so as to approximately follow the contours of the structural element.
  • the system preferably comprises means for processing the image in accordance with the shape of the display, which preferably adapts the image to the display shape or contours, for example by stretching portions of the image in dependence on the curvature of the display. This can further enhance the illusion that the structural element is, in effect, transparent, and can provide a more accurate view of the vehicle surroundings.
  • a rear view display system for a vehicle, comprising: at least one camera mounted on the outside of the vehicle adapted to capture an image of a rear view from the vehicle; and a rear view mirror including a reflective layer and a display layer, and wherein: the reflective layer is adapted to reflect a rear view when the display layer is inoperative, and the display layer, when operative, is adapted to display a rear view image based on the captured rear view image, the displayed image showing at least the view of the vehicle surroundings that would be seen by a vehicle operator in the reflective layer if the display layer were inoperative.
  • the displayed image provides more information than the reflected view, in particular by showing a more extensive view of the surroundings, for example a view which is not obstructed by internal features of the vehicle as is the view in a conventional mirror.
  • the view shown by the display layer may have substantially the same viewing angle as shown by the reflective layer or may have a greater viewing angle.
  • the display layer preferably comprises a substantially transparent electronic display which covers (at least some and preferably most or substantially all of) the reflective layer of the rear view mirror.
  • the invention provides a method of enhancing a vehicle operator's view from a vehicle having a structural element which in normal operation tends to obstruct the operator's view, comprising: forming an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element using a camera; and displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present on display means connected with the camera and mounted on the structural element.
  • the method may also include steps for performing the functions of the view enhancing system described herein.
  • the invention also provides a computer program or computer program product comprising software code adapted, when executed on a data processing apparatus, to perform a method as described herein and/or to perform functions of a view enhancing system as described herein.
  • a method of enhancing a vehicle operator's view in a vehicle having a structural element which in normal operation tends to obstruct the operator's view comprising: mounting on the vehicle a camera adapted to form an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element; and mounting display means connected with the camera on the structural element for displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present.
  • the display means preferably comprises a flexible display screen, and the method preferably comprises shaping the display screen to conform to the shape of the structural element.
  • Figure 1 is a top-down schematic view of a car
  • Figure 2 is a schematic cross-sectional view of a pillar
  • Figure 3 is a block diagram of a view enhancing system
  • Figures 4A and 4B show stylized views from a vehicle.
  • the view enhancing system described herein comprises two main parts: a series of displays provided on those structural elements (such as pillars) which ordinarily obstruct the driver's view, and a rear view display system for providing an enhanced rear view display in place of the conventional rear view mirror.
  • the system can allow the vehicle driver to have total, unobstructed vision all around the vehicle - essentially as if the uprights and pillars and other obstructions were transparent.
  • the first part of the system uses cameras and/or sensors facing outwards from the vehicle, the images from which are displayed on a number of displays visible to the driver.
  • the displays are embedded in or attached to the obstructive structures (pillars, uprights, etc.) in the vehicle so that the driver can use the displays to get a clear view of the road and surroundings. In this way, driving safety can be improved by enhancing visual perception, as otherwise obstructed areas now become part of the total viewing area for the driver.
  • the second part of the system uses a mirror display in place of the conventional rear view mirror.
  • the mirror display displays a rear view from the vehicle obtained by one or more cameras mounted at the rear of the vehicle. Since the cameras are mounted on the outside of the vehicle, the image displayed is free from obstructions, further enhancing the driver's visual perception and situational awareness.
  • Figure 1 is a top-down schematic view of a car 10.
  • Cars are commonly designed with three main sets of pillars for supporting the roof: A pair of "A"- pillars either side of the windscreen, a pair of "B"-pillars between the side windows and a pair of "C-pillars at the rear of the vehicle.
  • the driver's view out of the front of the car is typically obstructed mainly by the "A"-pillars.
  • the driver usually makes use of a combination of rear view mirrors: a wing mirror 12 on either side of the vehicle and a central rear view mirror 14 mounted inside the car under the roof.
  • the central rear view mirror 14 provides a view of the traffic behind the vehicle, though the view reflected usually also includes the "Capillars and any other obstructions (such as passengers, and luggage or other objects placed in front of the rear window). Both the driver's front view as well as the view presented by the rear view mirror are therefore incomplete.
  • the view enhancing system comprises a series of cameras 20, 22, 24 which are mounted, preferably recessed, on each view-obstructing pillar facing outwards (cameras are provided on the pillars on both sides of the vehicle, though for clarity, only those on one side are labelled with reference numerals).
  • Each camera is oriented so as to capture the portion of the surroundings which is obstructed by the pillar to which it is mounted, when seen from the perspective of the driver.
  • Six cameras are provided in the present example (one in each pillar), though a different number may be used. Alternatively, cameras may be located on the roof or in the wing mirrors.
  • the cameras are CCD or CMOS sensor devices, though any suitable image gathering devices or image sensors can be used.
  • the cameras typically provide resolutions of around 640x480 or higher, though lower resolution devices can also be used. Cameras can be fixed or auto-focus and have an automatically adjusted iris for night / daytime operation.
  • a display is provided on each view-obstructing pillar.
  • Each display receives the image from the camera mounted on the outside of the respective pillar and displays it, thereby enabling the driver to see the portion of the surroundings obstructed by the pillar as if the pillar were transparent.
  • Figure 2 shows, by way of example, a schematic cross-sectional view of an "A" pillar.
  • the "A" pillar is located between front windscreen 26 and side window 28. Recessed into the pillar is a camera 20 for forming an image of the portion of the view obstructed by the pillar.
  • a display 52 is provided on the inside surface of the pillar. The display 52 faces the vehicle driver and receives the image of the obstructed portion of the view from camera 20.
  • the display may be a flat display of suitable dimensions fixed to or recessed into the pillar, such as an LCD / TFT display.
  • the display is provided in the form of a shaped and/or flexible display which is attached to the pillar, and which is shaped so as to at least approximately follow the contours of the pillar.
  • the screen 52 is of a convex shape, though, depending on the shape of the pillar, concave or more complex screen shapes may be used.
  • the display covers substantially the entire width of the pillar as seen by the driver, so that the driver is provided with a continuous view in which the pillar display completes the view between adjacent windows as seamlessly as possible.
  • the display preferably also covers substantially the entire height of the pillar, though a smaller display could be used to save cost, for example covering only a central part of the pillar in the vertical direction.
  • Curved microglass may be used to provide a rigid, shaped screen which is formed to fit the shape of the pillar.
  • flexible display technology may be used to provide the shaped screen for the pillar displays.
  • Various flexible screen technologies can be used, including LEP (light-emitting polymer) / OLEP (organic light-emitting polymer) / LED (light emitting diode) / OLED (organic light emitting diode) / PoIyOLED (polymer OLED) based displays, typically on flexible plastic substrates, as well as flexible LCD screens.
  • Preferred examples use small molecule or polymer OLEP based screens due to their comparatively low power consumption.
  • the displays are preferably designed to blend into the vehicle's interior design, for example by having a similar appearance to the surface material of the pillars, when inactive.
  • a flexible protective coating or covering may be provided on the display to absorb impact (for example during an accident) and/or (where the display uses glass or other rigid materials) to contain any breakage under the protective layer in such an event.
  • FIG. 3 shows a block diagram of the view enhancement system.
  • Cameras 20, 22, 24 are connected to a video processing module 40.
  • the signals from the cameras are encoded into a suitable format by video encoding hardware (not shown) and/or software executed by the video processing module.
  • the video processing module 40 is further connected to the displays 52, 54, 56 mounted inside the vehicle on view-obstructing pillars. Either cabled (electrical or fibre optic) or wireless / RF (Radio Frequency) connections may be provided between the video processing module and the cameras and displays.
  • other sensors 62 for example for sensing the driver's viewing position as described below may also be connected to the video processing module 40.
  • the video processing module 40 may further be connected to memory 48 and control processor 50.
  • the control processor 50 may in turn be connected to a vehicle management system or vehicle network 58.
  • the video processing module may be directly connected to a vehicle network.
  • the cameras and/or displays may also be connected to the video processing module via a vehicle network rather than directly.
  • Persistent storage such as Flash ROM, may be provided for storing program code for the control processor and/or the video processing module.
  • a "black box" recorder 60 may optionally be provided for storing image data from the cameras and possibly other information for analysis in the event of an accident.
  • the video processing module 40 processes the image received from each camera and outputs the processed image to the associated display in real time.
  • the image is processed so as to produce an approximation of what the driver would see if the pillar were not present.
  • the processing performed includes modification of the image in dependence on the shape of the display.
  • the display is not flat but instead (approximately) follows the contours of the pillar to which it is attached, simply displaying the image received from the camera as if to a flat display may result in a warped image (from the point of view of the driver).
  • the video processing module therefore maps the image output from a camera to the given display shape. This involves stretching and/or contracting portions of the image in dependence on the screen contours.
  • the video processing module 40 maps the input image onto the display shape by expanding portions of the image to compensate for curvature of the display. Since the display (when flat) typically has a fixed pixel size, after mounting on the pillar, parts of the display may be curved and/or angled away from the viewer. Pixels in those parts of the display would appear smaller to the viewer. If displayed without modification, this could lead to an apparent contraction of the image at those display locations. To counteract this effect, the image processing module expands the image at those locations. Thus, a single pixel in the input image may be expanded into two pixels in the displayed image (in the direction of curvature of the screen), for example by duplicating the pixel value.
  • the mapping can be predefined, for example by way of a table mapping input pixels to output pixels, thus reducing the computational overhead of applying the mapping.
  • the display will typically have a higher resolution than the input image, though the video processing module may alternatively resize the image after stretching (such resizing could also form part of the mapping itself).
  • the processing performed by the video processing module may be configurable to allow for manual adjustments to the image output in dependence on the exact viewing position of the driver, for example in dependence on the driver's height and seat adjustment. In some examples, this may alternatively be achieved automatically by way of sensors in the seat, or by way of a sensor (e.g. an ultrasonic, infrared or image sensor) detecting the position of the driver's head or eyes. In one example, this is achieved using a camera mounted inside the vehicle in combination with image processing / feature recognition software to detect the position of the driver's head.
  • the camera on the outside of the pillar captures a larger image than is displayed on the pillar display, which displays only a segment of the captured image.
  • the video processing module selects the segment to be displayed based on the driver's viewing position (determined using image processing and feature recognition software as mentioned above), and adjusts this in real-time in response to driver movement.
  • the video processing module shifts the displayed .segment to the right by a corresponding amount. In this way, the view provided can remain seamless.
  • Vertical movement may similarly be translated into a vertical shift of the display segment selected from the whole of the captured image. This real-time adjustment may be provided instead of or in addition to manual adjustment as described above.
  • the user may also be able to adjust image settings such as focus, brightness and contrast.
  • the video processing module may comprise a single image processor
  • the video processing module may also provide processing for other aspects of the system, such as the rear view display system described in more detail below.
  • the video processing module may further comprise one or more image encoders for encoding the image output into a format suitable for display.
  • the video processing module may also comprise a power management module for managing power supply to the cameras and displays.
  • pillar displays are provided on each of the "A", "B" and “C” pillars, this need not necessarily be the case. Since the driver typically uses rear view mirrors to obtain a rear view, the pillar displays are typically most beneficial when used to enhance the generally forward view of the driver, i.e. the view during normal forward driving. Thus, in an alternative embodiment, displays could be provided on only the two "A" pillars to provide a substantially seamless view in the generally forward direction.
  • Figure 4A shows (in a stylised form) a view from a vehicle without the pillar displays described above.
  • Figure 4B shows the same scene, but with a pillar display 52 installed on the "A" pillar.
  • the pillar display completes the missing view by displaying the view normally obstructed by the "A" pillar.
  • a (reasonably) seamless, continuous view is provided between the windscreen 70 and the side window 72.
  • the second part of the view enhancing system is a rear view display system comprising a mirror display provided in place of the conventional central rear view mirror.
  • the mirror display comprises a display layer and a reflective layer.
  • the display layer When switched off or in "standby" mode, the display layer is substantially transparent, allowing light to pass through to the reflective layer behind, and the mirror display functions as a conventional mirror.
  • the mirror display When active and receiving an image, the mirror display functions as a display, displaying the received image.
  • the display is constructed using a clear cathode type LCD or LEP / OLED display.
  • One or more cameras are provided on the outside of the vehicle at the rear and obtain an image of the rear section of the vehicle's surroundings - substantially the same view as shown by a conventional rear view mirror. The image thus obtained is displayed on the mirror display. Because the cameras are mounted on the outside of the vehicle, instead of the reflection as seen in a conventional rear view mirror, including obstructions such as pillars, luggage and passengers, the driver sees an unobstructed view of the road / traffic behind the vehicle as received by the cameras.
  • the image may be provided by a single camera or by multiple cameras.
  • a single camera may, for example, be mounted in a central position on the roof or another appropriate location and collect a complete rear image and send it to the mirror display.
  • two cameras are provided on the outside of the "Capillars.
  • the images produced by the cameras are spliced together by the video processing module 40 taking account of any overlap in the cameras' fields of view to produce a single image for display on the mirror display.
  • the display component of the mirror display can be turned off (for example if some other component of the system has failed), in which case the mirror display functions as a conventional mirror. Also, if the display itself fails, the mirror display will act as a conventional mirror, thus providing a fail-safe system.
  • the two parts of the view enhancing system - that is, the pillar displays and the enhanced rear view display - may be provided independently or in combination.
  • the driver can be provided with a complete unobstructed view of the vehicle surroundings. Since the driver mostly uses the rear view mirrors for obtaining a view to the rear of the vehicle, it will typically be sufficient to provide pillar displays only on the front or "A"-pillars, or alternatively on the "A" and “B” pillars, but not on the "C” pillars. An unobstructed rear view is then still provided by the enhanced rear view display.
  • both the pillar displays and the rear view mirror display may be provided by way of projection systems, using projectors mounted inside the vehicle along with non-active display areas on the pillars and the rear view mirror.
  • the display areas on the pillars are provided in the form of reflective projection surfaces shaped to fit the pillars.
  • the projectors may use micro displays with a bright light shone on to the surface and the reflected image sent through a collection lens to the screen/display on the pillars and mirror (in full colour). To ensure correct projection, the image would in this case be inverted by the video processing module before projection.
  • small-scale projection systems including projection devices such as Texas Instruments (TM) digital light processors may be used.
  • the projection systems are installed in the roof of the car and receive the images recorded by the cameras mounted on the outside of the vehicle.
  • the projection devices project the road image onto the projection surfaces on the pillars and the rear view mirror.
  • a projection surface is provided on the pillars comprising a plastic surface with a reflective material or coating.
  • the coating is preferably arranged to reflect only light originating from the direction of the projector, and to be otherwise substantially non-reflective or transparent to prevent spurious reflections (for example of ambient light) which could be distracting to the driver.
  • the coating may, for example, use a material such as ITO (indium tin oxide), silver or gold.
  • the coating is chosen so as to provide a clear projected image in all environmental conditions including daylight, bright sunlight, or night time conditions.
  • the rear view obtained from the rear-facing camera(s) is similarly projected directly onto the rear view mirror, which may, for example, include a reflective coating as described above.
  • the mirror can be used conventionally if the projection system fails or is deactivated.
  • a sensor for example an infrared sensor
  • a sensor can be provided to detect the orientation of (or adjustment to) the mirror so as to enable adjustment of the projection system in response to adjustment of the mirror by the driver.
  • the displays used by the system may also be used to display additional information, for example information about the vehicle surroundings or the vehicle status.
  • Information about the vehicle surroundings may, for example, be obtained from cameras or other sensors mounted on the outside of the vehicle.
  • the information displayed may include warnings about approaching vehicles, for example in the form of warning icons.
  • Information about vehicle status may be obtained from an engine / vehicle management system and displayed, including speed, fuel levels and the like. Warning icons may also be displayed in response to specific conditions, such as a low fuel indicator.
  • the pillar displays are particularly suitable for the display of such additional information since the information can be made available to the driver without distracting the driver's attention away from the road, as would be the case if such information were displayed, for example, on the dashboard, and without detracting from the conventional view through the windscreen as would be the case if information displays were integrated into the windscreen.
  • additional cameras capture rear views corresponding to those normally obtained by way of the wing mirrors. These rear views are then displayed (e.g. in picture-in-picture style) at lower parts of the pillar displays on the left and right-hand "A" pillars respectively (though the system could be applied only on one side of the vehicle). This can bring the rear view (normally seen in the wing mirror) within the driver's peripheral vision, thus increasing the driver's awareness of the vehicle surroundings. In some cases, such displays may replace one or more wing mirrors entirely, thus removing the obstruction and resulting blind spot normally caused by the wing mirror.
  • Processing may be used to improve the image displayed, for example to reduce headlight glare from other vehicles (such processing could also be applied to images displayed on the pillar displays and rear view display already described). It will be understood that the present invention has been described above purely by way of example, and modification of detail can be made within the scope of the invention.
  • the rear view may also be displayed on a portion of the front windscreen, either by way of a transparent display integrated into the windscreen, or by way of a projection system projecting the image onto the windscreen.
  • the rear view can also be displayed or projected on an area of the dashboard to provide the driver with an unobstructed rear view whilst not being distracted from the road ahead.
  • the number of cameras and the number and shape of display screens used and the positioning of the cameras and display screens can be freely varied in dependence on requirements.
  • the system is described above using the example of a car, but different camera / screen arrangements may be suitable for other types of vehicles.
  • the video processing module may also provide image enhancement functions for this purpose.
  • two cameras located, for example, on the "C" pillars
  • the mirror_display has display portions covering a left-hand and a right-hand portion of the mirror, the central portion being a conventional mirror.
  • the left and right display portions then display the image from a respective one of the rear-facing cameras.
  • the displayed images and central conventional mirror portion together provide a complete view.
  • the two images may alternatively be collected using a triangular prism together with a single centrally mounted camera, the prism being arranged to collect two images of the obstructed portions of the view at the left and right- hand corners of the vehicle and deflect the images into the single camera to produce an image having separate left and right portions corresponding to the left and right portions of the view which are obstructed by the "C" pillars.
  • the image can then be separated into separate left- and right-hand images by the video-processing module.
  • aspects of the system may be applied to a wide range of different types of vehicles, including, for example, cars or vans, as well as aircraft cockpits, driver cabins on trains, and the like.
  • the rear view display system could be applied to a motorcycle, to provide, for example, a rear view display on the instrument panel or windscreen.
  • displays could be provided on other view-obstructing elements, such as door panels.
  • a display could be provided on an upper part of a door below the side window to provide an enhanced view to the side of the vehicle, which could be helpful during parking.
  • Similar systems could also be provided to provide a view through a boot or hood and the associated boot / hood space.
  • a display in the form of a wide strip could be provided across the dashboard showing an image angled down from the front of the vehicle, thus providing a view "through" the bonnet / engine space to enable the driver to see low objects such as a curb or bollards.
  • the video processing module may be omitted, so that the image from the cameras is displayed directly on the pillar displays. This may, for example, be appropriate where the display is relatively (or completely) flat. Also, in the example described above, the video-processing module may be bypassed in certain circumstances (for example if there is a syster ⁇ Lfailure).._T.he video processing module itself may be adapted to simply pass the image received from a camera directly to the relevant screen under such circumstances.
  • the video processing module may store images for later review, for example in the event of an accident. This may be combined with the recording of other data (such as vehicle speed, GPS data from a GPS receiver, and/or sound) to provide a "black box" recorder (e.g. recorder 60 of Figure 3).
  • a "black box" recorder e.g. recorder 60 of Figure 3
  • the system would typically maintain a store of images covering a specified time period, for example, the last 30 seconds. Where a camera is installed inside the vehicle, images of the occupants could also be stored.

Abstract

A view enhancing system for a vehicle is disclosed. The vehicle has a structural element which in normal operation tends to obstruct a vehicle operator's view. The system includes a camera (20, 22, 24) adapted to form an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element, and display means (52) connected with the camera and mounted on the structural element for displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present. In a disclosed example, the structural element is a pillar. A rear view display system for a vehicle having a camera mounted on the outside of the vehicle adapted to capture an image of a rear view from the vehicle and a rear view mirror including a reflective layer and a display layer is also disclosed.

Description

View enhancing system for a vehicle
The present invention relates to a view enhancing system for a vehicle.
Vehicles, in particular those with enclosed passenger / driver cabins (for example cars), typically have structural features such as pillars or uprights which obstruct the vehicle driver's view from the vehicle. Although modern vehicle design attempts to minimise the visual obstruction caused by such structural features, these attempts are limited by structural and safety considerations. The driver's field of view is therefore generally incomplete, to the detriment of driver and passenger safety.
Though rear view mirrors are provided on most vehicles to provide an improved view towards the rear of the vehicle, these only provide a limited field of vision which is still affected by the presence of obstructive structural elements as well as by passengers and objects inside the vehicle. The present invention seeks to alleviate some of these problems.
Accordingly, in a first aspect of the invention, there is provided a view enhancing system for a vehicle having a structural element which in normal operation tends to obstruct a vehicle operator's view, comprising: a camera adapted to form an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element; and display means connected with the camera and mounted on the structural element for displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present.
In this way, the vehicle operator's view can be enhanced, which can lead to greater visual and situational awareness and improved safety.
Preferably, in normal operation the structural element tends to obstruct the generally forward view of the vehicle operator. The vehicle operator usually obtains a view of the vehicle surroundings in one of two ways. Firstly, during normal forward driving, by looking in a generally forward direction, and secondly, when manoeuvring, by looking in rear view mirrors or by physically turning around. By generally forward view is preferably meant the operator's direct view during normal forward driving. By way of peripheral vision or slight movement of the head this may encompass not only the view directly ahead of the vehicle but also a view to the sides and ahead of the vehicle, where the view may, for example, be obstructed by structural elements such as "A" pillars. Thus, the generally forward view preferably lies within the forward 180 degree portion of the vehicle surroundings, and may lie within the forward 120 degree or forward 90 degree portion of the vehicle surroundings. Preferably, the vehicle operator has a field of view centred around a horizontal plane substantially at eye level, and the structural element obstructs a portion of the view in that plane. The display means preferably covers substantially the entire operator-facing portion of the structural element in that plane, that is to say substantially the entire portion of the structural element in that plane which is visible to the operator. In this way, a substantially seamless or continuous view may be provided.
The structural element is preferably an element required for the structural integrity of the vehicle, and may be an outer or outward-facing element which preferably directly obstructs the operator's view. The structural element preferably forms part of a support structure, such as the vehicle chassis or frame, and/or of an outer or covering structure, such as a shell or body. Thus, the structural element need not be a single integral body, but could be of complex construction, for example, comprising a frame element covered by a shell, body or trim element. Preferably, the structural element is a pillar, in which case the display means preferably covers substantially the entire width of the pillar as seen by the operator. The vehicle preferably has a plurality of pillars (e.g. for supporting a roof); and the system preferably comprises respective cameras and display means provided on at least two such pillars, preferably on two pillars provided either side of the vehicle's front windscreen. This can enhance the driver's main forward view. The system may of course be applied to cabriolet-type cars or other vehicles which do not have a roof but which nevertheless have pillars or obstructive structures.
The display means preferably comprises a flexible display screen (such as a flexible OLEP (Organic Light Emitting Polymer) screen), and is preferably shaped so as to approximately follow the contours of the structural element. The system preferably comprises means for processing the image in accordance with the shape of the display, which preferably adapts the image to the display shape or contours, for example by stretching portions of the image in dependence on the curvature of the display. This can further enhance the illusion that the structural element is, in effect, transparent, and can provide a more accurate view of the vehicle surroundings.
In a further aspect of the invention, there is provided a rear view display system for a vehicle, comprising: at least one camera mounted on the outside of the vehicle adapted to capture an image of a rear view from the vehicle; and a rear view mirror including a reflective layer and a display layer, and wherein: the reflective layer is adapted to reflect a rear view when the display layer is inoperative, and the display layer, when operative, is adapted to display a rear view image based on the captured rear view image, the displayed image showing at least the view of the vehicle surroundings that would be seen by a vehicle operator in the reflective layer if the display layer were inoperative. Preferably, the displayed image provides more information than the reflected view, in particular by showing a more extensive view of the surroundings, for example a view which is not obstructed by internal features of the vehicle as is the view in a conventional mirror. The view shown by the display layer may have substantially the same viewing angle as shown by the reflective layer or may have a greater viewing angle.
In this way, an enhanced rear view display can be provided which is not obstructed by the obstructions commonly visible in a conventional rear view mirror, such as structural elements or other objects in the vehicle interior. At the same time, when the display layer is inoperative (for example due to a technical failure), the mirror can function as a conventional mirror, thus providing a fail-safe system. The display layer preferably comprises a substantially transparent electronic display which covers (at least some and preferably most or substantially all of) the reflective layer of the rear view mirror.
In a further aspect, the invention provides a method of enhancing a vehicle operator's view from a vehicle having a structural element which in normal operation tends to obstruct the operator's view, comprising: forming an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element using a camera; and displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present on display means connected with the camera and mounted on the structural element. The method may also include steps for performing the functions of the view enhancing system described herein. The invention also provides a computer program or computer program product comprising software code adapted, when executed on a data processing apparatus, to perform a method as described herein and/or to perform functions of a view enhancing system as described herein.
In a further aspect of the invention, there is provided a method of enhancing a vehicle operator's view in a vehicle having a structural element which in normal operation tends to obstruct the operator's view, comprising: mounting on the vehicle a camera adapted to form an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element; and mounting display means connected with the camera on the structural element for displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present. The display means preferably comprises a flexible display screen, and the method preferably comprises shaping the display screen to conform to the shape of the structural element.
Preferred features of the present invention will now be described, purely by way of example, with reference to the accompanying drawings, in which:-
Figure 1 is a top-down schematic view of a car;
Figure 2 is a schematic cross-sectional view of a pillar;
Figure 3 is a block diagram of a view enhancing system; and
Figures 4A and 4B show stylized views from a vehicle. The view enhancing system described herein comprises two main parts: a series of displays provided on those structural elements (such as pillars) which ordinarily obstruct the driver's view, and a rear view display system for providing an enhanced rear view display in place of the conventional rear view mirror. The system can allow the vehicle driver to have total, unobstructed vision all around the vehicle - essentially as if the uprights and pillars and other obstructions were transparent.
The first part of the system uses cameras and/or sensors facing outwards from the vehicle, the images from which are displayed on a number of displays visible to the driver. The displays are embedded in or attached to the obstructive structures (pillars, uprights, etc.) in the vehicle so that the driver can use the displays to get a clear view of the road and surroundings. In this way, driving safety can be improved by enhancing visual perception, as otherwise obstructed areas now become part of the total viewing area for the driver.
The second part of the system uses a mirror display in place of the conventional rear view mirror. The mirror display displays a rear view from the vehicle obtained by one or more cameras mounted at the rear of the vehicle. Since the cameras are mounted on the outside of the vehicle, the image displayed is free from obstructions, further enhancing the driver's visual perception and situational awareness.
Figure 1 is a top-down schematic view of a car 10. Cars are commonly designed with three main sets of pillars for supporting the roof: A pair of "A"- pillars either side of the windscreen, a pair of "B"-pillars between the side windows and a pair of "C-pillars at the rear of the vehicle.
The driver's view out of the front of the car is typically obstructed mainly by the "A"-pillars. To see out of the back of the car, rather than turning around, the driver usually makes use of a combination of rear view mirrors: a wing mirror 12 on either side of the vehicle and a central rear view mirror 14 mounted inside the car under the roof. The central rear view mirror 14 provides a view of the traffic behind the vehicle, though the view reflected usually also includes the "Capillars and any other obstructions (such as passengers, and luggage or other objects placed in front of the rear window). Both the driver's front view as well as the view presented by the rear view mirror are therefore incomplete.
The view enhancing system comprises a series of cameras 20, 22, 24 which are mounted, preferably recessed, on each view-obstructing pillar facing outwards (cameras are provided on the pillars on both sides of the vehicle, though for clarity, only those on one side are labelled with reference numerals). Each camera is oriented so as to capture the portion of the surroundings which is obstructed by the pillar to which it is mounted, when seen from the perspective of the driver. Six cameras are provided in the present example (one in each pillar), though a different number may be used. Alternatively, cameras may be located on the roof or in the wing mirrors. The cameras are CCD or CMOS sensor devices, though any suitable image gathering devices or image sensors can be used. The cameras typically provide resolutions of around 640x480 or higher, though lower resolution devices can also be used. Cameras can be fixed or auto-focus and have an automatically adjusted iris for night / daytime operation.
Inside the vehicle, a display is provided on each view-obstructing pillar.
Each display receives the image from the camera mounted on the outside of the respective pillar and displays it, thereby enabling the driver to see the portion of the surroundings obstructed by the pillar as if the pillar were transparent.
Figure 2 shows, by way of example, a schematic cross-sectional view of an "A" pillar.
The "A" pillar is located between front windscreen 26 and side window 28. Recessed into the pillar is a camera 20 for forming an image of the portion of the view obstructed by the pillar. A display 52 is provided on the inside surface of the pillar. The display 52 faces the vehicle driver and receives the image of the obstructed portion of the view from camera 20.
The display may be a flat display of suitable dimensions fixed to or recessed into the pillar, such as an LCD / TFT display.
However, in preferred embodiments, the display is provided in the form of a shaped and/or flexible display which is attached to the pillar, and which is shaped so as to at least approximately follow the contours of the pillar. In the example of Figure 2, the screen 52 is of a convex shape, though, depending on the shape of the pillar, concave or more complex screen shapes may be used. The display covers substantially the entire width of the pillar as seen by the driver, so that the driver is provided with a continuous view in which the pillar display completes the view between adjacent windows as seamlessly as possible. The display preferably also covers substantially the entire height of the pillar, though a smaller display could be used to save cost, for example covering only a central part of the pillar in the vertical direction.
Curved microglass may be used to provide a rigid, shaped screen which is formed to fit the shape of the pillar. Alternatively, flexible display technology may be used to provide the shaped screen for the pillar displays. Various flexible screen technologies can be used, including LEP (light-emitting polymer) / OLEP (organic light-emitting polymer) / LED (light emitting diode) / OLED (organic light emitting diode) / PoIyOLED (polymer OLED) based displays, typically on flexible plastic substrates, as well as flexible LCD screens. Preferred examples use small molecule or polymer OLEP based screens due to their comparatively low power consumption.
The displays are preferably designed to blend into the vehicle's interior design, for example by having a similar appearance to the surface material of the pillars, when inactive. A flexible protective coating or covering may be provided on the display to absorb impact (for example during an accident) and/or (where the display uses glass or other rigid materials) to contain any breakage under the protective layer in such an event.
Figure 3 shows a block diagram of the view enhancement system. Cameras 20, 22, 24 are connected to a video processing module 40. The signals from the cameras are encoded into a suitable format by video encoding hardware (not shown) and/or software executed by the video processing module. The video processing module 40 is further connected to the displays 52, 54, 56 mounted inside the vehicle on view-obstructing pillars. Either cabled (electrical or fibre optic) or wireless / RF (Radio Frequency) connections may be provided between the video processing module and the cameras and displays. In addition to the cameras, other sensors 62 (for example for sensing the driver's viewing position as described below) may also be connected to the video processing module 40.
The video processing module 40 may further be connected to memory 48 and control processor 50. The control processor 50 may in turn be connected to a vehicle management system or vehicle network 58. Alternatively the video processing module may be directly connected to a vehicle network. In alternative arrangements, the cameras and/or displays may also be connected to the video processing module via a vehicle network rather than directly. Persistent storage, such as Flash ROM, may be provided for storing program code for the control processor and/or the video processing module. A "black box" recorder 60 may optionally be provided for storing image data from the cameras and possibly other information for analysis in the event of an accident.
The video processing module 40 processes the image received from each camera and outputs the processed image to the associated display in real time. The image is processed so as to produce an approximation of what the driver would see if the pillar were not present. The processing performed includes modification of the image in dependence on the shape of the display.
Specifically, since in some examples the display is not flat but instead (approximately) follows the contours of the pillar to which it is attached, simply displaying the image received from the camera as if to a flat display may result in a warped image (from the point of view of the driver). In such cases, the video processing module therefore maps the image output from a camera to the given display shape. This involves stretching and/or contracting portions of the image in dependence on the screen contours.
In a particular example, the video processing module 40 maps the input image onto the display shape by expanding portions of the image to compensate for curvature of the display. Since the display (when flat) typically has a fixed pixel size, after mounting on the pillar, parts of the display may be curved and/or angled away from the viewer. Pixels in those parts of the display would appear smaller to the viewer. If displayed without modification, this could lead to an apparent contraction of the image at those display locations. To counteract this effect, the image processing module expands the image at those locations. Thus, a single pixel in the input image may be expanded into two pixels in the displayed image (in the direction of curvature of the screen), for example by duplicating the pixel value.
Since the shape of the display is typically known (based on the design of the pillar in question), the mapping can be predefined, for example by way of a table mapping input pixels to output pixels, thus reducing the computational overhead of applying the mapping. To accommodate for the stretching of the image, the display will typically have a higher resolution than the input image, though the video processing module may alternatively resize the image after stretching (such resizing could also form part of the mapping itself). The processing performed by the video processing module may be configurable to allow for manual adjustments to the image output in dependence on the exact viewing position of the driver, for example in dependence on the driver's height and seat adjustment. In some examples, this may alternatively be achieved automatically by way of sensors in the seat, or by way of a sensor (e.g. an ultrasonic, infrared or image sensor) detecting the position of the driver's head or eyes. In one example, this is achieved using a camera mounted inside the vehicle in combination with image processing / feature recognition software to detect the position of the driver's head.
In one embodiment, the camera on the outside of the pillar captures a larger image than is displayed on the pillar display, which displays only a segment of the captured image. The video processing module selects the segment to be displayed based on the driver's viewing position (determined using image processing and feature recognition software as mentioned above), and adjusts this in real-time in response to driver movement. Thus, if the driver leans to the left in his seat, the video processing module shifts the displayed .segment to the right by a corresponding amount. In this way, the view provided can remain seamless. Vertical movement may similarly be translated into a vertical shift of the display segment selected from the whole of the captured image. This real-time adjustment may be provided instead of or in addition to manual adjustment as described above.
The user may also be able to adjust image settings such as focus, brightness and contrast. The video processing module may comprise a single image processor
(for example in the form of a digital signal processor or general purpose processor) for processing the images for each of the camera / display pairs, or may comprise multiple image processors (for example, one per camera / display pair). In one example, multiple DSPs are provided operating in a master / slave configuration, each DSP processing the images for multiple camera / display pairs. The video processing module may also provide processing for other aspects of the system, such as the rear view display system described in more detail below. The video processing module may further comprise one or more image encoders for encoding the image output into a format suitable for display.
The video processing module may also comprise a power management module for managing power supply to the cameras and displays. Although in the above example, pillar displays are provided on each of the "A", "B" and "C" pillars, this need not necessarily be the case. Since the driver typically uses rear view mirrors to obtain a rear view, the pillar displays are typically most beneficial when used to enhance the generally forward view of the driver, i.e. the view during normal forward driving. Thus, in an alternative embodiment, displays could be provided on only the two "A" pillars to provide a substantially seamless view in the generally forward direction.
The effect of the pillar display is illustrated by the stylised views of Figures 4A and 4B. Figure 4A shows (in a stylised form) a view from a vehicle without the pillar displays described above. As can be seen, the "A" pillar obstructs a portion of the driver's view. Figure 4B shows the same scene, but with a pillar display 52 installed on the "A" pillar. The pillar display completes the missing view by displaying the view normally obstructed by the "A" pillar. Thus, a (reasonably) seamless, continuous view is provided between the windscreen 70 and the side window 72. As mentioned above, the second part of the view enhancing system is a rear view display system comprising a mirror display provided in place of the conventional central rear view mirror.
The mirror display comprises a display layer and a reflective layer. When switched off or in "standby" mode, the display layer is substantially transparent, allowing light to pass through to the reflective layer behind, and the mirror display functions as a conventional mirror. When active and receiving an image, the mirror display functions as a display, displaying the received image.
The display is constructed using a clear cathode type LCD or LEP / OLED display.
One or more cameras are provided on the outside of the vehicle at the rear and obtain an image of the rear section of the vehicle's surroundings - substantially the same view as shown by a conventional rear view mirror. The image thus obtained is displayed on the mirror display. Because the cameras are mounted on the outside of the vehicle, instead of the reflection as seen in a conventional rear view mirror, including obstructions such as pillars, luggage and passengers, the driver sees an unobstructed view of the road / traffic behind the vehicle as received by the cameras.
The image may be provided by a single camera or by multiple cameras. A single camera may, for example, be mounted in a central position on the roof or another appropriate location and collect a complete rear image and send it to the mirror display. In another example, two cameras are provided on the outside of the "Capillars. The images produced by the cameras are spliced together by the video processing module 40 taking account of any overlap in the cameras' fields of view to produce a single image for display on the mirror display.
The display component of the mirror display can be turned off (for example if some other component of the system has failed), in which case the mirror display functions as a conventional mirror. Also, if the display itself fails, the mirror display will act as a conventional mirror, thus providing a fail-safe system.
The two parts of the view enhancing system - that is, the pillar displays and the enhanced rear view display - may be provided independently or in combination. By providing a combined system incorporating both the pillar displays and the enhanced rear view display, the driver can be provided with a complete unobstructed view of the vehicle surroundings. Since the driver mostly uses the rear view mirrors for obtaining a view to the rear of the vehicle, it will typically be sufficient to provide pillar displays only on the front or "A"-pillars, or alternatively on the "A" and "B" pillars, but not on the "C" pillars. An unobstructed rear view is then still provided by the enhanced rear view display.
Instead of the electronic displays as described above, both the pillar displays and the rear view mirror display may be provided by way of projection systems, using projectors mounted inside the vehicle along with non-active display areas on the pillars and the rear view mirror. In this case, the display areas on the pillars are provided in the form of reflective projection surfaces shaped to fit the pillars. The projectors may use micro displays with a bright light shone on to the surface and the reflected image sent through a collection lens to the screen/display on the pillars and mirror (in full colour). To ensure correct projection, the image would in this case be inverted by the video processing module before projection.
Alternatively, small-scale projection systems including projection devices such as Texas Instruments (TM) digital light processors may be used. The projection systems are installed in the roof of the car and receive the images recorded by the cameras mounted on the outside of the vehicle. The projection devices project the road image onto the projection surfaces on the pillars and the rear view mirror.
In the case of the pillar displays, a projection surface is provided on the pillars comprising a plastic surface with a reflective material or coating. The coating is preferably arranged to reflect only light originating from the direction of the projector, and to be otherwise substantially non-reflective or transparent to prevent spurious reflections (for example of ambient light) which could be distracting to the driver. The coating may, for example, use a material such as ITO (indium tin oxide), silver or gold. Preferably, the coating is chosen so as to provide a clear projected image in all environmental conditions including daylight, bright sunlight, or night time conditions.
The rear view obtained from the rear-facing camera(s) is similarly projected directly onto the rear view mirror, which may, for example, include a reflective coating as described above. The mirror can be used conventionally if the projection system fails or is deactivated. A sensor (for example an infrared sensor) can be provided to detect the orientation of (or adjustment to) the mirror so as to enable adjustment of the projection system in response to adjustment of the mirror by the driver.
The displays used by the system (both the pillar displays and the rear view display) may also be used to display additional information, for example information about the vehicle surroundings or the vehicle status. Information about the vehicle surroundings may, for example, be obtained from cameras or other sensors mounted on the outside of the vehicle. The information displayed may include warnings about approaching vehicles, for example in the form of warning icons. Information about vehicle status may be obtained from an engine / vehicle management system and displayed, including speed, fuel levels and the like. Warning icons may also be displayed in response to specific conditions, such as a low fuel indicator. The pillar displays (especially in the front or "A"-pillars) are particularly suitable for the display of such additional information since the information can be made available to the driver without distracting the driver's attention away from the road, as would be the case if such information were displayed, for example, on the dashboard, and without detracting from the conventional view through the windscreen as would be the case if information displays were integrated into the windscreen.
In one embodiment, additional cameras capture rear views corresponding to those normally obtained by way of the wing mirrors. These rear views are then displayed (e.g. in picture-in-picture style) at lower parts of the pillar displays on the left and right-hand "A" pillars respectively (though the system could be applied only on one side of the vehicle). This can bring the rear view (normally seen in the wing mirror) within the driver's peripheral vision, thus increasing the driver's awareness of the vehicle surroundings. In some cases, such displays may replace one or more wing mirrors entirely, thus removing the obstruction and resulting blind spot normally caused by the wing mirror. Processing may be used to improve the image displayed, for example to reduce headlight glare from other vehicles (such processing could also be applied to images displayed on the pillar displays and rear view display already described). It will be understood that the present invention has been described above purely by way of example, and modification of detail can be made within the scope of the invention.
For example, although a system using a mirror display provided in the central rear view mirror has been described, similar displays may also be used in the wing mirrors to provide an improved view.
Instead of a display integrated into the rear view mirror, the rear view may also be displayed on a portion of the front windscreen, either by way of a transparent display integrated into the windscreen, or by way of a projection system projecting the image onto the windscreen. Alternatively, the rear view can also be displayed or projected on an area of the dashboard to provide the driver with an unobstructed rear view whilst not being distracted from the road ahead.
The number of cameras and the number and shape of display screens used and the positioning of the cameras and display screens can be freely varied in dependence on requirements. The system is described above using the example of a car, but different camera / screen arrangements may be suitable for other types of vehicles.
Other types of sensors, for example infrared sensors / cameras, or low- light / starlight cameras, may be provided (instead of or in addition to normal cameras, or in the form of combined daylight and low-light / thermal cameras) to enhance the driver's view, for example at night or in bad weather conditions. The video processing module may also provide image enhancement functions for this purpose. In an alternative embodiment of the rear view display system, rather than displaying a complete rear view, two cameras (located, for example, on the "C" pillars) may capture the portion of the view obstructed by the respective pillar. In. this example, the mirror_display has display portions covering a left-hand and a right-hand portion of the mirror, the central portion being a conventional mirror. The left and right display portions then display the image from a respective one of the rear-facing cameras. The displayed images and central conventional mirror portion together provide a complete view. The two images may alternatively be collected using a triangular prism together with a single centrally mounted camera, the prism being arranged to collect two images of the obstructed portions of the view at the left and right- hand corners of the vehicle and deflect the images into the single camera to produce an image having separate left and right portions corresponding to the left and right portions of the view which are obstructed by the "C" pillars. The image can then be separated into separate left- and right-hand images by the video-processing module.
Aspects of the system may be applied to a wide range of different types of vehicles, including, for example, cars or vans, as well as aircraft cockpits, driver cabins on trains, and the like. In one example, the rear view display system could be applied to a motorcycle, to provide, for example, a rear view display on the instrument panel or windscreen.
Instead of pillars, displays (with corresponding cameras) could be provided on other view-obstructing elements, such as door panels. For example, a display could be provided on an upper part of a door below the side window to provide an enhanced view to the side of the vehicle, which could be helpful during parking. Similar systems could also be provided to provide a view through a boot or hood and the associated boot / hood space. For example, to assist in forward parking, a display in the form of a wide strip could be provided across the dashboard showing an image angled down from the front of the vehicle, thus providing a view "through" the bonnet / engine space to enable the driver to see low objects such as a curb or bollards.
In some examples, the video processing module may be omitted, so that the image from the cameras is displayed directly on the pillar displays. This may, for example, be appropriate where the display is relatively (or completely) flat. Also, in the example described above, the video-processing module may be bypassed in certain circumstances (for example if there is a systerτLfailure).._T.he video processing module itself may be adapted to simply pass the image received from a camera directly to the relevant screen under such circumstances.
The video processing module may store images for later review, for example in the event of an accident. This may be combined with the recording of other data (such as vehicle speed, GPS data from a GPS receiver, and/or sound) to provide a "black box" recorder (e.g. recorder 60 of Figure 3). The system would typically maintain a store of images covering a specified time period, for example, the last 30 seconds. Where a camera is installed inside the vehicle, images of the occupants could also be stored.

Claims

1. A view enhancing system for a vehicle having a structural element which in normal operation tends to obstruct a vehicle operator's view, comprising: a camera adapted to form an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element; and display means connected with the camera and mounted on the structural element for displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present.
2. A system according to Claim 1 , wherein the structural element in normal operation tends to obstruct the generally forward view of the vehicle operator.
3. A system according to Claim 1 or 2, wherein the vehicle operator has a field of. view centred aroum±a horizontal plane substantially at eye. level, and wherein the structural element obstructs a portion of the view in that plane.
4. A system according to Claim 3, wherein the display means covers substantially the entire operator-facing portion of the structural element in that plane.
5. A system according to any of the preceding claims, wherein the structural element forms part of a support structure of the vehicle, for example a chassis or frame.
6. A system according to any of the preceding claims, wherein the structural element is a pillar.
7. A system according to any of the preceding claims, the vehicle having a plurality of pillars for supporting a roof; the system comprising respective cameras and display means provided on at least two such pillars.
8. A system according to any of the preceding claims, the vehicle having a front windscreen bounded by two pillars; the system comprising a respective camera and respective display means provided on each of the two pillars.
9. A system according to any of the preceding claims, wherein the display means comprises an electronic display.
10. A system according to any of the preceding claims, wherein the display means comprises a flexible display screen.
11. A system according to any of the preceding claims, wherein the display means comprises an OLEP (Organic Light Emitting Polymer) screen.
12. A system according to any of Claims 1 to 8, wherein the display means comprises a projection surface, the system further comprising a projector for projecting the image onto the projection surface.
13. A system according to any of the preceding claims, wherein the display means is shaped according to the shape of the structural element.
14. A system according to Claim 13, wherein the display means is shaped so as to approximately follow the contours of the structural element.
15. A system according to Claim 13 or 14, further comprising means for processing the image in accordance with the shape of the display.
16. A system according to Claim 15, wherein the processing means adapts the image to the display shape or contours.
17. A system according to Claim 15 or 16, wherein the processing means is adapted to stretch or contract portions of the image.
18. A system according to any of the preceding claims, wherein the camera is mounted on the structural element.
19. A rear view display system for a vehicle, comprising: at least one camera mounted on the outside of the vehicle adapted to capture an image of a rear view from the vehicle; and a rear view mirror including a reflective layer and a display layer, and wherein: the reflective layer is adapted to reflect a rear view when the display layer is inoperative, and the display layer, when operative, is adapted to display a rear view image based on the captured rear view image, the displayed image showing at least the view of the vehicle surroundings that would be seen by a vehicle operator in the reflective layer if the display layer were inoperative.
20. A rear view display system according to Claim 19, wherein the display layer comprises a substantially transparent electronic display which covers at least some, and preferably: most or substantially all of. the reflective layer of the rear view mirror.
21. A system according to Claim 19 or 20, comprising a plurality of such cameras each adapted to receive an image of part of the rear view, and processing means adapted to generate a single image of the rear view from the images received by the cameras.
22. A method of enhancing a vehicle operator's view from a vehicle having a structural element which in normal operation tends to obstruct the operator's view, comprising: forming an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element using a camera; and displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present on display means connected with the camera and mounted on the structural element.
23. A method of enhancing a vehicle operator's view in a vehicle having a structural element which in normal operation tends to obstruct the operator's view, comprising: mounting on the vehicle a camera adapted to form an image of a region of the vehicle surroundings where the operator's view is obstructed by the structural element; and mounting display means connected with the camera on the structural element for displaying an image representing the view that would be seen by the operator if the structural element were transparent or not present.
24. A method according to Claim 23, wherein the display means comprises a flexible display screen, the method comprising shaping the display screen to conform to the shape of the structural element.
PCT/GB2005/003423 2004-09-06 2005-09-06 View enhancing system for a vehicle WO2006027563A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
GB0419747A GB2417847A (en) 2004-09-06 2004-09-06 Vehicle blind spot viewing system
GB0419747.1 2004-09-06
US61766304P 2004-10-13 2004-10-13
US60/617,663 2004-10-13

Publications (1)

Publication Number Publication Date
WO2006027563A1 true WO2006027563A1 (en) 2006-03-16

Family

ID=35149337

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2005/003423 WO2006027563A1 (en) 2004-09-06 2005-09-06 View enhancing system for a vehicle

Country Status (1)

Country Link
WO (1) WO2006027563A1 (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2045132A2 (en) 2007-10-02 2009-04-08 Aisin AW Co., Ltd. Driving support device, driving support method, and computer program
EP2177878A2 (en) * 2008-10-20 2010-04-21 Navteq North America, LLC Traffic display depicting view of traffic from within a vehicle
EP2251229A1 (en) * 2009-05-11 2010-11-17 Rheinmetall Landsysteme GmbH Camera monitor system for a vehicle or building door
WO2013022577A1 (en) * 2011-08-09 2013-02-14 Continental Automotive Systems, Inc. Field of view matching video display system
WO2013093450A3 (en) * 2011-12-22 2013-09-19 Vincent Demarest Driver display system
DE102014220524A1 (en) * 2014-10-09 2015-10-22 Conti Temic Microelectronic Gmbh Device for assisting a driver of a vehicle and vehicle
WO2016094801A1 (en) * 2014-12-12 2016-06-16 Hoyda Serge B System and process for viewing in blind spots
WO2017024458A1 (en) 2015-08-10 2017-02-16 Bayerische Motoren Werke Aktiengesellschaft System, method and apparatus for vehicle and computer readable medium
US10125918B2 (en) 2014-12-12 2018-11-13 Serge B. HOYDA Mounting system for a camera
US10138672B2 (en) 2014-12-12 2018-11-27 Serge B. HOYDA Mounting system for a camera
CN108944682A (en) * 2018-05-30 2018-12-07 东莞市路帆电子科技有限公司 A kind of vehicle blind zone imaging system and application method
US11124116B2 (en) 2014-12-12 2021-09-21 Serge B. HOYDA System and process for viewing in blind spots
WO2022128559A1 (en) * 2020-12-15 2022-06-23 Ams International Ag A monitoring arrangement, display enabled device and method of operating a monitoring arrangement
US11518309B2 (en) 2014-12-12 2022-12-06 Serge Hoyda LLC System and process for viewing in blind spots
DE102016104871B4 (en) 2015-03-24 2023-08-17 Toyota Jidosha Kabushiki Kaisha Structure for accommodating a peripheral information detection sensor, and self-propelled vehicle

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5956181A (en) * 1997-07-18 1999-09-21 Lin; William Two way mirror with dual functions of rear view mirror and video displayer
EP1285814A1 (en) * 2001-08-21 2003-02-26 Ford Global Technologies, Inc., A subsidiary of Ford Motor Company Monitoring device for vehicles
US20030151563A1 (en) * 2002-02-08 2003-08-14 Kulas Charles J. Reduction of blind spots by using display screens
EP1378395A1 (en) * 2002-07-02 2004-01-07 Kia Motors Co., Ltd. A monitoring system for monitoring a dead zone related to a front pillar of a vehicle
US20050168695A1 (en) * 2004-01-16 2005-08-04 Kabushiki Kaisha Honda Lock Vehicular visual assistance system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5956181A (en) * 1997-07-18 1999-09-21 Lin; William Two way mirror with dual functions of rear view mirror and video displayer
EP1285814A1 (en) * 2001-08-21 2003-02-26 Ford Global Technologies, Inc., A subsidiary of Ford Motor Company Monitoring device for vehicles
US20030151563A1 (en) * 2002-02-08 2003-08-14 Kulas Charles J. Reduction of blind spots by using display screens
EP1378395A1 (en) * 2002-07-02 2004-01-07 Kia Motors Co., Ltd. A monitoring system for monitoring a dead zone related to a front pillar of a vehicle
US20050168695A1 (en) * 2004-01-16 2005-08-04 Kabushiki Kaisha Honda Lock Vehicular visual assistance system

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2045132A2 (en) 2007-10-02 2009-04-08 Aisin AW Co., Ltd. Driving support device, driving support method, and computer program
EP2045132A3 (en) * 2007-10-02 2010-06-02 Aisin AW Co., Ltd. Driving support device, driving support method, and computer program
US8089512B2 (en) 2007-10-02 2012-01-03 Aisin Aw Co., Ltd. Driving support device, driving support method and computer program
CN101404122B (en) * 2007-10-02 2013-01-02 爱信艾达株式会社 Driving support device, driving support method, and computer program
EP2177878A2 (en) * 2008-10-20 2010-04-21 Navteq North America, LLC Traffic display depicting view of traffic from within a vehicle
EP2251229A1 (en) * 2009-05-11 2010-11-17 Rheinmetall Landsysteme GmbH Camera monitor system for a vehicle or building door
WO2013022577A1 (en) * 2011-08-09 2013-02-14 Continental Automotive Systems, Inc. Field of view matching video display system
US9102269B2 (en) 2011-08-09 2015-08-11 Continental Automotive Systems, Inc. Field of view matching video display system
WO2013093450A3 (en) * 2011-12-22 2013-09-19 Vincent Demarest Driver display system
DE102014220524A1 (en) * 2014-10-09 2015-10-22 Conti Temic Microelectronic Gmbh Device for assisting a driver of a vehicle and vehicle
US9776568B2 (en) 2014-12-12 2017-10-03 Serge B. HOYDA System and process for viewing in blind spots
US11124116B2 (en) 2014-12-12 2021-09-21 Serge B. HOYDA System and process for viewing in blind spots
WO2016094801A1 (en) * 2014-12-12 2016-06-16 Hoyda Serge B System and process for viewing in blind spots
US11518309B2 (en) 2014-12-12 2022-12-06 Serge Hoyda LLC System and process for viewing in blind spots
US10046703B2 (en) 2014-12-12 2018-08-14 Serge B. HOYDA System and process for viewing in blind spots
US10125918B2 (en) 2014-12-12 2018-11-13 Serge B. HOYDA Mounting system for a camera
US10138672B2 (en) 2014-12-12 2018-11-27 Serge B. HOYDA Mounting system for a camera
US10967791B2 (en) 2014-12-12 2021-04-06 Serge B. HOYDA System and process for viewing in blind spots
DE102016104871B4 (en) 2015-03-24 2023-08-17 Toyota Jidosha Kabushiki Kaisha Structure for accommodating a peripheral information detection sensor, and self-propelled vehicle
EP3334621A4 (en) * 2015-08-10 2019-03-13 Bayerische Motoren Werke Aktiengesellschaft System, method and apparatus for vehicle and computer readable medium
WO2017024458A1 (en) 2015-08-10 2017-02-16 Bayerische Motoren Werke Aktiengesellschaft System, method and apparatus for vehicle and computer readable medium
CN107848460A (en) * 2015-08-10 2018-03-27 宝马股份公司 For the system of vehicle, method and apparatus and computer-readable medium
CN108944682A (en) * 2018-05-30 2018-12-07 东莞市路帆电子科技有限公司 A kind of vehicle blind zone imaging system and application method
WO2022128559A1 (en) * 2020-12-15 2022-06-23 Ams International Ag A monitoring arrangement, display enabled device and method of operating a monitoring arrangement

Similar Documents

Publication Publication Date Title
WO2006027563A1 (en) View enhancing system for a vehicle
US10315573B2 (en) Method for displaying information to vehicle driver
US10029621B2 (en) Rear view camera system using rear view mirror location
CN108602465B (en) Image display system for vehicle and vehicle equipped with the same
US10525886B2 (en) Display system, electronic mirror system and movable-body apparatus equipped with the same
US8643724B2 (en) Multi-camera vision system for a vehicle
KR101778173B1 (en) A rearview assembly of a vehicle for displaying images
US20060244829A1 (en) Vehicular image display apparatus
JP6333488B1 (en) Vehicle sunshade
JP2003267129A (en) Vehicular mirror having built-in monitor device
KR20160091293A (en) Side Mirror Camera System For Vehicle
WO2014130049A1 (en) Systems and methods for augmented rear-view displays
JP6232691B2 (en) VEHICLE DISPLAY CONTROL DEVICE, VEHICLE DISPLAY DEVICE, AND VEHICLE DISPLAY CONTROL METHOD
US20090122139A1 (en) Display System for a Motor Vehicle
CN112693397A (en) Vehicle exterior imaging system
JP6302610B1 (en) Awning device
GB2417847A (en) Vehicle blind spot viewing system
JP2000272415A (en) Drive support image composition device
KR20180001963U (en) Camera and display module to remove the invisible zone around car fender
JP6564898B2 (en) Vehicle sunshade
US20190378449A1 (en) Color modification system and methods for vehicle displays
KR102339522B1 (en) Integrated vehicle and driving information display method and apparatus
CN204845719U (en) Three -dimensional driving image auxiliary device
JP2006137209A (en) Anti-glare device for vehicle
CN113448115A (en) Display device and control circuit

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase