WO2017078771A1 - Method and device for panoramic imaging - Google Patents

Method and device for panoramic imaging Download PDF

Info

Publication number
WO2017078771A1
WO2017078771A1 PCT/US2016/020144 US2016020144W WO2017078771A1 WO 2017078771 A1 WO2017078771 A1 WO 2017078771A1 US 2016020144 W US2016020144 W US 2016020144W WO 2017078771 A1 WO2017078771 A1 WO 2017078771A1
Authority
WO
WIPO (PCT)
Prior art keywords
optical
image
panoramic
image sensor
lens
Prior art date
Application number
PCT/US2016/020144
Other languages
French (fr)
Inventor
Zhaolong SHEN
Wei XI
Ke Huang
Original Assignee
Shen Zhaolong
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shen Zhaolong filed Critical Shen Zhaolong
Priority to CN201680001783.2A priority Critical patent/CN109478011A/en
Publication of WO2017078771A1 publication Critical patent/WO2017078771A1/en

Links

Classifications

    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B37/00Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
    • G03B37/04Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe with cameras or projectors providing touching or overlapping fields of view
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B2217/00Details of cameras or camera bodies; Accessories therefor
    • G03B2217/002Details of arrangement of components in or on camera body
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B39/00High-speed photography
    • G03B39/06High-speed photography using light-guides for transferring image frame or elements thereof into different array, e.g. into a line

Definitions

  • the subject disclosure relates to an imaging method and device for forming panoramic images or videos with single or multiple image sensors.
  • Panoramic imaging methods and devices have been devised in the past.
  • the existing conventional methods can be categorized into three types.
  • the first type of methods are based on digital image stitching. Multiple images are taken by arranging multiple cameras pointing to different directions. A panoramic image can be generated by stitching these images together. This type of method increases system complexity, size, power consumption and cost, therefore it is not suitable for applications that are sensitive to aforementioned factors.
  • the second type of methods are based on ultra wide-angle lens, which creates a wide panoramic or hemispherical image but with strong visual distortion.
  • the third type of methods are similar to the second. Instead of using the wide-angle lens, a mirror is utilized to direct the field of view onto the image sensor. Besides the same disadvantage as the wide-angle methods, this type of methods suffer from a blind area in the center of the image sensor.
  • the panoramic camera device comprises of a set of two or more optical lens units, respectively, wherein each lens includes one or more optical elements to collect a portion of the panoramic scene and form a subimage on an image surface; an optical image conduit module comprising a plurality of optical fiber waveguides that are optically coupled to each optical lens unit to receive the light from the image surface, respectively, wherein each optical fiber waveguide is bent and/or twisted by suitable angles to transfer the light away from its input facet onto the output facet, which points towards the main image sensor; an optical conduction mechanism for transmitting the light emitted from the output facet of the optical image conduit onto the main image sensor; a main image sensor for converting the received light signal into the digital image; a supporting fixture for fastening the optical image conduit and the optical conduction mechanism; and an image processing module for constructing the panoramic image from the digital image signal.
  • optical image conduit module comprising a plurality of optical fiber waveguides transfers multiple desired field of views to the single image sensor.
  • the image sensor consequently converts the light into digital signal for further processing.
  • the digital image signal is subsequently divided into multiple segments.
  • the image distortions caused by lenses, both in geometry and light intensity, are corrected by software algorithm.
  • the software correction parameters can be obtained from manufacturing calibration process. All the segments are stitched in a suitable order to reconstruct the panoramic image after distortion correction. Comparing to conventional stitching approaches, the disclosed method has both low computation demand and less distortion advantages. This image or image sequence is then compressed for storage or transmission.
  • FIG. 1 shows a flow diagram according to an embodiment of the present invention wherein a camera device receives segmented panoramic image via a plurality of lens and transfer through an optical image conduit module comprising a plurality of optical waveguides to a main image sensor, and processes the image in order to recreate seamless panoramic image ;
  • FIG. 2 illustrates a coordination system for describing a lens orientation and the field of view of a lens according to an embodiment of the panoramic camera device
  • FIG. 3 illustrates an optical waveguide is bent along its center line by ⁇ degree according to an embodiment of the panoramic camera device
  • FIG. 4 illustrates an optical waveguide is twisted by degree according to an embodiment of the panoramic camera device
  • FIG. 5 illustrates an optical waveguide is firstly twisted by 3 ⁇ 4 degree and then bent by f3 ⁇ 4 degree according to an embodiment of the panoramic camera device
  • FIG. 6 illustrates an exemplary panoramic optical system to create 360- degree panoramic image in accordance with an embodiment of the present invention
  • FIG. 7 illustrates of the exterior view of a 360-degree panoramic camera device in accordance with an exemplary embodiment
  • FIG. 8 illustrates two configurations that transfer image from an optical image conduit module to the image sensor according to an embodiment of the panoramic camera device;
  • FIG. 8- A illustrates an optical image conduit module where its output facet is directly bonded on the image sensor;
  • FIG. 8-B illustrates an indirect relay lens approach that transfer the light of image from the output facet of the optical image conduit module to the image sensor via a relay lens ;
  • FIG. 9 illustrates an embodiment of optical image camera system that cap- tures the field of view of the half sphere;
  • FIG. 9- A shows an exemplary waveguide bundle configuration where the output facet aspect ratio is 4:3;
  • FIG. 9-B shows an exemplary waveguide bundle configuration where the output facet aspect ratio is 1: 1 ;
  • FIG. 10 illustrates an embodiment of optical image camera system that captures the field of view of the full sphere.
  • FIG. 10- A shows an exemplary waveguide bundle configuration where the output facet aspect ratio is 4:3;
  • FIG. 10-B shows an exemplary waveguide bundle configuration where the output facet aspect ratio is 1: 1 ;
  • FIG. 11 illustrates an embodiment that the panoramic picture is separated into four segments and projected onto single image sensor
  • FIG. 12 illustrates the procedures of the panoramic camera calibration.
  • FIG. 12-A shows the calibration procedure for the direct bonding configuration as shown in FIG. 8- A .
  • FIG. 12-B shows the calibration procedure for the relay lens configuration as shown in FIG. 8-B.
  • FIG. 12-A also represents the adaptive calibration procedure on the user side;
  • FIG. 13 illustrates process of digitizing, distortion-correction, and re-construction of panoramic image.
  • FIG. 1 shows a panoramic camera system according to an embodiment of the present invention comprising four lenses 101, an optical image conduit module comprising a plurality of optical fiber waveguides 102, an image sensor 103, and an image process module 104 that receives and processes the raw image stream in order to produce panoramic image.
  • the image process module 104 executes the diagram of calibration stage 105 to generate an image correction mapping grid 107 and stores in a memory module.
  • the image process module 104 executes the diagram of real-time panoramic stitching stage 106 which applies image correction mapping grid 107 to remove the image distortion and stitch a plurality of image segments to generate panoramic image.
  • the panoramic image is further encoded and transfer to a suitable storage device 108, such as memory card, hard disk drive, or any other data storage means.
  • a suitable storage device 108 such as memory card, hard disk drive, or any other data storage means.
  • the encoded panoramic image/video can also be transmitted through streaming methods 108 such as Wi-Fi, bluetooth, high-speed cellular data connection, etc.
  • the panoramic camera system of the present invention comprises a plurality of optical lenses.
  • FIG. 2 illustrates the field of view (FOV) 3 ⁇ 4 of a single optical lens unit according to an embodiment of the present invention.
  • a said panoramic scene N is defined as the annular area between two dashed lines 202 on a sphere.
  • the FOV Pi of an optical lens unit 201 is described as,
  • ⁇ 3 ⁇ 4, /3 ⁇ 4 and o 3 ⁇ 4 are horizontal FOV, vertical FOV and the orientation of said individual optical lens unit 201, respectively.
  • Each individual optical lens unit 201 captures a portion of a panoramic scene N, such that a plurality of optical lenses capture a image P that contains the panoramic scene N, i.e.
  • the orientations Oj of said image lenses 201 are separated by suitable angle depending on each said lens unit's horizontal FOV ⁇ 3 ⁇ 4 to satisfy
  • FIG. 3 shows an exemplary bent optical fiber waveguide according to an embodiment of the present invention.
  • An optical fiber waveguide comprises of millions of optical fibers with diameter as small as a few micrometers. Each optical fiber can transmit light rays to the output side with very little loss as a result of the total internal reflection effect of optical fiber.
  • the center line of an optical fiber waveguide 301 is bent with radius r and bending angle ⁇ degree. The light rays that enter from input facet 301 are rotated by ⁇ degree and transmitted to the output facet 302. In this way, the image from input facet 302 can be transferred to the output facet 303 with orientation changing by ⁇ degree.
  • FIG. 4 shows an exemplary twisted optical fiber waveguide according to an embodiment of the present invention.
  • an optical fiber waveguide 401 is twisted along its center line by 7 degree.
  • the image of input facet 402 can be transmitted to the output facet 403 by rotating 7 degree.
  • FIG. 5 shows an exemplary twisted and bent optical fiber waveguide according to an embodiment of the present invention.
  • an optical fiber waveguide 501 is firstly twisted along its center line by ⁇ , ⁇ degree, and its center line is then bent by f3 ⁇ 4 degree.
  • the image of input facet 503 can be transmitted to the output facet 502 with orientation rotating by f3 ⁇ 4 and ⁇ , ⁇ degree relative to respectively.
  • the input facet 502 can be orientated to point to an arbitrary direction whereas the orientation of the output facet 503 is fixed.
  • FIG. 6 shows an exemplary panoramic camera system according to an embodiment of the present invention.
  • four optical lenses 601, 602, 603, and 604 are evenly placed in an annulus volume.
  • Each optical lens is designed to capture over 90-degree horizontal FOV.
  • the orientations of the four optical lens in the example shown are
  • o ⁇ orientation of optical lens 601
  • 3 ⁇ 4 orientation of optical lens 602
  • 03 orientation of optical lens 603
  • o 4 orientation of optical lens 604.
  • the set of four optical lenses in the shown example is able to capture 360-degree image.
  • the image captured by each individual lens are transmitted through an optical image conduit module comprising four optical fiber waveguides 605, 606, 607, and 608 to the main image sensor 609.
  • the four optical fiber waveguides are bonded together with suitable optical adhesive to make a common output facet.
  • the output facet is polished and bonded to the main image sensor 609.
  • more than four optical lenses can be used to capture 360-degree image.
  • the separation angle of neighboring optical lens is determined by 360/ where M is the total number of optical lenses.
  • the horizontal FOV ⁇ 3 ⁇ 4 is required to be larger than 360/M to cover the said panoramic scene.
  • FIG. 7 shows an exemplary embodiment of exterior view of the panoramic camera system according to an embodiment of the present invention.
  • Four lenses 601, 602, 603, and 604 are evenly mounted on their lens holders. Due to assembly variation, it is subject to have tolerable assembly errors in six degree-of- freedom (rotation in pitch, yaw, roll and displacement in (x, y, z) and resulting in the shift of desired picture in x and y axes.
  • the assembly deviation from the normal positions can be optimized in the manufacturing processes and the resulted optical performance is acceptable to the optics design.
  • the entire enclosure is waterproof, insensitive to temperature and robust to shock, which allows the device to be mounted on exterior wall, vehicle, helmet, remote controlled mobile robots, and can be used in hazard environments such as high elevation or under water.
  • the top cover is flat and can be magnetic for ease of attachment to metal surface.
  • Four lens holders are even drilled on side of the enclosure.
  • Optical lenses are mounted into the lens holders where suitable waterproof bonding material may be used to prevent water leaking.
  • the location and tilt angle of the optical lenses can be configured differently depending on various applications and environments where the camera is installed.
  • the camera can be configured to capture lower hemisphere of view if installed on ceiling.
  • the lower part of the enclosure encloses the fiber optic image conduits and an image sensor. It is sealed and enforced to provide water, pressure, shock resistance.
  • FIG. 6 shows two types of approaches to couple the fiber bundle and image sensor according to an embodiment of the present invention to get satisfactory result.
  • FIG. 8-A illustrates the output facet of the optical image conduit module is directly bonded to the image sensor 609 with suitable optical adhesive.
  • the bonding procedure should be performed in a clean room environment in order to avoid dust particles from getting trapped in the bonding layer.
  • the selection of the optical adhesive for bonding should consider those that have a low shrinkage upon curing, high transparency for the luminescent light and low viscosity.
  • the bonding adhesive are also expected to be thermal and chemical stable, and has a high mechanical strength and low coefficient of thermal expansion. Comparing to traditional lens, fiber optic provide much higher light collecting efficiency ranging from 50% to 90%. In low light applications, the benefit of direct bonding can be more evident.
  • FIG. 8-B illustrates an indirect relay lens approach that transfer the light of image from the output facet of the optical image conduit module to the image sensor 609 via a relay lens 801.
  • this approach increases spacing to connect the output facet of the optical image conduit module to the image sensor, therefore it may increase the overall size of the panoramic camera.
  • the indirect relay lens approach offers better design flexibility.
  • the magnification ratio r is determined by the distance b from the output facet of the optical image conduit module to the relay lens 801 and the distance a from relay lens 801 to the image sensor 609 with the following equation
  • magnification ratio r By adjusting magnification ratio r, this approach allows to choose different size of image sensor 609 and optical fiber waveguide, and therefore provides more design flexibility.
  • FIG. 9 shows an exemplary hemisphere camera system according to an embodiment of the present invention.
  • four optical lenses 901, 902, 903, and 904 are evenly placed in an annulus volume.
  • Each optical lens is designed to capture over 90-degree horizontal FOV.
  • the orientations of the four optical lens in the example shown are
  • o ⁇ orientation of optical lens 901
  • 3 ⁇ 4 orientation of optical lens 902
  • 03 orientation of optical lens 903
  • o 4 orientation of optical lens 904.
  • the FOV of each individual lens are designed to have horizontal FOV ⁇ 3 ⁇ 4 > 90 and vertical FOV /3 ⁇ 4 > 90.
  • FIG. 9-A shows an exemplary optical image conduit module configuration according to an embodiment of the present invention.
  • four optical fiber waveguides are bundled together to transfer the received images from four optical lenses to the output facets.
  • the input and output facets of each individual optical fiber waveguide use the same aspect ratio format as image sensor.
  • the four optical fiber waveguides are bonded together with suitable optical adhesive to form a common output facet, which ensures bonded output facet maintains the same aspect ratio as image sensor.
  • FIG. 9-B shows another exemplary optical image conduit module configuration according to an embodiment of the present invention.
  • four optical fiber waveguides are bonded together to transfer the captured images from four optical lens to the output facets.
  • the input and output facets of each individual optical fiber waveguide use the aspect ratio 1:1.
  • the camera system uses commercial available image sensor with aspect ratio of 4:3 or 16:9, using the optical fiber waveguide with aspect ratio 1:1 , the sensor format efficiency could loss 25% to 45%.
  • image sensor 905 may be designed to have aspect ratio of 1: 1.
  • both examples of FIG. 9 shows that the image from the bonded output facet is transferred through the optical relay lens to the image sensor.
  • said bonded output facet of optical fiber waveguide bundles can be directly bonded to said image sensor with appropriate optical adhesive.
  • FIG. 10 shows an exemplary full-sphere camera system according to an embodiment of the present invention.
  • Two said hemisphere camera systems are mounted side by side to capture a full-sphere view, where the upper and lower one receive upper and lower hemisphere view respectively, and transfer images to their corresponding image sensor for post processing.
  • Two types of optical image conduit module configurations can be applied according to the image sensor aspect ratio format.
  • FIG. 10-A shows an exemplary optical image conduit module configuration where the output facet aspect ratio is 4:3.
  • FIG. 10-B shows an exemplary optical image conduit module configuration where the output facet aspect ratio is 1:1.
  • said image sensor 103 receives image from said optical image conduit module 102.
  • FIG. 11-A illustrates an exemplary panoramic image where it is divided into four segments Ni, N 2 , N3 and N 4 according to an embodiment of the present invention. Each segment corresponds to a subimage of 90 degree view angle.
  • FIG. 11-B illustrates the image received by said image sensor 103.
  • each subimage may have different rotation angle.
  • said optical image conduit module comprising four optical fiber waveguides (605, 606, 607, and 608) are used for transferring the image from said four optical lenses to the said image sensor 103, where four subimages P 1 ? P 2 , P3, and P 4 corresponds to the images that are received from said four optical lenses 601, 602, 603, and 604 respectively.
  • Subimages P 2 and P 3 havel80 degree orientation difference comparing to subimages Pi and P 4 .
  • the said lenses 101 and said relay lens 801 are both subject to standard lens distortion which can be described by camera homography model and lens distortion model.
  • the camera homography model can change during operation if the camera undergoes mechanical impact which shifts the relative transnational and rotational position between 1) optical lens units 101 and said input facets of optical image conduit; 2) said output facet of optical image conduit and said relay lens 801; 3) said relay lens 801 and said image sensor 609. Therefore, the calibration process is necessary for reconstructing panoramic image from subimages. The thorough calibration is done at manufacturing time, and adaptive calibration is done at normal operation time.
  • the calibration procedure is part of the manufacturing process.
  • the calibration is done after the completion of overall assembly.
  • the camera is placed in the panoramic image calibration environment (step 1201) .
  • the image that is captured by each lens 601-604 is transferred to the output facet of said optical image conduit.
  • the boundaries between optical fiber waveguides 605-608 at said output facet are identified by searching the dead zones (step 1202) .
  • Multiple pictures are taken using standard lens calibration charts (e.g. numbered checkerboard images) with different orientation to ensure no singularities.
  • each subimage is captured and rotated according to the configuration (step 1206) .
  • the overlapping between lenses are determined by finding the matching features among subimages (step 1207) .
  • the final calibration results are stored in the persistent memory (step 1210) .
  • said image sensor 801 and said relay lens 609 are assembled first (step 1211) and be placed into single camera calibration environment (step 1212). Then the camera parameters and lens distortion of the relay lens are calibrated (step 1213). After the verification (step 1214-1215), the optical image conduit and input lenses are assembled for the remaining calibration steps 1201-1210.
  • the said panoramic camera may be subject to strong mechanical impact such as strong mechanical shock.
  • strong mechanical shock In the cases of corner temperatures, uneven distortion in the assembly can occur due to the thermal characteristics of the material of housing or lens holder. Both factors can change the camera and lens' parameters especially on said lens units 101 and the optical image conduit because they are further away from the image sensor 609 than the relay lens 801.
  • the mechanical distortion causes observable image quality degradation to the final panoramic image, such as failure to remove overlapping or losing continuity in stitching the panoramic scene, and radial image distortion, etc. Therefore, it's necessary to perform adaptive calibration to restore the panoramic image quality on the user side.
  • the adaptive calibration on the user side follows the same procedure as illustrated in FIG. 12-A. By using pre-calibrated camera and lens parameters as initial values, the said adaptive calibration procedure can achieve much faster convergence rate on aforementioned parameters.
  • FIG. 13 illustrates the process of image digitization and post processing in the digital signal processor in operating mode.
  • Subimages are captured by each said lens unit 101 and directed onto said single image sensor 609 (step 1301-1302).
  • the raw digital image is then divided into multiple segments (step 1303).
  • the image processing algorithm applies the lens distortion correction model and overlapping removal model (step 1304) to all subimages. After correction, the subimages are rotated and stitched to form the final panoramic image (step 1305).

Abstract

A panoramic imaging method and device is disclosed to capture panoramic scene. The panoramic camera device comprises of a set of two or more optical lens units to collect a set of the panoramic scene segments, an optical image conduit module comprising a plurality of optical waveguides that are optically coupled to each optical lens unit and transfer to a main image sensor to form digital image signal. An image processing module reconstruct the panoramic image from the digital image signal.

Description

METHOD AND DEVICE FOR PANORAMIC IMAGING
FIELD OF THE INVENTION
[0001] The subject disclosure relates to an imaging method and device for forming panoramic images or videos with single or multiple image sensors.
BACKGROUND OF THE INVENTION
[0002] Panoramic imaging methods and devices have been devised in the past. The existing conventional methods can be categorized into three types.
[0003] The first type of methods are based on digital image stitching. Multiple images are taken by arranging multiple cameras pointing to different directions. A panoramic image can be generated by stitching these images together. This type of method increases system complexity, size, power consumption and cost, therefore it is not suitable for applications that are sensitive to aforementioned factors.
[0004] The second type of methods are based on ultra wide-angle lens, which creates a wide panoramic or hemispherical image but with strong visual distortion.
[0005] The third type of methods are similar to the second. Instead of using the wide-angle lens, a mirror is utilized to direct the field of view onto the image sensor. Besides the same disadvantage as the wide-angle methods, this type of methods suffer from a blind area in the center of the image sensor. OBJECTS OF THE INVENTION
[0006] It is an object of this invention to provide a panoramic imaging system with high picture quality yet low cost and low power-consumption.
SUMMARY OF THE INVENTION
[0007] This present invention discloses an innovative camera device that instantaneously and continuously captures panoramic scene. The panoramic camera device comprises of a set of two or more optical lens units, respectively, wherein each lens includes one or more optical elements to collect a portion of the panoramic scene and form a subimage on an image surface; an optical image conduit module comprising a plurality of optical fiber waveguides that are optically coupled to each optical lens unit to receive the light from the image surface, respectively, wherein each optical fiber waveguide is bent and/or twisted by suitable angles to transfer the light away from its input facet onto the output facet, which points towards the main image sensor; an optical conduction mechanism for transmitting the light emitted from the output facet of the optical image conduit onto the main image sensor; a main image sensor for converting the received light signal into the digital image; a supporting fixture for fastening the optical image conduit and the optical conduction mechanism; and an image processing module for constructing the panoramic image from the digital image signal.
[0008] In the optical image system, two or more lenses are arranged apart in space with desired constellation, to capture individual images simultaneously, which are used to reconstruct a panoramic image. Several configurations but not limited to those are disclosed to form 360-degree panoramic scene, hemisphere panoramic scene and full-sphere panoramic scene respectively. Furthermore, an optical image conduit module comprising a plurality of optical fiber waveguides transfers multiple desired field of views to the single image sensor. The image sensor consequently converts the light into digital signal for further processing.
[0009] The digital image signal is subsequently divided into multiple segments. The image distortions caused by lenses, both in geometry and light intensity, are corrected by software algorithm. The software correction parameters can be obtained from manufacturing calibration process. All the segments are stitched in a suitable order to reconstruct the panoramic image after distortion correction. Comparing to conventional stitching approaches, the disclosed method has both low computation demand and less distortion advantages. This image or image sequence is then compressed for storage or transmission.
[0010] Other objects, features and advantages of the present invention will be apparent from the accompanying drawings and from the detailed descriptions that follow below.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] For a more complete understanding of the invention, reference is made to the following description and accompanying drawings, in which:
[0012] FIG. 1 shows a flow diagram according to an embodiment of the present invention wherein a camera device receives segmented panoramic image via a plurality of lens and transfer through an optical image conduit module comprising a plurality of optical waveguides to a main image sensor, and processes the image in order to recreate seamless panoramic image ;
[0013] FIG. 2 illustrates a coordination system for describing a lens orientation and the field of view of a lens according to an embodiment of the panoramic camera device;
[0014] FIG. 3 illustrates an optical waveguide is bent along its center line by Θ degree according to an embodiment of the panoramic camera device;
[0015] FIG. 4 illustrates an optical waveguide is twisted by degree according to an embodiment of the panoramic camera device;
[0016] FIG. 5 illustrates an optical waveguide is firstly twisted by ¾ degree and then bent by f¾ degree according to an embodiment of the panoramic camera device;
[0017] FIG. 6 illustrates an exemplary panoramic optical system to create 360- degree panoramic image in accordance with an embodiment of the present invention;
[0018] FIG. 7 illustrates of the exterior view of a 360-degree panoramic camera device in accordance with an exemplary embodiment ;
[0019] FIG. 8 illustrates two configurations that transfer image from an optical image conduit module to the image sensor according to an embodiment of the panoramic camera device; FIG. 8- A illustrates an optical image conduit module where its output facet is directly bonded on the image sensor; FIG. 8-B illustrates an indirect relay lens approach that transfer the light of image from the output facet of the optical image conduit module to the image sensor via a relay lens ;
[0020] FIG. 9 illustrates an embodiment of optical image camera system that cap- tures the field of view of the half sphere; FIG. 9- A shows an exemplary waveguide bundle configuration where the output facet aspect ratio is 4:3; In another example, FIG. 9-B shows an exemplary waveguide bundle configuration where the output facet aspect ratio is 1: 1 ;
[0021] FIG. 10 illustrates an embodiment of optical image camera system that captures the field of view of the full sphere. FIG. 10- A shows an exemplary waveguide bundle configuration where the output facet aspect ratio is 4:3; In another example, FIG. 10-B shows an exemplary waveguide bundle configuration where the output facet aspect ratio is 1: 1 ;
[0022] FIG. 11 illustrates an embodiment that the panoramic picture is separated into four segments and projected onto single image sensor;
[0023] FIG. 12 illustrates the procedures of the panoramic camera calibration. FIG. 12-A shows the calibration procedure for the direct bonding configuration as shown in FIG. 8- A . FIG. 12-B shows the calibration procedure for the relay lens configuration as shown in FIG. 8-B. FIG. 12-A also represents the adaptive calibration procedure on the user side; and
[0024] FIG. 13 illustrates process of digitizing, distortion-correction, and re-construction of panoramic image.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
[0025] The embodiment showing in this section along with the associated figures are for exemplary illustration. The design of components, the number of lens/fiber optic image conduits and their arrangement may vary in accordance with applications and design requirements.
[0026] FIG. 1 shows a panoramic camera system according to an embodiment of the present invention comprising four lenses 101, an optical image conduit module comprising a plurality of optical fiber waveguides 102, an image sensor 103, and an image process module 104 that receives and processes the raw image stream in order to produce panoramic image. During initial characterization of panoramic camera system, the image process module 104 executes the diagram of calibration stage 105 to generate an image correction mapping grid 107 and stores in a memory module. After calibration completes, the image process module 104 executes the diagram of real-time panoramic stitching stage 106 which applies image correction mapping grid 107 to remove the image distortion and stitch a plurality of image segments to generate panoramic image. The panoramic image is further encoded and transfer to a suitable storage device 108, such as memory card, hard disk drive, or any other data storage means. The encoded panoramic image/video can also be transmitted through streaming methods 108 such as Wi-Fi, bluetooth, high-speed cellular data connection, etc.
[0027] The panoramic camera system of the present invention comprises a plurality of optical lenses. FIG. 2 illustrates the field of view (FOV) ¾ of a single optical lens unit according to an embodiment of the present invention. A said panoramic scene N is defined as the annular area between two dashed lines 202 on a sphere. The FOV Pi of an optical lens unit 201 is described as,
Figure imgf000007_0001
, where <¾, /¾ and o¾ are horizontal FOV, vertical FOV and the orientation of said individual optical lens unit 201, respectively. A plurality of optical image lenses are placed in an annulus volume with orientation o¾ = { 1, ^}, where φ is the angle between z axis 203 and o¾, and is the angle between x axis 204 and the projection of Oj on xy plane. Each individual optical lens unit 201 captures a portion of a panoramic scene N, such that a plurality of optical lenses capture a image P that contains the panoramic scene N, i.e.
The orientations Oj of said image lenses 201 are separated by suitable angle depending on each said lens unit's horizontal FOV <¾ to satisfy
Figure imgf000008_0001
, where {ΨΙ, Ψι ν} is the orientation of a reference image lens o\ .
[0028] FIG. 3 shows an exemplary bent optical fiber waveguide according to an embodiment of the present invention. An optical fiber waveguide comprises of millions of optical fibers with diameter as small as a few micrometers. Each optical fiber can transmit light rays to the output side with very little loss as a result of the total internal reflection effect of optical fiber. In the example shown, the center line of an optical fiber waveguide 301 is bent with radius r and bending angle Θ degree. The light rays that enter from input facet 301 are rotated by Θ degree and transmitted to the output facet 302. In this way, the image from input facet 302 can be transferred to the output facet 303 with orientation changing by Θ degree.
[0029] FIG. 4 shows an exemplary twisted optical fiber waveguide according to an embodiment of the present invention. In the example shown, an optical fiber waveguide 401 is twisted along its center line by 7 degree. As a result, the image of input facet 402 can be transmitted to the output facet 403 by rotating 7 degree. [0030] FIG. 5 shows an exemplary twisted and bent optical fiber waveguide according to an embodiment of the present invention. In the example shown, an optical fiber waveguide 501 is firstly twisted along its center line by η,ι degree, and its center line is then bent by f¾ degree. As a result, the image of input facet 503 can be transmitted to the output facet 502 with orientation rotating by f¾ and η,ι degree relative to
Figure imgf000009_0001
respectively. Using optical fiber waveguide 501, the input facet 502 can be orientated to point to an arbitrary direction whereas the orientation of the output facet 503 is fixed.
[0031] FIG. 6 shows an exemplary panoramic camera system according to an embodiment of the present invention. In the example shown, four optical lenses 601, 602, 603, and 604 are evenly placed in an annulus volume. Each optical lens is designed to capture over 90-degree horizontal FOV. The orientations of the four optical lens in the example shown are
0l = {90, 0}, o2 = {90, 90}, o3 = {90, 180}, and o4 = {90, 270}, where o\ is orientation of optical lens 601, (¾ is orientation of optical lens 602, 03 is orientation of optical lens 603, and o4 is orientation of optical lens 604. As the horizontal FOV is greater than the separation angle of neighboring optical lens, the set of four optical lenses in the shown example is able to capture 360-degree image. The image captured by each individual lens are transmitted through an optical image conduit module comprising four optical fiber waveguides 605, 606, 607, and 608 to the main image sensor 609. In the example shown, 605 and 607 are bended optical fiber waveguide with bending angle Θ = 90 degree; 606 and 608 are twisted and bended optical fiber waveguide with twisting angle 7 = 90 degree and bending angle Θ = 90 degree. The four optical fiber waveguides are bonded together with suitable optical adhesive to make a common output facet. The output facet is polished and bonded to the main image sensor 609. In another example where input and output facets of the optical fiber waveguides are of square format, the optical image conduit module comprises four bended optical fiber waveguide with bending angle Θ = 90 degree. In another example, more than four optical lenses can be used to capture 360-degree image. The separation angle of neighboring optical lens is determined by 360/ where M is the total number of optical lenses. The horizontal FOV <¾ is required to be larger than 360/M to cover the said panoramic scene.
[0032] FIG. 7 shows an exemplary embodiment of exterior view of the panoramic camera system according to an embodiment of the present invention. Four lenses 601, 602, 603, and 604 are evenly mounted on their lens holders. Due to assembly variation, it is subject to have tolerable assembly errors in six degree-of- freedom (rotation in pitch, yaw, roll and displacement in (x, y, z) and resulting in the shift of desired picture in x and y axes. The assembly deviation from the normal positions can be optimized in the manufacturing processes and the resulted optical performance is acceptable to the optics design.
[0033] In the exemplary embodiment, the entire enclosure is waterproof, insensitive to temperature and robust to shock, which allows the device to be mounted on exterior wall, vehicle, helmet, remote controlled mobile robots, and can be used in hazard environments such as high elevation or under water.
[0034] In the exemplary embodiment, the top cover is flat and can be magnetic for ease of attachment to metal surface. Four lens holders are even drilled on side of the enclosure. Optical lenses are mounted into the lens holders where suitable waterproof bonding material may be used to prevent water leaking. The location and tilt angle of the optical lenses can be configured differently depending on various applications and environments where the camera is installed. For example, the camera can be configured to capture lower hemisphere of view if installed on ceiling. The lower part of the enclosure encloses the fiber optic image conduits and an image sensor. It is sealed and enforced to provide water, pressure, shock resistance.
[0035] In the exemplary panoramic camera system shown in FIG. 6 , the output facet of optical image conduit module transmit to image sensor 609. In order to successfully couple light from an imaging fiber bundle to the image sensor, the image sensor and fiber bundle must be in very close proximity. Light emerges from the individual fibers at large angles, and a gap between fiber and image sensor will lead to a loss in resolution. FIG. 8 shows two types of approaches to couple the fiber bundle and image sensor according to an embodiment of the present invention to get satisfactory result.
[0036] FIG. 8-A illustrates the output facet of the optical image conduit module is directly bonded to the image sensor 609 with suitable optical adhesive. To ensure satisfactory quality, the bonding procedure should be performed in a clean room environment in order to avoid dust particles from getting trapped in the bonding layer. The selection of the optical adhesive for bonding should consider those that have a low shrinkage upon curing, high transparency for the luminescent light and low viscosity. The bonding adhesive are also expected to be thermal and chemical stable, and has a high mechanical strength and low coefficient of thermal expansion. Comparing to traditional lens, fiber optic provide much higher light collecting efficiency ranging from 50% to 90%. In low light applications, the benefit of direct bonding can be more evident.
[0037] FIG. 8-B illustrates an indirect relay lens approach that transfer the light of image from the output facet of the optical image conduit module to the image sensor 609 via a relay lens 801. Comparing to the direct bonding approach in FIG. 8- A, this approach increases spacing to connect the output facet of the optical image conduit module to the image sensor, therefore it may increase the overall size of the panoramic camera. On the other hand, the indirect relay lens approach offers better design flexibility. In the example shown, the magnification ratio r is determined by the distance b from the output facet of the optical image conduit module to the relay lens 801 and the distance a from relay lens 801 to the image sensor 609 with the following equation
a
r = b -
By adjusting magnification ratio r, this approach allows to choose different size of image sensor 609 and optical fiber waveguide, and therefore provides more design flexibility.
[0038] FIG. 9 shows an exemplary hemisphere camera system according to an embodiment of the present invention. In the example shown, four optical lenses 901, 902, 903, and 904 are evenly placed in an annulus volume. Each optical lens is designed to capture over 90-degree horizontal FOV. The orientations of the four optical lens in the example shown are
0l = {45, 0}, o2 = {45, 90}, o3 = {45, 180}, and o4 = {45, 270}, where o\ is orientation of optical lens 901, (¾ is orientation of optical lens 902, 03 is orientation of optical lens 903, and o4 is orientation of optical lens 904. The FOV of each individual lens are designed to have horizontal FOV <¾ > 90 and vertical FOV /¾ > 90.
[0039] As most commercially available image sensors use unequal aspect ratio format, for example 4:3 or 16:9. In order to optimize the image sensor utilization efficiency, FIG. 9-A shows an exemplary optical image conduit module configuration according to an embodiment of the present invention. In the example shown, four optical fiber waveguides are bundled together to transfer the received images from four optical lenses to the output facets. The input and output facets of each individual optical fiber waveguide use the same aspect ratio format as image sensor. 905 and 907 are bent optical fiber waveguide with bending angle Θ = 90 degree; 906 and 908 are twisted and bent optical fiber waveguide with twisting angle = 90 degree and bending angle Θ = 45 degree. The four optical fiber waveguides are bonded together with suitable optical adhesive to form a common output facet, which ensures bonded output facet maintains the same aspect ratio as image sensor.
[0040] FIG. 9-B shows another exemplary optical image conduit module configuration according to an embodiment of the present invention. In the example shown, four optical fiber waveguides are bonded together to transfer the captured images from four optical lens to the output facets. The input and output facets of each individual optical fiber waveguide use the aspect ratio 1:1. Due to choice of the aspect ratio, four optical fiber waveguides 915, 916, 917, and 918 are all bent optical fiber waveguide with bending angle Θ = 90 degree. Choice of this configuration may help reduce optical fiber waveguide manufacturing cost. However, if the camera system uses commercial available image sensor with aspect ratio of 4:3 or 16:9, using the optical fiber waveguide with aspect ratio 1:1 , the sensor format efficiency could loss 25% to 45%. In order to maximize the sensor utilization efficiency, image sensor 905 may be designed to have aspect ratio of 1: 1.
[0041] Both examples of FIG. 9 shows that the image from the bonded output facet is transferred through the optical relay lens to the image sensor. In another embodiment, similar to the example of FIG. 8-A, said bonded output facet of optical fiber waveguide bundles can be directly bonded to said image sensor with appropriate optical adhesive.
[0042] In another embodiment, FIG. 10 shows an exemplary full-sphere camera system according to an embodiment of the present invention. Two said hemisphere camera systems are mounted side by side to capture a full-sphere view, where the upper and lower one receive upper and lower hemisphere view respectively, and transfer images to their corresponding image sensor for post processing. Two types of optical image conduit module configurations can be applied according to the image sensor aspect ratio format. In one example, FIG. 10-A shows an exemplary optical image conduit module configuration where the output facet aspect ratio is 4:3. In another example, FIG. 10-B shows an exemplary optical image conduit module configuration where the output facet aspect ratio is 1:1.
[0043] In another embodiment, said image sensor 103 receives image from said optical image conduit module 102. FIG. 11-A illustrates an exemplary panoramic image where it is divided into four segments Ni, N2, N3 and N4 according to an embodiment of the present invention. Each segment corresponds to a subimage of 90 degree view angle. FIG. 11-B illustrates the image received by said image sensor 103. Depending on the configuration of said optical image conduit module described in FIG. 8, each subimage may have different rotation angle. In the example shown, said optical image conduit module comprising four optical fiber waveguides (605, 606, 607, and 608) are used for transferring the image from said four optical lenses to the said image sensor 103, where four subimages P1 ? P2, P3, and P4 corresponds to the images that are received from said four optical lenses 601, 602, 603, and 604 respectively. Subimages P2 and P3 havel80 degree orientation difference comparing to subimages Pi and P4.
[0044] The said lenses 101 and said relay lens 801 are both subject to standard lens distortion which can be described by camera homography model and lens distortion model. In the practical applications, the camera homography model can change during operation if the camera undergoes mechanical impact which shifts the relative transnational and rotational position between 1) optical lens units 101 and said input facets of optical image conduit; 2) said output facet of optical image conduit and said relay lens 801; 3) said relay lens 801 and said image sensor 609. Therefore, the calibration process is necessary for reconstructing panoramic image from subimages. The thorough calibration is done at manufacturing time, and adaptive calibration is done at normal operation time.
[0045] In the embodiment illustrated in the flow diagram of FIG. 12, the calibration procedure is part of the manufacturing process. In one embodiment, as in the case of direct bundle as illustrated in FIG. 8-A, the calibration is done after the completion of overall assembly. The camera is placed in the panoramic image calibration environment (step 1201) . As mentioned before, the image that is captured by each lens 601-604 is transferred to the output facet of said optical image conduit. The boundaries between optical fiber waveguides 605-608 at said output facet are identified by searching the dead zones (step 1202) . Multiple pictures are taken using standard lens calibration charts (e.g. numbered checkerboard images) with different orientation to ensure no singularities. Then the camera parameters and lens distortion models are calibrated for each lens based on the subimages using common single camera calibration algorithm (step 1203) . After the verification (step 1204-1205) , each subimage is captured and rotated according to the configuration (step 1206) . Then the overlapping between lenses are determined by finding the matching features among subimages (step 1207) . After the final verification (step 1208-1209) , the final calibration results are stored in the persistent memory (step 1210) . [0046] In the other embodiment, as in the case of relay lens as illustrated in FIG. 8- B, the camera parameters and lens distortion need to be calibrated separately before the calibration of said optical lens units 101. During manufacturing process, said image sensor 801 and said relay lens 609 are assembled first (step 1211) and be placed into single camera calibration environment (step 1212). Then the camera parameters and lens distortion of the relay lens are calibrated (step 1213). After the verification (step 1214-1215), the optical image conduit and input lenses are assembled for the remaining calibration steps 1201-1210.
[0047] On the user side, the said panoramic camera may be subject to strong mechanical impact such as strong mechanical shock. In the cases of corner temperatures, uneven distortion in the assembly can occur due to the thermal characteristics of the material of housing or lens holder. Both factors can change the camera and lens' parameters especially on said lens units 101 and the optical image conduit because they are further away from the image sensor 609 than the relay lens 801. In the worst case, the mechanical distortion causes observable image quality degradation to the final panoramic image, such as failure to remove overlapping or losing continuity in stitching the panoramic scene, and radial image distortion, etc. Therefore, it's necessary to perform adaptive calibration to restore the panoramic image quality on the user side. The adaptive calibration on the user side follows the same procedure as illustrated in FIG. 12-A. By using pre-calibrated camera and lens parameters as initial values, the said adaptive calibration procedure can achieve much faster convergence rate on aforementioned parameters.
[0048] The flow diagram FIG. 13 illustrates the process of image digitization and post processing in the digital signal processor in operating mode. Subimages are captured by each said lens unit 101 and directed onto said single image sensor 609 (step 1301-1302). The raw digital image is then divided into multiple segments (step 1303). The image processing algorithm applies the lens distortion correction model and overlapping removal model (step 1304) to all subimages. After correction, the subimages are rotated and stitched to form the final panoramic image (step 1305).

Claims

What is claimed:
1. An optical system for observing the panoramic scene, comprising of: a plurality of optical modules, comprising: a set of two or more optical lens units, respectively, wherein each lens includes one or more optical elements to collect a portion of the panoramic scene and form a subimage on an image surface; an optical image conduit module comprising a plurality of optical fiber waveguides that are optically coupled to each optical lens unit to receive the light from the corresponding image surface, respectively, wherein each optical fiber waveguide is bent and/or twisted by suitable angles to transfer the light away from its input facet onto the output facet, which points towards the main image sensor; an optical conduction mechanism for transmitting the light emitted from the output facet of the optical image conduit onto the main image sensor; a main image sensor for converting the received light signal into the digital image; a supporting fixture for fastening the optical image conduit and the optical conduction mechanism; an image processing module for constructing the panoramic image from the digital image signal.
2. The system of claim 1, wherein said individual optical lens unit captures a portion of said panoramic scene. The field of view (FOV) of each said optical lens unit is
, where <¾, /¾ and o¾ are the horizontal FOV, vertical FOV and the orientation of said individual lens unit, respectively.
3. The system of claim 1, wherein said a plurality of optical image lenses are placed in an annulus volume with orientation Oj = {01, 01^}· The orientations Oi of said image lenses are separated by suitable angle depending on each said lens unit's horizontal FOV <¾ to satisfy
Figure imgf000019_0001
, where {01, 01^} is the orientation of a reference image lens o\.
4. The system of claim 1, wherein said a set of optical image lenses capture the image that contains the panoramic scene, i.e.
P = {J P ^ N
, where N represents the panoramic scene.
5. The system of claim 1, wherein the input facet of each said optical fiber waveguide is optically coupled to the image surface of the corresponding optical lens unit by a suitable coupling method including optical adhesive bonding, to receive the light from the corresponding image surface.
6. The system of claim 1, wherein said optical image conduit module combines a plurality of twisted and bent optical fiber waveguides with suitable twisting and bending angles, where their input and output facets allow unequal aspect ratio formats. The output facets of optical fiber waveguides are bonded to form a common output facet. The object image at said common output facet is then transmitted to said image sensor through said optical conduction mechanism. The said common output facet maintains unequal aspect ratio format.
7. The system of claim 1 , wherein said optical image conduit module combines a plurality of bent optical fiber waveguides with suitable bending angles, where their input and output facets allows equal aspect ratio format. The output facets of optical fiber waveguides are bonded to form a common output facet. The object image at said common output facet is then transmitted to said image sensor through said optical conduction mechanism. The said common output facet maintains equal aspect ratio format.
8. The system of claim 1 , wherein optical conduction mechanism is to directly bond common facet of said optical image conduit to the image sensor with suitable optical clear adhesive. Said image sensor receives light signal seamlessly from said optical image conduit.
9. The system of claim 1 , wherein said optical panoramic imaging system, comprising said optical lens units, said optical image conduit, and said main image sensor, is mounted in said supporting fixture. The top cover of said supporting fixture can be magnetic to attach to suitable surface.
10. The system of claim 1 , wherein said a set of optical lens units are placed in an annulus volume with each lens unit captures a portion of the annulus panoramic scene j . The union of all portions Pt contains the annulus panoramic scene N.
11. The system of claim 1 , where in said a set of optical lens units are placed in a hemisphere volume with each lens unit captures a portion of the hemisphere panoramic scene P¾. The union of all portions Ρ¾ contains the hemisphere panoramic scene N.
12. The system of claim 1, where in said a set of optical lens units are placed in a full-sphere volume with each lens unit captures a portion of the full-sphere panoramic scene Pj. The union of all portions Pj contains the full-sphere panoramic scene N.
13. The system of claim 1, wherein said image processing module removes the distortion and overlap in said digital image from said image sensor.
14. The system of claim 1, wherein said image processing module splits said corrected digital image into multiple segments and rotates and stitch each segment into final said panoramic image.
15. An optical system for observing the panoramic scene, comprising of: a plurality of optical elements, comprising: a set of two or more optical lens units, respectively, wherein each lens includes one or more optical elements to collect a portion of the panoramic scene and form a subimage on an image surface; an optical image conduit module comprising a plurality of optical fiber waveguides that are optically coupled to each optical lens unit to receive the light from the image surface, respectively, wherein each optical fiber waveguide is bent by an angle to transfer the light away from its input facet onto the output facet, which points towards the main image sensor; a main image sensor for converting the received light signal into the digital image; a relay lens between the optical image conduit and the image sensor to project the object image from the output facet onto the main image sensor; a supporting fixture for fastening the optical image conduit and/or the optical conduction mechanism; an image processing module for constructing the panoramic image from the digital image signal.
The system of claim 15, wherein said relay lens, as an alternative solution to claim 8, projects the objective image from said output facet of said optical image conduit onto said image sensor. The magnification ratio is determined by:
a
PCT/US2016/020144 2015-11-03 2016-03-01 Method and device for panoramic imaging WO2017078771A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201680001783.2A CN109478011A (en) 2015-11-03 2016-03-01 Method and apparatus for panoramic imagery

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201562249946P 2015-11-03 2015-11-03
US62/249,946 2015-11-03

Publications (1)

Publication Number Publication Date
WO2017078771A1 true WO2017078771A1 (en) 2017-05-11

Family

ID=58662428

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2016/020144 WO2017078771A1 (en) 2015-11-03 2016-03-01 Method and device for panoramic imaging

Country Status (2)

Country Link
CN (1) CN109478011A (en)
WO (1) WO2017078771A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106851201A (en) * 2017-02-09 2017-06-13 苏州慧景光电科技有限公司 Vehicle-mounted panoramic image system and its scaling method based on fibre optic image transmission technology

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020064341A1 (en) * 2000-11-27 2002-05-30 Fauver Mark E. Micro-fabricated optical waveguide for use in scanning fiber displays and scanned fiber image acquisition
US20070247519A1 (en) * 2005-03-05 2007-10-25 Wag Display Corporation, Inc. Display System with Moving Pixels for 2D and 3D Image Formation
US20150207990A1 (en) * 2012-08-20 2015-07-23 The Regents Of The University Of California Monocentric lens designs and associated imaging systems having wide field of view and high resolution

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2252043A6 (en) * 1973-10-25 1975-06-13 Mas Roland Panoramic cine photography viewing head - images are transmitted by light pipes to film or to closed circuit TV
CN100582694C (en) * 2008-01-23 2010-01-20 天津大学 Common image face multi-optical spectrum imaging technology
CN101221350B (en) * 2008-01-23 2010-06-09 天津大学 Image surface partitioning multi-optical channel common image surface imaging device
CN101256275B (en) * 2008-04-08 2011-08-31 天津大学 Microminiaturization co-image face panoramic imagery technique
EP2369382A1 (en) * 2010-03-22 2011-09-28 Schott Corporation Rigid multi-directional imaging bundle and imaging assembly incorporating the same

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020064341A1 (en) * 2000-11-27 2002-05-30 Fauver Mark E. Micro-fabricated optical waveguide for use in scanning fiber displays and scanned fiber image acquisition
US20070247519A1 (en) * 2005-03-05 2007-10-25 Wag Display Corporation, Inc. Display System with Moving Pixels for 2D and 3D Image Formation
US20150207990A1 (en) * 2012-08-20 2015-07-23 The Regents Of The University Of California Monocentric lens designs and associated imaging systems having wide field of view and high resolution

Also Published As

Publication number Publication date
CN109478011A (en) 2019-03-15

Similar Documents

Publication Publication Date Title
US20170139131A1 (en) Coherent fiber array with dense fiber optic bundles for light-field and high resolution image acquisition
KR100599423B1 (en) An omnidirectional imaging apparatus
JP4247113B2 (en) Method for capturing a panoramic image using a rectangular image sensor
CN105324984A (en) Method and system for generating multi-projection images
CN103685917A (en) Image processor, image processing method and program, and imaging system
CN105744137A (en) System for real-time correction and splicing of image formed by fish-eye lens through using hardware circuit
WO2010116370A1 (en) Camera systems having multiple image sensors combined with a single axis mechanical gimbal
US9503638B1 (en) High-resolution single-viewpoint panoramic camera and method of obtaining high-resolution panoramic images with a single viewpoint
CN102117008A (en) Spherical panoramic monitoring device
TW201719267A (en) A wide field of view camera for integration with a mobile device
JP2007517264A (en) Multidimensional imaging apparatus, system and method
KR101689534B1 (en) Multiscale Imaging System
CN101271187A (en) Non-dead angle binocular solid all-directional vision sensing equipment
CN104836943A (en) Planar distribution multi-camera panorama shooting system and realization method thereof
WO2017078771A1 (en) Method and device for panoramic imaging
CN205647708U (en) System for use hardware circuit to correct and concatenation image that fisheye lens becomes is real -time
JP2002262307A (en) Image input system
JP5083979B2 (en) Image enhancement method and imaging apparatus
US20100289881A1 (en) Camera for multiple perspective image capturing
CN108604055B (en) Omnidirectional catadioptric lens with odd-order aspheric profile or multiple lenses
CN105892022A (en) Image acquisition device and panorama camera
KR101789317B1 (en) A Multiscale imaging system
JPH114373A (en) Method and system for configurating full circumference panorama image
JP2004153605A (en) Image pickup device and system for transmitting pick-up image
US20110242272A1 (en) Rigid multi-directional imaging bundle and imaging assembly incorporating the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16862609

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16862609

Country of ref document: EP

Kind code of ref document: A1