CN101554043B - Data packet processing method and system for image sensor - Google Patents
Data packet processing method and system for image sensor Download PDFInfo
- Publication number
- CN101554043B CN101554043B CN200780020045.3A CN200780020045A CN101554043B CN 101554043 B CN101554043 B CN 101554043B CN 200780020045 A CN200780020045 A CN 200780020045A CN 101554043 B CN101554043 B CN 101554043B
- Authority
- CN
- China
- Prior art keywords
- data
- data set
- imageing sensor
- read
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims description 10
- 238000000034 method Methods 0.000 claims abstract description 75
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000004458 analytical method Methods 0.000 claims description 22
- 230000033001 locomotion Effects 0.000 claims description 19
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 13
- 230000004913 activation Effects 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 230000003213 activating effect Effects 0.000 claims description 3
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 230000000295 complement effect Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 229910044991 metal oxide Inorganic materials 0.000 claims description 3
- 150000004706 metal oxides Chemical class 0.000 claims description 3
- 239000004065 semiconductor Substances 0.000 claims description 3
- 230000008859 change Effects 0.000 claims description 2
- 238000010223 real-time analysis Methods 0.000 claims description 2
- 230000001747 exhibiting effect Effects 0.000 claims 1
- 238000003860 storage Methods 0.000 abstract description 14
- 238000004590 computer program Methods 0.000 abstract description 8
- 238000003384 imaging method Methods 0.000 abstract 3
- 230000006870 function Effects 0.000 description 19
- 230000008569 process Effects 0.000 description 16
- 238000012958 reprocessing Methods 0.000 description 6
- 230000008901 benefit Effects 0.000 description 4
- 238000012937 correction Methods 0.000 description 4
- 206010034972 Photosensitivity reaction Diseases 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000004091 panning Methods 0.000 description 3
- 230000036211 photosensitivity Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000005267 amalgamation Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 238000004321 preservation Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000003760 hair shine Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B37/00—Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
Abstract
The invention relates to a method and an arrangement for processing records of imaging sensors, a corresponding computer program, and a corresponding computer-readable storage medium which can be usedparticularly for creating panoramic photographs with the aid of mobile terminals, e.g. a digital camera, a mobile telephone, or similar. In said method for processing records of imaging sensors, seve ral successive records of an imaging sensor are read at least in part, records read during reading of additional records are analyzed in order to determine matching areas within the read records, and data of several successive records is combined to a panoramic image during output via data output means.
Description
Technical field
The invention relates to data set processing method and treatment system, its corresponding computer program and the corresponding computer-readable storage medium of imageing sensor; Especially can be used for through using some mobile terminal devices; For example equipment such as digital camera, mobile phone is processed panoramic picture.
Background technology
At present there have been some can make the equipment of panoramic picture.But these known arrangement all need use some special optical elements, and object is demonstrated on some optics or imageing sensor with desired panoramic format, for example film (like that) or CCD chip etc.Need extra material at this optical element that uses.Particularly, maybe can carry out the equipment that sizable extensive angle is taken, the fringe cost of great number more can occur sometimes for high-end high-quality equipment.
And present other known distant view photograph method for making; The method that particularly digital camera adopted then is that the requirement user carries out necessary manual stack location and calibration to camera; So that a plurality of image energies are bonded with each other, it is automatically that this process has part, but needs manual adjustment mostly.Some preferably method can give the user some promptings, for example point out a last photo, so that the user positions with translucent vertical bar.But for some scenes of difficulty relatively, when just reference object has more similar a plurality of part, stone walling for example, operating usually a lot of problems can occur.
Another kind of actual conditions are capture video, from these videos, extract panorama through graphical analysis afterwards.This requirement stores lot of data in the finite storage space very much at one; Particularly for small-sized equipment; Picture digital camera and mobile phone if image resolution ratio is higher, have not had memory space so soon; Otherwise just reduce shooting quality, because the quality of the single image in the video capture is lower.Can't determine that in the process of taking which image will be used for panoramic pictures and extract.
The another kind of selection is to take and store a plurality of individual photos; Desired memory space and processing time can increase along with the increase of shooting angle and resolution; The quality of single photo and combined photograph also will increase thereupon, so can exceed the capacity of mini-plant.
In above two kinds of distant view photograph generation methods mentioning, the photo that the user can't preview will generate, and can only see current captured photo/data set.And the data set that has absorbed also can't be seen.
Described a kind of method among the Japan publication application JP 2006-135386, its video pictures with the multi-section camera is combined into a Zhang Quanjing photo.Can the single photo that all cameras are taken at one time be combined as a Zhang Quanjing especially shines.The processing of sequential photograph is not described among the JP 2006-135386.
Summary of the invention
The object of the present invention is to provide a kind of data set processing method and treatment system, its corresponding computer program and corresponding computer-readable storage medium of imageing sensor; It can overcome the drawback of known arrangement; And can realize the automatic shooting of distant view photograph, and shooting angle can be selected freely.
Above-mentioned purpose realizes through the invention scheme with claim 1,17,23 and 24 said characteristics.And Rights attached thereto require the preferred embodiment of the present invention is described.
According to a special advantage of the present invention, one does not possess special full shot, and only is the equipment of loading onto the common type of lens, mobile phone for example, and photo also can pan.The method that the data set to imageing sensor of the application of the invention is handled, a rapid adjoining land of data treatment element ability reads the data set of an imageing sensor at least in part, so just can increase these and take performance.When other data of imageing sensor just are being read, simultaneously can to before the data that read analyze, be preferably automatically and analyze, to calculate the zone that is superimposed in the read data group.Preferably, if the data owner of having read analyze as view data, in image data set, finding the image-region that is superimposed.At this image processing method that can adopt all to be fit to.On the basis of the corresponding picture congruent region that is calculated, the data set of organizing in the continuous data set will mutual successively amalgamation be the photo of a view picture more.The photo of this view picture has comprised those data that constitute scenes, and this scene is bigger than the scene that the wall scroll data set of imageing sensor is constituted.Preferably, the data that generate the view picture photo can be preserved together, and/or through data output device output.The data of this view picture photo can be with the output of the form of vision, or forward on other the data processing equipment with the form of file, further to handle etc.
Preferably, all read through the whole data set of method of the present invention imageing sensor.Only need very low resolution and carry out analysis of image data, like this, even if inefficient equipment also can very fast reading of data.
According to the system that the data set of imageing sensor is handled of the present invention; Comprise at least one imageing sensor; And at least one have the storage device data processing equipment or; Perhaps at least one imageing sensor, at least one has data processing equipment and at least one of storage medium and carries out the device of data output.Wherein, This system construction becomes can make many groups of consecutive data sets of at least one imageing sensor can be read by part at least; And the data set after reading can be analyzed; So just can calculate in the data set of having read corresponding congruent region, and will a plurality of consecutive data sets (being preferably the data set of single image transducer) and be kept in the storage device together, and/or it is exported through data output device about the regional information that is superimposed with each other.Wherein, in a kind of output, be preferably in the output of vision, the data of this at least one imageing sensor can be spliced into single image.Carrying out vision when output, preferably, view data illustrates on display screen, and the image that on display screen, appears can be widened gradually through the view data in the data set of reading subsequently and comes.In the preferred embodiment of system of the present invention, imageing sensor is ccd sensor (CCD=Charge Coupled Device charge coupled device) or cmos sensor (CMOS=Complementary Metal Oxide Semiconductor complementary metal oxide semiconductors (CMOS)).
According to a preferred embodiment of system of the present invention, make optical element that scene forms images at least one imageing sensor place the front side of this imageing sensor.This optical element is preferably lens combination, for example camera lens.
Preferably, system of the present invention can comprise active element, is used for activating reading the imageing sensor data.This active element can be applied to have the camera or the mobile phone of camera function, and for example it can be a shutter.
The device that is used for data output preferably also contains the device that the data of at least one imageing sensor is carried out vision output.
A preferred embodiment according to the method for the invention can activate shutter function through this active element, wherein, when shutter function is kept constantly being activated, will organize consecutive data set on the reading images transducer more.Preferably, under the situation that shutter function is activated, the continual reading of data of meeting that is to say that single image can read faster in order successively.
According to an embodiment of system of the present invention, have at least an imageing sensor to be integrated on the mobile terminal device, for example digital camera, palmtop PC (PDA) or mobile phone.In these cases, active element shutter just.As long as shutter pinned, data set that will the reading images transducer is to its analysis and be spliced into the photo of a view picture.Also have a kind of optional mode to be, thereby through active element being carried out disposable action beginning first time shutter operation, to active element being carried out one-time action end second time process.Pressing and unclamping between the active element (shutter), perhaps for the first time with the blanking time of for the second time active element being moved in, shutter function is activated, the data set in the imageing sensor is read out.The data set or at least a portion data that read in by the process of Dispensable continuous activation in shutter function are used in the photo that pieces together a view picture.When data set reads,, just can generate distant view photograph through the imageing sensor that between the scene of picked-up, relatively moves.Go up at mobile terminal device (for example digital camera, PDA or mobile phone) under the situation of integrated image sensor, just can under the situation that shutter is pinned, shake mobile terminal device, take required scene, just can obtain distant view photograph then.If at a vehicle photographs photo that goes, also can generate the photo of amalgamation so without mobile camera, just distant view photograph.That is to say, need only imageing sensor and need exist relative motion just passable between the scene of picked-up.
According to another preferred embodiment of the inventive method,, the data of reading between two photos that link to each other successively, to draw a 2D conversion between the view data that just two are read successively when being analyzed.Preferably, this 2D conversion draws through 2D homography matrix or projection conversion.This projection conversion can only realize through for example Lukas-kanade algorithm.
Preferably, be continuous to the analysis of data set, and/or carry out real-time analysis.Particularly preferably, this analysis also produces the data that move the path of running through scene about imageing sensor, for example realizes according to the luminous flux of the image data set that reads continuously.
According to another preferred embodiment of the method for the invention, the result of data analysis is used to calculate another data set that when needs and preserve the index map picture that is used for scene, and/or how add this component to stored each minute on the picture.Preferably, this computational process is automatically carried out.Here said component just refers to a part of distant view photograph.
In another preferred embodiment of the present invention, the data set of different resolution is read by imageing sensor.This analysis also can be carried out in the data set of low resolution.For this reason, have only the image data set of low resolution to be read.If on the data of having preserved, add a new data set, this data set can be read (and preservation) with high-resolution so.The processing of this scheme is very fast, because only need to analyze the small data group, the required memory space of this outer analysis is also less.
With next photo with abundant overlapping characteristic add on the existing image method just as previously discussed, but have only the high-resolution data of imageing sensor (for example " view finder frame ") to be read.Only when another photo is added up, a photo that preferably, is preferably complete image capture control (for example focus automatically, make public measurement, white balance, flash of light ...) just can be read.When taking pictures with better image picked-up control forms; Characteristic (as shadow situation etc.) according to customer requirements and/or scene; When shooting will join the free hand drawing in the distant view photograph; Consider some additional parameters of the free hand drawing that constitutes distant view photograph, for example focus automatically, make public measurement, white balance, flash of light etc.When camera system is taken, all to readjust at every turn, also can these be set at fixed value in advance to the parameter of taking these parameters.
Because, adopt short or long time of delay according to the type of camera, so will when stop camera motion to user reminding with sound and/or visual information with preferably or preferred complete image picked-up control forms when taking pictures.
In the preferred embodiment of the present invention, check motion whether to stop, just taking then with the parameter of having surveyed.
If take, show the preview of upgrading on the display screen, remind the user will continue the motion of camera with sound and/or visual information.
The following advantage of the interruption of camera motion/stop to have:
A) before stopping, the movement velocity of camera can be very fast,
B) can avoid fully and imageing sensor, camera lens and scene characteristic (for example brightness and distance) motion blur phenomenon relevant, that possibly occur.
C) can carry out taking photos by using flashlights.
Considering under the situation of some additional parameters, stop motion, and absorb control forms with better image and take pictures, have following advantage:
A) can take those and just carry out later the spliced image data; Because these image files possibly not match with main storage, and directly write on the non-volatile read-only storage (for example flash disk, hard disk ...) through the encoder element (for example being present in usually in the mobile phone of being with camera function) of a hardware supports.Afterwards, these high-resolution photos and those are drawn and transform for high-resolution photo and will be preserved by the low resolution single photo, to superpose.
B) when panning photo; Can let those equipment of the automatic pan-shot function of support can not let view data directly read through interrupting motion by imageing sensor; Can only use the view finder frame; And the equipment in a), mentioned of picture will use encoder element all the time, with the high-resolution view data of direct preservation.
C) can be by high-resolution data set; On the basis of the transformational structure that has drawn with the extra processing time; Become meticulousr to the 2D transformation result between two pictures that link to each other successively, and/or calculate again with the method that requires more computing time.
The preferred embodiment according to the present invention, as stated, low resolution image data (for example view finder frame) can splice, and calculate and preview being used for, and high-resolution view data engages in reprocessing.
In aforesaid image acquisition method; Can remind the user when should interrupt the motion of camera through a feedback signal (sound/visual information); To obtain the photo of required quality, this method can realize through analysis of image data method of the present invention fully, because according to analysis; Obtain some conclusions to the data set of imageing sensor, these conclusions are that photographic images is necessary.In one embodiment, can infer through analyzing, when taking the photo that need add, camera motion is too fast, gives the user with this indication then.Can and have prospective quality by the present speed shooting if need be bonded into the single photo of panorama photograph, then preferably not send cue.
In another preferred embodiment of the present invention, even if the motion of camera is not interrupted or when postponing to interrupt, the also shooting photo that need engage.Because possibly take low-quality photo in the case, this photo possibly transfer high-quality photo in the later stage.
In another preferred embodiment of the present invention, if interrupt too lately, the user need carry out counter motion.
Another preferred embodiment according to the present invention is being found under the problematic situation of photographic quality, and fuzzy that motion causes for example being arranged, and can require the user to do a counter motion so, so that can take the data set of a better quality again.
The big advantage of another of the inventive method is, compares with former traditional data analytical form, and not all data set of reading all needs to preserve.As long as through analysis to the data set read, show have one the sense data group need not add on the image of having preserved, so just needn't preserve it.For example; The stack of current group of read data and prestored digital image also comprises other image-region, and draws through analysis, and it is suitable that the data set that next bar is read is spliced on the image of having preserved for generating distant view photograph; In this case, can be with last data set deletion.The data set of this next bar may be preserved, only if the data set that is right after thereafter is more suitable for constituting distant view photograph than it.Through this scheme, can greatly reduce required memory space.
In another preferred embodiment of the present invention; Can be through continuously and/or with the form of live preview the progress of distant view photograph being presented at the vision output device; With the process in the assessment distant view photograph forming process, this vision output device is preferably display or screen.Such live preview is that utilization of the present invention provides support, because through such live preview, the user just can observe whole shooting process, adjusts then, like the speed of adjustment manual movement, to obtain best effect.
A kind of computer program that the data set of imageing sensor is handled of being used for; Be installed on the memory of data processing equipment at this computer program after; Make data processing equipment can realize the method that the data set of imageing sensor is handled, wherein many continuous data groups of imageing sensor at least the part be read, the data set after reading is analyzed again; Calculating in the data set of having read corresponding congruent region, and
-many continuous data sets are together with the information about corresponding congruent region, and/or
-many continuous data sets are kept in a file, and/or
-many continuous data sets are overlapped into an image when being exported by the device of data output.
This computer program can be downloaded (paying, free arbitrarily obtains or the need password) from data network or communication network.Computer program provided by the invention can be applied on the data processing equipment that is connected in the data network through a method, and is of claim 22 for this method, and computer program can be from electronic data network, for example an internet download.
In order to use the method that the data set of imageing sensor is handled of the present invention; Use the storage medium of a computer-readable; A procedure stores wherein is loaded into this program on the memory of data processing equipment the back and just can have used this data processing equipment, and the method that can use the data set to imageing sensor of this invention to handle; Many data sets quilts of imageing sensor are part read successively at least; And the data set after reading is analyzed, so that from the data set of having read, draw corresponding congruent region, and
-many continuous data sets are together with the information about corresponding congruent region, and/or
-many continuous data sets are kept in a file, and/or
-many continuous data sets are overlapped into an image when being exported by the device of data output.
For example,, when friend meets, for example in party, only need, carry out once " around candid photograph " like digital camera, mobile phone or PDA, just can all participants be taken with traditional camera through using method of the present invention.Equally, the method for the application of the invention can be carried out pan-shot to sight spot or mountain peak etc.Also can take historical sights etc. with the complete angle of freely selecting.The angle of photograph taking is not limited in 360 degree.
Not hard to imagine, the distant view photograph shooting of carrying out through method of the present invention is very easy, only need pin the shutter of photographing device (like digital camera, mobile phone etc.), shakes just passable then with the angle of needs.Be installed in software on the photographing device and can control from the reading of the data of imageing sensor, to the analysis of data set, view data is overlapped into distant view photograph, and the generation of distant view photograph does not need the user to carry out other operation again.Therefore, whole operation all is automatically, when distant view photograph is taken, or after taking, does not require that all the user carries out any operation.
Description of drawings
Below with reference to accompanying drawing embodiments of the invention are described further, wherein:
Fig. 1 is an exemplary process flow when panning photo,
Fig. 2 is to having full resolution pricture sketch map congruent region, that stored.
Embodiment
Is example at this with the digital camera, describes the present invention.But the present invention is not limited to this concrete exemplary embodiment, and the present invention can have much other execution modes, and wherein, imageing sensor can be installed on other equipment or the device.
Among the embodiment that is here set forth, imageing sensor (can be CCD chip or CMOS chip) is integrated and is installed on the digital camera.In the present embodiment, the optical element that has optical parametric is contained in this imageing sensor front side, and the optical element of present embodiment is a camera lens, can image in imageing sensor to the scene that camera is taken through this optical element.In order to realize method of the present invention, be integrated on the data processing equipment (for example processor) in the digital camera and have the operation method system.This operation method system is controlling the reading, analyze, preserve of data set, and component is to the combination of whole figure.
In this brief once more general introduction method of the present invention:
1. beginning: read first image data set, and preserve (in this enforcements, first photo can be used to the panorama photograph all the time),
2. read second data set, calculate the congruent region position (for example through 2D conversion or Lukas-Kanade algorithm) of carrying out finally stored image data set,
3. as if the threshold value of congruent region, then delete current data set, and proceed step 2, or carry out step 4 greater than regulation,
4. preserve image data set and it received on the distant view photograph through stack,
5. whether calculate panoramic memory full, perhaps by the shutter shooting that is through with, if, then: stop, otherwise proceed step 2.
In this embodiment, the data set of imageing sensor (the relating to scene) of normally being read fully.If what analyze is the lower all images data set of resolution, rather than the whole image data group, so whole flow process can be accelerated.
In reading process, as motion picture projection, individual photo can be read with very fast speed and analyze successively.This image can be data set completely (full figure), component or the convergent-divergent figure of transducer.
The data set of data processing equipment read sensor is handled this data set, is preferably to handle this data, and then reads next bar sensor data set at once.The frequency of reading images sensor data set depends on transducer, data processing equipment system, system loading, or the speed of treatment facility.
In an embodiment of the inventive method, the analysis that sensor data set is carried out comprises following step:
1. but selection operation is handled the projected image of photo, for example cylindrical projection;
2. through 2D homography matrix or projection conversion, calculate two 2D conversions between the continuous images data set, wherein, the projection conversion is to carry out with simple mode such as Lukas-Kanade algorithm;
3. with 2D conversion meeting addition always,, after reaching threshold value, just change next step over to up to having reached predetermined threshold value;
4. current images and image are before superposeed, and save the data in data storage or the read-only storage.Utilize the view data of present image just can the image data set of the distant view photograph of storing (or distant view photograph component) be widened in succession.Preferably, all images data of distant view photograph can be kept in the single file.
Several steps in the shooting process, promptly the reading, analyze and preserve of image data set can be carried out under the situation that shutter function is activated always, that is, when required image is absorbed, carry out always.
In another embodiment of this image data set processing method, last, in post-processing operation, the projection of view picture distant view photograph is handled.Preferably, generally can in the above said step 1 of being projected in of the image data set of having read, not carry out when handling, can carry out this operation.
In another embodiment of this image data set processing method; High-resolution data set is preserved preceding or afterwards; But be the conversion that is used to splice data set is carried out meticulous/recomputate handle and splicing is actual take place before, every individual data group is carried out projection operation.Preferably, especially in the above said step 1 of being projected in of the image data set of having read, do not carry out when handling, can carry out this operation.
In an embodiment of the inventive method; The result who analyzes computing is received and is used; Be preferably full automatic calculating under the situation of the optical parametric of considering camera lens (optical element); When want other partial image of saving scenario, and it is added on the distant view photograph component of having preserved.To be continued to use through analyzing the result that computing extracts, be preferably full automatic calculating, how the image data value with the image data set of existing component adds on the component of the distant view photograph of having preserved.After in the end image data set is added in the branch diagram data of distant view photograph, have the basic data group of a view picture distant view photograph.This data set of view picture distant view photograph can be under the situation of having carried out corresponding image processing, and analyze and the process of splicing in the final version of view picture distant view photograph is shown.Particularly according to more than said step 1 when the projected image of every data set that reads in of single image has been carried out processing, can use situation about saying here.
According to another embodiment, to carry out reprocessing to whole, wherein, under the situation of the optical parametric of the exercise data of considering reception and camera lens, distortion that possibly occur distant view photograph etc. is revised.If do not carry out above said optional step 1, so just can carry out this post-processing step.
In another embodiment of the inventive method, the result who the operational analysis computing to be drawn is to let the user real time preview image data splicing be the process of distant view photograph on the display of digital camera.
According to another embodiment of the invention, the method for image data set being handled is integrated in the camera function of mobile phone, PDA or digital camera.In the case; Be used for the analysis of operating and controlling image data and the software of processing and the operating system of mobile phone, PDA or digital camera and adapt, for example
operating system of single edition or many editions.Preferably, under the exposal model of equipment, open this software.Function through such is integrated, and the shooting of distant view photograph will be in the repertoire scope of camera function.In this way; Resolution carried out the function freely selected; The browser browsed of contrast valut, zoom function, and at mobile terminal device and exterior storage element (like storage card; Or other data processing equipment such as PC) between carry out the function that data shift, may be used to the shooting of distant view photograph.
In another embodiment, the method for image data set being handled and the camera function of mobile phone are superimposed.Based on this scheme, through just can begin to carry out the shooting of distant view photograph momently by joystick.First photo taking can be presented on the display or screen of mobile phone.Through about or move up and down mobile phone, photo afterwards can be folded on first photo successively.And the progress that on display or screen, can see distant view photograph formation.Along with the adding of view data, it is big that distant view photograph can become.In the present embodiment, after the shooting of distant view photograph finishes, only need the of short duration again joystick that clicks just can stop to take.But in an alternative embodiment of the invention, when reaching maximum pan-shot value, for example 360 degree can finish to take automatically.
After the shooting of distant view photograph finishes, can let photo turn at once and turn 90 degrees, become the photo of vertical direction.Whether the user can also use all functions under the mobile phone photograph pattern in addition, can carry out zoom to distant view photograph such as the user, be willing to like him with inspection shooting quality or object details wherein.
Operation when in another embodiment, panning photo can be accomplished through the button by mobile phone.For example, the function of each button can be arranged as follows:
Numerical key " 5 ":
Browse the distant view photograph of full resolution form, also promptly: the pixel of photo is equivalent to the pixel of display or screen;
Numerical key " 1 ", " 2 ", " 3 ", " 4 ", " 6 ", " 7 ", " 8 " and " 9 ":
Distant view photograph is pushed away toward a direction, and this direction is corresponding to the relative position of the button that is adopted on the keyboard with " 5 ", for example: the upper left that will on display or screen, see current distant view photograph by numerical key " 3 ";
Numerical key " 0 ":
Show the view picture distant view photograph;
Button " * " and " # ":
Zoom function progressively.
According to an alternative embodiment of the invention, not only can distant view photograph shooting level or vertical, can also process " pictorial poster form ".In the present embodiment; For example can camera elder generation level (or vertical) be moved towards a direction; Then under the prerequisite of not interrupting shooting process; Again with its in the position of skew certain altitude with opposite direction toward moved back, above or below the distant view photograph that generates for the first time, will engage the distant view photograph of follow shot for the first time like this.Moving like this can make a circulation to repeat repeatedly.
Below reprocessing is described further:
Fig. 1 a and 1b have showed the embodiment that realizes the high-resolution joint via reprocessing.
For carrying out reprocessing; All acquisition parameters (for example time for exposure, photosensitivity ISO setting, white balance, aperture, focal length etc.) all can be preserved (on a lot of equipment; It is preserved with the form of the metadata of commutative graphic file (EXIF) form, and the present embodiment method can adopt this form)
The high-resolution that has reprocessing in the present embodiment engages the mixing realized resolution, and it is not contained in the volatile memory, through using optional storage means, for example individual photo piecemeal of high-resolution is handled.
It can also be revised individual or view picture photo; As adjust contrast, brightness and white balance; This be through in preferred whole picked-up control procedure, every photo picked-up control is calculated with preserve under the processing of parameter (for example photosensitivity setting, time for exposure, aperture, white balance, color adaptation); And realize that through the processing of comparison film overlap-add region image information these photo overlap-add region are formed with the picture that absorbs the same scene fragment that parameter obtains according to the different images in should preferred whole picked-up control.
The post-exposure correction
Fig. 2 has showed the high resolution picture 1-4 that preserves, and calculate they are preferable when superimposed and before mixing, become the overlap-add region A-C of a Zhang Quanjing photo (see Fig. 1 a 100) at image.The acquisition parameters (for example time for exposure, aperture, photosensitivity/ISO adjusting, white balance etc.) in this preferred whole picked-up control that is used for photo 1-4 is all preserved; For example be kept in the EXIF metadata, it is write on the jpeg file through the autocoding element).
Zone A has defined the same image content of the acquisition parameters with photo 1 and photo 2; Below similarly the zone B and C are arranged.
According to the information of acquisition parameters, photo is superimposed in pairs by logical order, so that as far as possible little to the integral body correction of these photos.
Through the conversion value (for example contrast, brightness ...) that the parameter of correction is calculated in the analysis (for example through column analysis or other analysis mode) of two photo overlap-add region, just can revise to relative its adjacent photo of every photo.
Adjusting to different white balances between these photos also is similar to this.If in acquisition parameters to the precise information (for example actual color temperature value) of white balance, so through the analysis of overlap-add region A-C and the corrected Calculation of every composition comparison film are obtained suitable order, to keep the little correction of trying one's best of all single photo.
Claims (23)
1. the data set processing method of an imageing sensor, this method is used to generate panoramic picture, and this method provides and has been used to activate at least a method that reads said data set from least one imageing sensor; Wherein, Many continuous data groups of imageing sensor part at least are read, and when some additional data groups were read, the data set that has read was analyzed; Automatically to calculate the Qu Cheng that is superimposed in the read data group; When being used to activate the continual activation of at least a method of reading, add data set in the panoramic picture to and automatically depend on the result who is analyzed by data set and calculated, and automatically
The data of-the data set that is added are together with the information about corresponding congruent region, and/or
The data of-the data set that is added are stored in a file, and/or
-the data set that is added is overlapped into an image by the device output of data output the time.
2. method according to claim 1 is characterized in that, when activating shutter function continuously, data set is read continuously.
3. ask 1 or 2 described methods according to right peace, it is characterized in that, those data sets that when continuous, disposable activation shutter function, are read are superimposed as an image.
4. method according to claim 1 is characterized in that, in the time will joining further data set in the superimposed image, sends signal.
5. method according to claim 1 is characterized in that, carry out superimposed before; The different acquisition parameters of allotment canned data group, wherein, said allotment is through considering acquisition parameters; Analysis based on the superimposed image-region of each data set is carried out; The mode of allotment does, selects best-match order, makes that the change of all data sets is minimum.
6. method according to claim 1 is characterized in that, the analysis of read data group is comprised the 2D conversion between the adjacent successively data set of calculating.
7. method according to claim 6 is characterized in that, the calculating of said 2D conversion comprises 2D homography matrix and/or projection conversion.
8. method according to claim 1 is characterized in that, said the analysis of read data group is comprised following steps:
-calculate the line bonus image that when often will preserve by the scene of imageing sensor picked-up, and/or
-many partial images are superimposed as a scene.
9. method according to claim 1 is characterized in that, data set is carried out continuous and/or real-time analysis.
10. method according to claim 1 is characterized in that, the data set of imageing sensor is read as full figure, component or convergent-divergent figure.
11. method according to claim 1 is characterized in that, the data set of different resolution is read.
12. method according to claim 11 is characterized in that, analyzes with the data set of low resolution.
13. method according to claim 11 is characterized in that, reads the data set that will add on the said canned data group with high-resolution.
14. method according to claim 1 is characterized in that, preserves at least a portion read data group, wherein, said canned data constitutes a scene, and said scene is bigger than the scene of the data set formation of imageing sensor picked-up.
15. method according to claim 14 is characterized in that, the distortion of canned data is revised the exercise data of wherein also handling optical parametric and/or calculating.
16. method according to claim 1 is characterized in that, said imageing sensor is with respect to the scene motion that will absorb when the reading of data group.
17. method according to claim 16 is characterized in that, said motion is:
-level,
-vertical, or
-full of twists and turns form.
18. method according to claim 1 is characterized in that, to carried out Visual Display output continuously by the scene of the data exhibiting of preserving.
19. system that carries out the data set processing method of imageing sensor; This data set processing method of carrying out imageing sensor is used to generate panoramic picture; Wherein said system comprises at least one imageing sensor, at least one data processing equipment and is used to activate at least a device from least one imageing sensor reading of data group; Said system constitutes can let the many groups of data sets that link to each other of at least one imageing sensor at least partly be read, and when some additional data groups were read, the data set that has read was analyzed; Automatically to calculate the zone that is superimposed in the read data group; When being used to activate the continual activation of at least a device of reading, add data set in the panoramic picture to and automatically depend on the result who is analyzed by data set and calculated, and automatically
The data of-the data set that is added are together with the information about corresponding congruent region, and/or
The data of-the data set that is added are stored in a file, and/or
-the data set that is added is overlapped into an image by the device output of data output the time.
20. system according to claim 19 is characterized in that, said at least one imageing sensor is:
-ccd sensor (CCD=electric charge convergence device) or
-based on the transducer (CMOS=complementary metal oxide semiconductors (CMOS)) of CMOS.
21., it is characterized in that said system comprises according to claim 19 or 20 described systems:
-at least one optical element is used for the picture at least one imageing sensor formation scene
-at least one active element is used for activating to the reading of said at least one imageing sensor data set, and/or
-at least one is used for said at least one imageing sensor shot image data is carried out the device of vision output.
22. the system according to claim 19 is confused is characterized in that, said at least one imageing sensor is integrated in the mobile terminal device.
23. system according to claim 22 is characterized in that, said mobile terminal device is:
-digital camera,
-palmtop PC PDA or
-mobile phone.
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| DE200610025651 DE102006025651A1 (en) | 2006-05-29 | 2006-05-29 | Processing method for data sets of imaging sensors, involves reading multiple data sets of imaging sensor in part, and data of multiple successive data sets are combined to panoramic image during output via data output medium |
| DE102006025651.4 | 2006-05-29 | ||
| DE102007005998 | 2007-02-02 | ||
| DE102007005998.3 | 2007-02-02 | ||
| PCT/EP2007/055090 WO2007138007A1 (en) | 2006-05-29 | 2007-05-25 | Method and arrangement for processing records of imaging sensors, corresponding computer program, and corresponding computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101554043A CN101554043A (en) | 2009-10-07 |
| CN101554043B true CN101554043B (en) | 2012-12-05 |
Family
ID=38650430
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200780020045.3A Active CN101554043B (en) | 2006-05-29 | 2007-05-25 | Data packet processing method and system for image sensor |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN101554043B (en) |
| DE (1) | DE102006025651A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI444753B (en) * | 2010-11-16 | 2014-07-11 | Altek Corp | Image capturing device and adjusting method of exposure time thereof |
| CN103685952B (en) * | 2013-12-06 | 2017-07-14 | 宇龙计算机通信科技(深圳)有限公司 | Terminal and image processing method |
| JP6588462B2 (en) * | 2013-12-23 | 2019-10-09 | アールエスビーブイ，リミティド ライアビリティ カンパニー | Wide-field retinal image acquisition system and method |
| CN110493539B (en) * | 2019-08-19 | 2021-03-23 | Oppo广东移动通信有限公司 | Automatic exposure processing method, processing device and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1998034195A1 (en) * | 1997-01-30 | 1998-08-06 | Yissum Research Development Company Of The Hebrew University Of Jerusalem | Generalized panoramic mosaic |
| US6018349A (en) * | 1997-08-01 | 2000-01-25 | Microsoft Corporation | Patch-based alignment method and apparatus for construction of image mosaics |
| WO2006041443A1 (en) * | 2004-09-23 | 2006-04-20 | Agere Systems Inc. | Mobile communication device having panoramic imagemaking capability |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4577765B2 (en) * | 2004-11-02 | 2010-11-10 | Ｋｄｄｉ株式会社 | Moving image synthesizer |
-
2006
- 2006-05-29 DE DE200610025651 patent/DE102006025651A1/en not_active Ceased
-
2007
- 2007-05-25 CN CN200780020045.3A patent/CN101554043B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1998034195A1 (en) * | 1997-01-30 | 1998-08-06 | Yissum Research Development Company Of The Hebrew University Of Jerusalem | Generalized panoramic mosaic |
| US6018349A (en) * | 1997-08-01 | 2000-01-25 | Microsoft Corporation | Patch-based alignment method and apparatus for construction of image mosaics |
| WO2006041443A1 (en) * | 2004-09-23 | 2006-04-20 | Agere Systems Inc. | Mobile communication device having panoramic imagemaking capability |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101554043A (en) | 2009-10-07 |
| DE102006025651A1 (en) | 2007-12-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6471777B2 (en) | Image processing apparatus, image processing method, and program | |
| KR101341265B1 (en) | Method and arrangement for processing data sets of imaging sensors, a corresponding computer program, and a corresponding computer-readable storage medium | |
| US9013589B2 (en) | Digital image processing apparatus and digital image processing method capable of obtaining sensibility-based image | |
| US20100214445A1 (en) | Image capturing method, image capturing apparatus, and computer program | |
| US20090103778A1 (en) | Composition determining apparatus, composition determining method, and program | |
| US20140092272A1 (en) | Apparatus and method for capturing multi-focus image using continuous auto focus | |
| CN103986867A (en) | Image shooting terminal and image shooting method | |
| CN107409166A (en) | Panning lens automatically generate | |
| JP2006215266A (en) | Imaging apparatus, image storage device, imaging method, storing method and program | |
| JP2006141005A (en) | System and method for correcting image perspective | |
| CN101554043B (en) | Data packet processing method and system for image sensor | |
| KR20050109190A (en) | Wide image generating apparatus and method using a dual camera | |
| US8547454B2 (en) | Digital image photographing apparatuses and methods of controlling the same to provide location information | |
| WO2019000715A1 (en) | Method and system for processing image | |
| CN113473018B (en) | Video shooting method and device, shooting terminal and storage medium | |
| CN112640420B (en) | Control method, device, equipment and system of electronic device | |
| JP6252632B2 (en) | Imaging apparatus and program | |
| JP4923674B2 (en) | Digital camera, focus position specifying method, program | |
| JP2006019852A (en) | Image pickup device, zoom image pickup method and program | |
| CN117793534A (en) | Zoom method and device, electronic equipment and readable storage medium | |
| JP5975127B2 (en) | Imaging apparatus and program | |
| CN114760412A (en) | Method and device for acquiring full-field high-sharpness image and movable platform | |
| KR20090008560A (en) | Apparatus and method for emboding panorama function of camera in portable terminal | |
| CN116980759A (en) | Shooting method, terminal, electronic device and readable storage medium | |
| CN117177046A (en) | Shooting processing method, processor, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| ASS | Succession or assignment of patent right |
Owner name: NOKIA OYJ Free format text: FORMER OWNER: BRENNER THOMAS Effective date: 20101210 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: BERLIN, GERMANY TO: ESPOO, FINLANG |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20101210 Address after: Espoo, Finland Applicant after: NOKIA Corp. Address before: Berlin Applicant before: BIT-SIDE GmbH |
|
| C53 | Correction of patent of invention or patent application | ||
| CB03 | Change of inventor or designer information |
Inventor after: Brenner Thomas Inventor after: Battke Henrik Inventor after: Gaebler Frank Inventor before: Brenner Thomas Inventor before: Battke Henrik |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: INVENTOR; FROM: THOMAS BRENNER HENRIK BATTKE TO: THOMAS BRENNER HENRIK BATTKE GUBLER FRANCK |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: NAVTEQ LLC. Free format text: FORMER OWNER: NOKIA OY Effective date: 20130712 Owner name: NOKIA OY Free format text: FORMER OWNER: NOKIA OYJ Effective date: 20130712 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20130712 Address after: Holland HOFEN feld Patentee after: Navteq B.V. Address before: Espoo, Finland Patentee before: NOKIA Corp. Effective date of registration: 20130712 Address after: Espoo, Finland Patentee after: NOKIA Corp. Address before: Espoo, Finland Patentee before: NOKIA Corp. |
|
| C56 | Change in the name or address of the patentee | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Holland HOFEN feld Patentee after: HERE Global B.V. Address before: Holland HOFEN feld Patentee before: NAVTEQ B.V. |
|
| CP02 | Change in the address of a patent holder | ||
| CP02 | Change in the address of a patent holder |
Address after: Holland Ian Deho Finn Patentee after: HERE Global B.V. Address before: Holland HOFEN feld Patentee before: HERE Global B.V. |