WO2006006095A1

WO2006006095A1 - Method and apparatus for processing a sequence of images based on image analysis

Info

Publication number: WO2006006095A1
Application number: PCT/IB2005/052147
Authority: WO
Inventors: Estelle Lesellier; Antoine Chouly
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2004-07-07
Filing date: 2005-06-28
Publication date: 2006-01-19

Abstract

A sequence of images (I(1), .., I(N)) is processed in the following manner. A current image (I(k)) is analyzed (SPL, HIB) so as to obtain an analysis result (H(k)) for the current image. The analysis result (H(k)) for the current image is compared (PHC) with an analysis result (H(k-1)) for a previous image (I(k-1)) so as to detect whether there is a substantial difference between the respective analysis results (H(k), H(k-1)) or not. The current image (I(k)) is processed (IMP) in a manner (PP(k-i)) that is similar to the manner in accordance with which the previous image (I(k- 1)) has been processed if there is not a substantial difference between the respective analysis results (H(k), H(k- 1)). The current image (I(k)) is processed (IMP) in a different manner (PP(k), PP(k-i-j)) if there is a substantial difference between the respective analysis results (H(k), H(k-1)).

Description

Method and apparatus for processing a sequence of images based on image analysis.

FIELD OF THE INVENTION

An aspect of the invention relates to a method of processing a sequence of images. The processing may comprise, for example, a contrast enhancement based on histogram analysis. The sequence of images may originate from, for example, a low bit rate compressed video signal. Other aspects of the invention relate to a video apparatus and a computer program product for a video apparatus. The video apparatus may be, for example, a cellular phone or a personal digital assistant (PDA).

BACKGROUND OF THE INVENTION

The contrast of an image may be enhanced in the following manner. A histogram is build for the image. The histogram is a table that specifies, for each possible value of a luminance pixel in the image, the number of pixels in the image that have this value. The histogram thus indicates if the image concerns a substantially dark scene, a substantially bright scene, or a scene in between those two. The histogram further indicates if the image concerns a relatively contrasted scene or a scene with a relatively modest contrast. The histogram is used to establish a non- linear function in accordance with which the image is processed. The non-linear function is chosen so that luminance pixels are more evenly distributed throughout the range of possible values.

United States Patent published under number 2001/0052945 describes a video apparatus that comprises a histogram modifier to match luminance signals for separate pixels to prescribed values. The histogram modifier comprises a first memory with a first look-up table to correct the luminance signals. A second memory with a second look-up table is provided. The values within the second look-up table are derived from the values in the first look-up table and applied to correct color-difference signals (U and V).

SUMMARY OF THE INVENTION

According to an aspect of the invention, a sequence of images is processed in the following manner. A current image is analyzed so as to obtain an analysis result for the current image. The analysis result for the current image is compared with an analysis result for a previous image so as to detect whether there is a substantial difference between the respective analysis results or not. The current image is processed in a manner that is similar to the manner in accordance with which the previous image has been processed if there is not a substantial difference between the respective analysis results. The current image is processed in a different manner if there is a substantial difference between the respective analysis results.

The invention takes the following aspects into consideration. It may seem optimal to process images in the following manner. An optimal processing is established for each image individually. The optimal processing is established in accordance with a predefined method, or algorithm, that is applied to each image. Each image is thus processed in accordance with an optimal set of processing parameters, which has been established for that particular image. For example, conventional contrast enhancement techniques establish a nonlinear function for each image individually on the basis of one or more histograms. The image is then processed in accordance with the nonlinear function. In a sequence of images, it may occur that there are relatively many subsequent images whose respective contents differ to an appreciable extent. This may occur, for example, when the sequence of images comprises a scene wherein one or more photographs are taken with a flash. Another example is a TV news program with news flashes that repetitively interrupt a presenter scene. Further examples are scenes with one or more fast moving objects or scenes with relatively fast zoom- in or zoom-out effects. In such cases, the apparently optimal approach described hereinbefore will generally produce processing parameters that appreciably differ from one image to another image. Consequently, there will frequently be an appreciable change in the processing of one and the other image. This may lead to annoying artifacts, which degrade the image quality subjectively perceived by human beings. Stated popularly, the medicine becomes worse than the illness in such critical sequences of images.

In accordance with the aspect of the invention described hereinbefore, a current image is processed in a manner that is similar to the manner in accordance with which a previous image has been processed if there is not a substantial difference between respective analysis results for those images. The current image is processed in a different manner only if there is a substantial difference between the respective analysis results. Consequently, processing parameters will generally remain constant throughout a series of subsequent images rather than being adjusted image by image. A particular image may indeed not be optimally processed. Nonetheless, the image will generally be processed in a manner that is not far from optimal. However, artifacts due to frequent and appreciable changes in image processing are reduced. Consequently, although individual images may not be optimally processed, the overall effect will be that image quality as perceived by human beings is better than with the apparently optimal approach. Another advantage of the invention relates to the following aspects. In principle, it is possible to achieve satisfactory image quality with the following approach. A sequence of images is first analyzed before any image is processed. Accordingly, processing parameters can be established that will provide a satisfactory overall result when the images are processed. However, such an approach requires an appreciable amount of data storage capacity and processing power. Consequently, the approach will be relatively costly. In contradistinction, the invention requires a relatively modest amount of data storage capacity and processing power only. This is because it is sufficient to store the analysis results of the previous image only and, furthermore, it is sufficient to compare these analysis results with those of the current picture. Consequently, the invention allows cost efficient implementations .

These and other aspects of the invention will be described in greater detail hereinafter with reference to drawings.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram that illustrates a portable video apparatus.

FIG. 2 is a block diagram that illustrates an image improvement circuit. FIG. 3 is a functional diagram that illustrates operations that the image improvement circuit carries out.

FIG. 4 is a diagram that illustrates an image split operation. FIGS. 5A, 5B, and 5C are diagrams that illustrate a histogram comparison operation.

FIG. 6 is a conceptual diagram that illustrates a sequence of images that concern a TV news program.

FIG. 7 is a diagram that illustrates operations that the image improvement circuit carries out when processing the sequence of images.

DETAILED DESCRIPTION FIG. 1 illustrates a portable video apparatus PVA. The portable video apparatus PVA comprises a receiver REC and a display device DPL. The receiver REC comprises a printed circuit board PCB on which an image improvement circuit HC and other components are mounted. The image improvement circuit HC forms part of a signal processing chain that produces a video display signal VID in response to a received signal INP. The display device DPL displays the video display signal VID.

FIG. 2 illustrates the image improvement circuit IIC. The image improvement circuit IIC comprises an input buffer IBU, a processing circuit CPU, a program memory PMEM, a data memory DMEM, an output buffer OBU, and a bus BS, which couples the aforementioned elements to each other. The image improvement circuit IIC receives a sequence of images 1(1), .., 1(N) in which 1(1) is the first image of the sequence, 1(N) is the last image of the sequence, and N is an integer that represents the number of images comprised in the sequence. The image improvement circuit IIC processes the images so as to provide a sequence of processed images Ip(I), .., Ip(N). The processing of the images comprises various different operations. The program memory PMEM comprises a set of instructions, i.e. software, which causes the processing circuit CPU to effect these various different operations. The data memory DMEM stores intermediate results of the operations. An operation may be defined by a software module, such as, for example, a subroutine.

FIG. 3 is a functional diagram of the image improvement circuit IIC, which illustrates the operations that the image improvement circuit IIC carries out. In FIG. 3, operations, or functions, are represented as blocks. A block may thus correspond to a software module in the form of, for example, a subroutine. The various blocks will be described hereinafter as if they were functional entities for reasons of ease of description. In FIG. 3, solid lines represent transfers of data, broken lines represent control actions. FIG. 3 illustrates that the image improvement circuit IIC comprises two main functional entities: an image processor IMP and a processing controller CTRL. The processing controller CTRL comprises the following functional entities: an image splitter SPL, a histogram builder HIB, a previous histogram comparator PHC, a previous histogram- change comparator PHCC, and a parameter calculator PAC. FIG. 3 further illustrates a previous histogram memory PHM, a previous histogram-change memory PHCM, a current parameter memory CPM and a previous parameter memory PPM. These memory functions represent storage of data into the data memory DMEM, which is illustrated in FIG. 2, and retrieval of data from this physical memory. That is, each memory function illustrated as a block in FIG. 3 uses one or more memory locations, which are defined by addresses, in the data memory DMEM.

The image improvement circuit HC, which is functionally represented in FIG. 3, operates as follows. The image processor IMP sequentially processes the images comprised in the sequence of images 1(1), .., 1(N). That is, the image processor IMP processes the first image 1(1) of the sequence and, subsequently, the second image 1(2) of the sequence, and so on, until its last image 1(N). In the present description, I(k) denotes a current image, which is an image that the image improvement circuit ICC is processing at a certain instant, k being an integer value in a range comprised between 1 and N. That is, when k=l the image improvement circuit HC processes the first image 1(1); when k=N the image improvement circuit IIC processes the last image 1(N). In the present description, I(k-l) denotes the previous image, which is the image that the image improvement circuit IIC has processed just before the processing of the current image I(k).

The processing controller CTRL applies to the image processor IMP processing parameters PP(k-i) that are present in the current parameter memory CPM. The processing parameters PP(k-i) define the image processing: the manner in which the image processor IMP processes the current image I(k). The processing parameters PP(k-i) received by the image processor IMP change only when the content of the current parameter memory CPM is updated. That is, the processing parameters PP(k-i) remain constant until the current parameter memory CPM is updated. The processing parameters PP(k-i) will generally remain constant throughout the processing of a certain number of consecutive images, rather than being modified image by image.

In the present description, i denotes a variable that indicates the number of images that preceded the current image I(k) and that were processed in accordance with the same processing parameters. The variable i thus indicates when the most recent update of the current parameter memory CPM took place. The variable i may have any integer value comprised in a range between 1 and k-1.

Let it be assumed, for example, that i=3. In that case, the 3 images that preceded the current image I(k) were processed in the same manner as the current image I(k). These 3 images are I(k-l), I(k-2), and I(k-3). The current parameter memory CPM was most recently updated 3 images before the current image I(k) occurred; that is, the current parameter memory CPM was updated at image I(k-3). Let it be assumed that k=103: the current image is image 1(103). Let it further be assumed that the most recent update of the current parameter memory CPM took place at image 1(100). In that case, the variable i has indeed the value 3, i=3. The current parameter memory CPM comprises processing parameters PP(IOO). These processing parameters PP(IOO) will remain in the current parameter memory CPM until there is a new update. It should be noted that the variable k increments unit by unit while the image improvement circuit HC processes successive images: k=103, k=104, k=105, etc. Consequently, the variable i will also increment unit by unit, i=3, i=4, i=5, as long as the processing controller CTRL does not update the current parameter memory CPM: PP(IOO) remains in the current parameter memory CPM.

The following describes when and how the processing controller CTRL updates the current parameter memory CPM. It will be assumed that the image processor IMP carries out a contrast enhancement. The image processor IMP provides an output luminance pixel value in response to an input luminance pixel value in accordance with a nonlinear function. The input luminance pixel value is the value of a luminance pixel in an input image. The output luminance pixel value is the value of the corresponding luminance pixel in the corresponding processed image. The processing parameters define the characteristics of the nonlinear function, which defines the relationship between an input luminance pixel value and an output luminance pixel value. The processing parameters thus define the characteristics of the contrast enhancement.

FIG. 4 illustrates the operation of the image splitter SPL, which forms part of the processing controller CTRL. The image splitter SPL splits the current image I(k) into four image areas of substantially equal size. Accordingly, an upper left image area IA(k)(ul), an upper right image area IA(k)(ur), a lower left image area IA(k)(ll) and a lower right image area IA(k)(lr) is obtained. It will be assumed hereinafter that the size of each image is 176 pixels wide and 144 pixels high, which corresponds to a horizontal and vertical resolution, respectively. An image thus comprises 25 344 pixels. An image area comprises 6336 pixels, a quarter of the pixels comprised in the image. It will further be assumed that a luminance pixel is represented by a binary number that comprises 8 bits. There are 255 possible values that a luminance pixel may have. In FIG. 3, the reference sign IA(k) designates the 4 image areas of the current image illustrated in FIG. 4.

The histogram builder HIB builds a histogram for each image area in the current image I(k). Accordingly, the histogram builder HIB builds a histogram H(k)(ul) for the upper left image area IA(k)(ul), a histogram H(k)(ur) for the upper right image area IA(k)(ur), a histogram H(k)(ll) for the lower left image area IA(k)(ll), and a histogram H(k)(lr) for the lower right image area IA(k)(lr). The histogram for an image area is a table that specifies, for each possible value of a luminance pixel, the number of pixels in the image area that have this value. The histogram thus indicates if the relevant image area concerns a substantially dark scene, a substantially bright scene, or a scene in between those two. The histogram further indicates if the relevant image area concerns a relatively contrasted scene or a scene with a relatively modest contrast. In FIG. 3, the reference sign H(k) designates the 4 histograms for the current image.

The previous histogram comparator PHC compares the histogram of an image area with the histogram of a corresponding image area in the previous image I(k-1). Accordingly, the previous histogram comparator PHC compares the histogram H(k)(ul) with histogram H(k-l)(ul), which is the histogram for the upper left image, area of the previous image I(k-1). The previous histogram comparator PHC further compares the histogram

H(k)(ur) with histogram H(k-l)(ur), the histogram H(k)(ll) with histogram H(k- I)(Il), and the histogram H(k)(lr) with histogram H(k- I)(Ir).

The histograms H(k-l)(ul), H(k-l)(ur), H(k-l)(ll) and H(k-l)(lr) of the previous image have been stored in the previous histogram memory PHM, which is illustrated in FIG. 3. In FIG. 3, the reference sign H(k-l) designates the 4 histograms for the previous image. The histograms H(k-1) of the previous image will be replaced by the histograms H(k) of the current image after the previous histogram comparator PHC has carried out the necessary comparisons. In effect, the variable k is incremented by one unit: the current image will be the previous image when the subsequent image is processed. The histograms of the current image will thus be available for comparison with the histograms of the subsequent image, which histograms have yet to be built.

Figures 5A-5C illustrates the operation of the previous histogram comparator PHC. FIG. 5 A represents the histogram H(k)(ul) of the upper left image area in the current image. The horizontal axis represents pixel values PV, the vertical axis represents number of pixels NP. FIG. 5B represents the histogram H(k-l)(ul) of the upper left image area in the previous image. FIG. 5C illustrates a cross-section histogram HC(k, k-l)(ul). Let it be assumed that the histogram H(k)(ul) of FIG. 5A is superposed on the histogram H(k-l)(ul) of FIG. 5B. In that case, there will be a surface that the histograms have in common. This common surface, which can also be considered as an overlap, corresponds to the cross-section histogram HC(k, k-l)(ul) illustrated in FIG. 5C.

The cross-section histogram HC(k, k-l)(ul) illustrated in FIG. 5C also can be obtained in the following manner. The number of pixels NP in the histogram H(k)(ul), which is illustrated in FIG. 5 A, and the number of pixels NP in the histogram H(k-l)(ul), which is illustrated in FIG. 5B, are considered for each possible value PV of the luminance pixel. The lower of these two numbers constitutes the number of pixels NP for the value PV of the luminance pixel concerned in the cross-section histogram HC(k, k-l)(ul), which is illustrated in FIG. 5C. For example, let it be assumed that in the histogram H(k)(ul), which is illustrated in FIG. 5 A, there are 7 luminance pixels that have the value 125. Let it further be assumed that in the histogram H(k-l)(ul), which is illustrated in FIG. 5B, there are 3 luminance pixels that have the value 125. In that case, the cross-section histogram HC(k, k-l)(ul), which is illustrated in FIG. 5C, will specify the number 3 for the luminance pixel value 125.

The previous histogram comparator PHC calculates the total number of pixels present in the cross-section histogram HC(k, k-l)(ul) illustrated in FIG. 5C. The histogram H(k)(ul) of the upper left image area in the current picture can be considered similar to the histogram H(k-l)(ul) of the upper left area in the previous picture, if the total number of pixels in the cross-section histogram HC(k, k-l)(ul) is relatively high. In contradistinction, it can be considered that there is a substantial difference between the histograms H(k)(ul) and H(k-l)(ul), if the total number of pixels in the cross-section histogram HC(k, k-l)(ul) is relatively low.

For example, let it be considered that the upper left image area IA(k)(ul) in the current picture concerns a bright scene. In that case, there will be relatively many luminance pixels that have a relatively high value, whereas there will be relatively few luminance pixels that have a relatively low value. Let it further be considered that the upper left image area IA(k-l)(ul) in the previous picture concerns a dark scene. In that case there will be relatively few luminance pixels that have a relatively high value, whereas there will be relatively many luminance pixels that have a relatively low value. Consequently, there will be a relatively small overlap between the histograms H(k)(ul) and H(k-l)(ul) for the current image and the previous image, respectively. As a result, the total number of pixels in the cross-section histogram HC(k, k-l)(ul) will be relatively modest.

The previous histogram comparator PHC detects whether the total number of pixels comprised in the cross-section histogram HC(k, k-l)(ul), which is illustrated in FIG. 5C, is below a threshold value or not. It can be said that there is a local histogram change if the total number of pixels comprised in the cross-section histogram HC(k, k-l)(ul) is below the threshold value. The term local relates to the fact that the histograms concern a certain area of the image, namely the upper left image area. It can be said that there is no local histogram change if the total number of pixels comprised in the cross-section histogram HC(k, k-l)(ul) is above the threshold level or equal to the threshold level. It has been mentioned hereinbefore that an image area comprises 6336 pixels. The threshold value may be, for example, 4000 pixels, which is 63% of the total number of pixels comprised in an image area. In that case, the previous histogram comparator PHC will detect a local histogram change if the cross-section histogram HC(k, k-l)(ul) comprises less than 4000 pixels, which is the threshold value. Conversely, the previous histogram comparator PHC will detect no local histogram change if the cross-section histogram HC(k, k-l)(ul) comprises 4000 pixels or more.

The previous histogram comparator PHC establishes a cross-section histogram for each image area in the current image I(k). Accordingly, a cross-section histogram HC(k, k-l)(ur) for the upper right image area, a cross-section histogram HC(k, k- I)(Il) for the lower left image area, and a cross-section histogram HC(k, k- I)(Ir) for the lower right image area is obtained in the same manner as the cross-section histogram HC(k, k-l)(ul) for the upper left image area, which has been described hereinbefore. The previous histogram comparator PHC further detects for each of those image areas whether there is a local histogram change or not. This detection is done in the same manner as described hereinbefore with reference to the upper left image area.

The previous histogram comparator PHC counts the number of local histogram changes for the current image, which number may be comprised between 0 and 4. The previous histogram comparator PHC detects a major histogram change for the current image I(k) if there are 3 or 4 local histogram changes. Conversely, the previous histogram comparator PHC detects no major histogram change if there are less than 3 local histogram changes. In case the previous histogram comparator PHC detects 1 or 2 local histogram changes, the processing controller CTRL considers that there is a local histogram change only for the current image, but no major histogram change. In case the previous histogram comparator PHC detects no major histogram change, the content of the current parameter memory CPM remains unchanged. The image processor IMP will receive the same processing parameters for processing the current image as the image processor IMP has received for processing the previous image. Consequently, the image processor IMP will process the current image in the same manner as the previous image. The contrast enhancement for the current image is thus the same as the contrast enhancement for the previous image.

In case the previous histogram comparator PHC detects a major histogram change, the processing controller CTRL activates the previous histogram-change comparator PHCC. The previous histogram-change comparator PHCC searches for another previous image further back in the sequence of images, which produced histograms similar to the histograms of the current image. Such a search may be successful, for example, when the sequence of images comprises two different scenes that repetitively occur in an alternate fashion. An example is a dialogue between two persons whereby the camera repetitively moves from one to the other person. Another example is a TV news program with the news flashes inserted therein.

More specifically, the previous histogram-change comparator PHCC compares each of the 4 histograms H(k)(ul), H(k)(ur), H(k)(ll), and Hk(Ir) for the current image with a corresponding histogram that is stored in the previous histogram-change memory PHCM. The previous histogram-change memory PHCM contains 4 histograms H(k-i-l)(ul), H(k-i-l)(ur), H(k-i-l)(ll), and H(k-i-l)(lr). These are the histograms for the image I(k-i-l) that immediately preceded the image I(k-i) for which the previous histogram comparator PHC previously detected a major histogram change. In FIG. 3, the reference sign H(k-i-l) designates the 4 histograms for the image I(k-i-l). The previous histogram-change comparator PHCC compares the histogram

H(k)(ul) with the histogram H(k-i-l)(ul), the histogram H(k)(ur) with the histogram H(k-i- l)(ur), the histogram H(k)(ll) with the histogram H(k-i- I)(Il)₅ and the histogram H(k)(lr) with the histogram H(k-i-l)(lr). Once these comparisons have been made, the processing controller CTRL moves the 4 histograms H(k-1), which are in present in the previous histogram memory PHM, to the previous histogram-change memory PHCM. The 4 histograms H(k-1) are namely the histograms of the image I(k-l) that immediately precedes the image I(k) for which a major histogram change has just been detected.

The previous histogram-change comparator PHCC carries out histogram comparisons in a manner that is substantially similar to the previous histogram comparator PHC. That is, the previous histogram-change comparator PHCC establishes a cross-section histogram for each image area. The previous histogram-change comparator PHCC counts the number of pixels comprised in each of the 4 cross-section histograms. The previous histogram-change comparator PHCC detects whether the number of pixels is below the threshold value or not. Accordingly, for each of the 4 image areas, the previous histogram-change comparator PHCC detects whether the histogram for the current image is substantially similar to the corresponding histogram in the previous histogram-change memory PHCM or not. Let it be assumed that the respective histograms are similar for 3 or all 4 image areas. In that case, the previous histogram-change comparator PHCC establishes that there is a histogram match. The histogram match occurs when the current image I(k) has histogram characteristics that are substantially similar to those of the image I(k-i-l) that immediately preceded the image I(k-i) for which a major histogram change was previously detected. However, the previous histogram-change comparator PHCC establishes that there is no histogram match if there are less than 3 image areas that have substantially similar histograms.

The processing controller CTRL carries out the following operations when the previous histogram-change comparator PHCC establishes that there is no histogram match. In that case, the parameter calculator PAC calculates new processing parameters PP(k) on the basis of the 4 histograms H(k)(ul), H(k)(ur), H(k)(ll), and H(k)(lr) that the histogram builder HIB has built for the respective image areas in the current image I(k). The parameter calculator PAC may operate, for example, in a manner similar to what has been described in the US patent application published under number 2001/0052945.

The processing controller CTRL updates the current parameter memory CPM with the new processing parameters PP(k) that the parameter calculator PAC has calculated. The new processing parameters PP(k) will thus replace the processing parameters PP(k-i) that were present in the current parameter memory CPM. These processing parameters PP(k-i), on the basis of which the previous image was processed, will be moved to the previous parameter memory PPM. The previous parameter memory PPM thus comprises processing parameters that were applied to the image processor IMP before the most recent major histogram change. The image processor IMP will process the current image I(k) in accordance with the new processing parameters PP(k) that have been established on the basis of the histograms H(k)(ul), H(k)(ur), H(k)(ll), and H(k)(lr) for the current image. Subsequent images will be processed in the same manner as long as the previous histogram comparator PHC does not detect a major histogram change. The processing controller CTRL carries out the following operations when the previous histogram-change comparator PHCC detects a histogram match. The processing controller CTRL replaces the processing parameters PP(k-i) that are stored in the current parameter memory CPM by the processing parameters PP(k-i-j) that are stored in the previous parameter memory PPM. The latter processing parameters PP(k-i-j) are the processing parameters that the image processor IMP used to process images I(k-i-j), .., I(k-i-l). The images I(k-i-j), .., I(k-i-l) have histogram characteristics comparable to those of the current image I(k).

In the description, j is a variable that indicates the number of images between the most recently detected major histogram change and the one-but-most recently detected major histogram change. That is, at the instant of the current image I(k), i images have occurred since the most recent major histogram change and i+j images have occurred since the one-but-most recent major histogram change. For example, let it be assumed that i=5 and that j=10. In that case, a major histogram change has occurred 5 images ago and another major histogram change has occurred 15 images ago; 10 images have occurred in between those two major histogram changes without any further major histogram change. It has been noted hereinbefore that the variable k increments unit by unit while the image improvement circuit IIC processes successive images and that the variable i will equally increment unit by unit as long as the processing controller CTRL does not update the current parameter memory CPM. The variable j will also increment unit by unit, like variables k and i, as long as no major histogram change is detected.

As already explained hereinbefore, the processing controller CTRL moves the processing parameters in the current parameter memory CPM to the previous parameter memory PPM when a major histogram change has been detected. Consequently, in case the previous histogram-change comparator PHCC detects a histogram match, the processing controller CTRL will swap the respective contents of the current parameter memory CPM and the previous parameter memory PPM.

FIGS. 6 and 7 illustrate the operations described hereinbefore. FIG. 6 illustrates a sequence of images that concerns a TV news program. In FIG. 6, k is the image index number. The TV news program comprises a presenter scene PRES, a first news flash scene NWFLl, and a second news flash scene NWFL2. There is a change from the presenter scene PRES to a first news flash scene NWFLl at image 1(100), which is the first image of the first news flash scene NWFLl. Image 1(99) is the last image of the presenter scene PRES before the change to the first news flash scene NWFLl . There is a subsequent change from the first news flash scene NWFLl back to the presenter scene PRES at image 1(200), which is the first image of the return to the presenter scene PRES. Image 1(199) is the last image of the first news flash scene NWFLl. There is yet another subsequent change from the presenter scene PRES to the second news flash scene NWFL2 at image 1(250), which is the first image of the second news flash scene NWFL2. Image 1(249) is the last image of the presenter scene PRES before the change to the second news flash scene NWFL2.

FIG. 7 illustrates the respective contents of the current parameter memory CPM, the previous parameter memory PPM, the previous histogram memory PHM, and the previous histogram-change memory PHCM throughout the TV news program illustrated in FIG. 6. Like in FIG. 6, k is the image index number. The current parameter memory CPM comprises processing parameters PP(I) from image 1(1) until image 1(99), which images belong to the presenter scene PRES. The processing parameters PP(I) have been derived from the histograms of image 1(1).

The previous histogram comparator PHC detects a major histogram change at the occurrence of image 1(100), which is the first image of the first news flash scene NWFLl . It is assumed that the previous histogram-change PHCC comparator does not detect any histogram match. The parameter calculator PAC, which is illustrated in FIG. 3, derives processing parameters PP(IOO) from the histograms for image 1(100). The processing controller CTRL replaces the processing parameters PP(I) in the current parameter memory CPM by the processing parameters PP(IOO). Before doing so, the processing controller CTRL moves the processing parameters PP(I), which applied to the presenter scene PRES, to the previous parameter memory PPM. An arrow in FIG. 7 illustrates this operation. The current parameter memory CPM comprises the processing parameters PP(IOO) from image 1(100) until image 1(199). Accordingly, the first news flash scene NWFLl will be processed in accordance with those processing parameters PP(IOO), which are derived from the first image 1(100) of that scene.

At image 1(100), the previous histogram memory PHM contains histograms H(99) for image 1(99), which is the last image in the presenter scene PRES before the change to the first news flash scene NWFLl. Since there is a major histogram change, the processing controller CTRL stores the histograms H(99) in the previous histogram-change memory

PHCM. An arrow in FIG. 7 illustrates this operation. Accordingly, the histograms H(99) for the last image in the presenter scene PRES, which is image 1(99) are thus present in the previous histogram-change memory PHCM when the subsequent change from the first news flash scene NWFLl back to the presenter scene PRES occurs at image 1(200). The previous histogram comparator PHC detects a major histogram change at the occurrence of image 1(200), which is the first image of the return to the presenter scene PRES. The previous histogram-change comparator PHCC will then compare the histograms of image 1(200) with the histograms H(99) stored in the previous histogram-change memory PHCM. It is recalled that these histograms H(99) are the histograms for image 1(99), which is the last image in the presenter scene PRES before the change to the first news flash scene NWFLl occurred.

It is assumed that the previous histogram-change comparator PHCC detects a histogram match at image 1(200). That is, there are at least 3 histograms for image 1(200) that match with the corresponding histograms H(99) stored in the previous histogram-change memory PHCM. The processing controller CTRL will swap the processing parameters PP(IOO) in the current parameter memory CPM, with the processing parameters PP(I) in the previous parameter memory PPM. Two arrows in FIG. 7 illustrate this operation. The current parameter memory CPM comprises the processing parameters PP(I) from image 1(200) until image 1(249). Accordingly, the return to the presenter scene PRES will be processed in accordance with those processing parameters PP(I), which are derived from the first image 1(1) of the previous presenter scene PRES.

At image 1(200), the previous histogram memory PHM contains histograms H(199) for image 1(199), which is the last image in the presenter scene PRES before the change to the first news flash scene NWFLl . Since there is a major histogram change, the processing controller CTRL stores the histograms H(199) in the previous histogram-change memory PHCM. An arrow in FIG. 7 illustrates this operation, which is carried out once the previous histogram-change comparator PHCC has carried out the histogram match detection described in the preceding paragraph. Accordingly, the histograms H(199) for the last image in the first news flash scene NWFLl, which is image 1(199), are thus present in the previous histogram-change memory PHCM when the subsequent change from the presenter scene PRES to the second news flash scene NWFL2 occurs at image 1(250).

The previous histogram comparator PHC detects a major histogram change at the occurrence of image 1(250), which is the first image of the second news flash scene NWFL2. It is assumed that the previous histogram-change PHCC comparator does not detect any histogram match. The parameter calculator PAC, which is illustrated in FIG. 3, derives processing parameters PP(250) from the histograms for image 1(250). The processing controller CTRL replaces the processing parameters PP(I) in the current parameter memory CPM by the processing parameters PP(250). Before doing so, the processing controller CTRL moves the processing parameters PP(I), which applied to the presenter scene PRES, to the previous parameter memory PPM. An arrow in FIG. 7 illustrates this operation.

The current parameter memory CPM comprises the processing parameters PP(250) from image 1(250) until a new major histogram change is detected. Accordingly, the second news flash scene NWFL2 will be processed in accordance with those processing parameters PP(250), which are derived from the first image 1(250) of that scene. At image 1(250), the previous histogram memory PHM contains histograms H(249) for image 1(249), which is the last image in the presenter scene PRES before the change to the second news flash scene NWFL2. Since there is a major histogram change, the processing controller CTRL stores the histograms H(249) in the previous histogram-change memory PHCM. An arrow in FIG. 7 illustrates this operation.

It has been mentioned hereinbefore that the previous histogram comparator PHC may detect a local histogram change. A local histogram change occurs when the histograms of the current image and the previous image differ for 1 or 2 image areas only. A scene wherein a photograph is taken with a flash may give rise to a local histogram change, for example if the flash illuminates a part of an image only. It may happen that the previous histogram comparator PHC detects a major histogram change at the end of the flash, whereas it has detected merely a local histogram change when the flash occurred. In that case, the processing controller CTRL mistakenly concludes that there is a new scene and will update the current processing parameters accordingly. This, however, is not necessary because the scene has not changed: the image just before the flash is substantially similar to the image just after the flash.

It may therefore be advantageous that the processing controller CTRL carries out the following operations when a local histogram change is detected. The processing controller CTRL replaces the content of the previous parameter memory PPM by the content of the current parameter memory CPM. The processing controller CTRL further replaces the content of the previous histogram-change memory PHCM with the content of the previous histogram memory PHM. Accordingly, it can be avoided that the processing parameters are unnecessarily updated as a result of, for example, a photograph flash.

CONCLUDING REMARKS

The detailed description hereinbefore with reference to the drawings illustrates the following characteristics. A sequence of images (1(1), .., 1(N)) is processed in the following manner. A current image (I(k)) is analyzed (image splitter SPL and histogram builder HIB) so as to obtain an analysis result (histograms H(k)) for the current image. The analysis result for the current image is compared with an analysis result (histograms H(k-1)) for a previous image so as to detect whether there is a substantial difference between the respective analysis results (H(k), H(k-1)) or not (the previous histogram comparator PHC carries out this comparison: less than 3 local histogram changes means no substantial difference, and 3 or 4 local histogram changes means there is a substantial difference). The current image is processed in a manner that is similar to the manner in accordance with which the previous image has been processed if there is not a substantial difference between the respective analysis results (in that case, the content of current parameter memory CPM remains unchanged: the same processing parameters PP(k-i) are applied). The current image is processed in a different manner if there is a substantial difference between the respective analysis results (in that case, the content of the current parameter memory CPM is modified: different processing parameters are applied for the current picture, processing parameters PP(k) or PP(k-i-j)).

The detailed description hereinbefore further illustrates the following optional characteristics. When there is a substantial difference between the respective analysis results (histograms H(k) and H(k-1)), the analysis result for the current image (histograms H(k)) is compared with an analysis result for another previous image (histograms H(k-i-l)) so as to detect whether these respective analysis results are similar (the previous histogram-change comparator PHCC carries out this comparison). The current image (I(k)) is processed in a manner that is similar to the manner in accordance with which the other previous image (I(k-i- I)) has been processed if the respective analysis results are similar (when there is a histogram match, the current image I(k) is processed in accordance with the processing parameters PP(k- i-j) in the previous parameter memory PPM, which processing parameters PP(k-i-j) were applied to image I(k-i)). These characteristics allow further improvement of image quality because processing characteristics will change less frequently, which could otherwise cause artifacts. The detailed description hereinbefore further illustrates the following optional characteristics. When there is a substantial difference between the respective analysis results (histograms H(k) and H(k-1)), a manner is defined in accordance with which the current image (I(k)) is to be processed, the manner being defined on the basis of the analysis result of the current image (the parameter calculator PAC derives new processing parameters PP(k) from the histograms for the current image, the new processing parameters PP(k) are stored in the current parameter memory CPM, and the current image I(k) will be processed in accordance with those new processing parameters). These characteristics allow further improvement of image quality because, if processing characteristics need to change, it is preferable that this change is based on the most recent event. The detailed description hereinbefore further illustrates the following optional characteristics. The image analysis is arranged so that an analysis is carried out for various different areas of the current image (the upper left, the upper right, the lower left, and the lower right image area). An analysis for an image area results in an area analysis result (the histograms H(k)(ul), H(k)(ur), H(k)(ll), and H(k)(lr)). The difference detection is arranged so that the respective area analysis results for the current image are compared with respective corresponding area analysis results for the previous image (the previous histogram comparator PHC builds cross-section histograms HC(k, k-l)(ul), HC(k, k-l)(ur), HC(k, k-l)(U), and HC(k, k- I)(Ir)). A substantial difference is detected when the number of area analysis results that are substantially different exceeds a threshold value (the number of local histogram changes are counted; 3 or 4 local histogram changes means there is a major histogram change). These characteristics allow further improvement of image quality because the number of different areas for which the analysis is carried out constitutes a further parameter that can be used to better distinguish image content changes that require processing adaptation from image content changes that do not require processing adaptation.

The aforementioned characteristics can be implemented in numerous different manners. In order to illustrate this, some alternatives are briefly indicated. The processing of the sequence of images may comprise a color enhancement or a contour enhancement, or both: the invention can be applied to any type of image processing. There are numerous different manners to analyze an image, building histograms is merely an example. For example, a frequency spectrum analysis can be used to analyze image. Furthermore, it is not mandatory to split an image into different areas in order to analyze image, although such an approach may have advantages. For example, it is possible to build a single histogram for the entire image. In the detailed description hereinbefore, the image splitter SPL splits the image into 4 different areas. The split factor 4 has been chosen in view of the resolution of the images: 176 pixels horizontally by 144 pixels vertically. In an application with a different image resolution, a different split factor may be more suitable.

There are numerous different manners to detect whether an analysis result for an image substantially differs from an analysis result of another image, or not. It is not mandatory to detect local changes and to determine whether the number of local changes is below a threshold value or not, although such an approach may have advantages. For example, it is possible to directly compare a single histogram for an entire image with a single histogram for another entire image.

There are many different manners to process a current image differently with respect to the previous image. For example, processing parameters for the current image may be retrieved from an internal or external file. In that respect, it should be noted that it is not mandatory to search for an image that preceded the current image and that produced comparable analysis results. For example, referring to the image improvement circuit HC, which is functionally illustrated in FIG. 3, the previous histogram-change comparator PHCC may be omitted. It is possible to modify the image improvement circuit HC so that the processing controller CTRL systematically calculates new processing parameters when a major histogram change is detected rather than trying to search for past solutions.

It is understood that the present invention is not limited to the afore-mentioned embodiments, and variations or modifications may be made without departing from the spirit and scope of the invention as defined in the appended claims.

It is clear, for instance, that there are numerous ways of implementing functions by means of items of hardware or software, or both. In this respect, the drawings are very diagrammatic and represent only one possible embodiment of the invention. Thus, although a drawing shows different functions as different blocks, this does not exclude that a single item of hardware or software carries out several functions, or that an assembly of items of hardware or software or both carry out a function.

The remarks made herein before demonstrate that the detailed description with reference to the drawings, illustrates rather than limits the invention. There are numerous alternatives, which fall within the scope of the appended claims. Any reference sign in a claim should not be construed as limiting the claim. The word "comprising" does not exclude the presence of other elements or steps than those listed in a claim. The word "a" or "an" preceding an element or step does not exclude the presence of a plurality of such elements or steps.

Claims

Claims.

1. A method of processing a sequence of images (1(1), .., 1(N)), said method comprising: an image analysis step (SPL, HIB) in which a current image (I(k)) is analyzed so as to obtain an analysis result (H(k)) for the current image; a difference detection step (PHC) in which said analysis result for the current image (H(k)) is compared with an analysis result (H(k-l)) for a previous image (I(k-

I)) so as to detect whether there is a substantial difference between the respective analysis results (H(k), H(k-l)) or not; and a processing step (IMP) in which the current image (I(k)) is processed in a manner (PP(k-i)) that is similar to the manner in accordance with which the previous image (I(k-l)) has been processed if there is not a substantial difference between the respective analysis results (H(k), H(k-l)), or in which the current image (I(k)) is processed in a different manner (PP(k), PP(k-i-j)) if there is a substantial difference between the respective analysis results (H(k), H(k-l)).

2. A method as claimed in claim 1, said method comprising: a similarity detection step (PHCC), which is carried out when there is a substantial difference between the respective analysis results, in which step the analysis result for the current image (H(k)) is compared with an analysis result (H(k-i-l)) for another previous image so as to detect whether these respective analysis results (H(k), H(k-i-l)) are similar or not, the current image being processed in a manner (PP(k-i-j)) that is similar to the manner in accordance with which the other previous image has been processed if the respective analysis results (H(k), H(k-i-l)) are similar.

3. A method as claimed in claim 1, said method comprising: - a processing definition step (PAC), which is carried out when there is a substantial difference between the respective analysis results, in which step a manner is defined (PP(k)) in accordance with which the current image (I(k)) is to be processed, the manner (PP(k)) being defined on the basis of the analysis result (H(k)) of the current image

(I(k)).

4. A method as claimed in claim 1 , wherein: the image analysis step (SPL, HIB) is arranged so that an analysis is carried out for various different areas (IA(k)(ul), IA(k)(ur), IA(k)(ll), IA(k)(lr)) of the current image (I(k)), an analysis for an image area (IA(k)(ul)) resulting in an area analysis result (H(k)(ul)), and wherein the difference detection step (PHC) is arranged so that the respective area analysis results (H(k)) for the current image (I(k)) are compared with respective corresponding area analysis results (H(k-1)) for the previous image (I(k-1)), a substantial difference being detected when the number of area analysis results that are substantially different, exceeds a threshold value.

5. A method as claimed in claim 1, wherein the image analysis step (SPL, HIB) comprises a histogram building step in which a histogram of luminance pixel values of the current image is built.

6. A method as claimed in claim 6, wherein the difference detection step (PHC) comprises a histogram comparison step in which a cross-section histogram (HC(k, k-l)(ul)) is build, the cross-section histogram (HC(k, k-l)(ul)) being a cross section of a histogram (H(k)(ul)) for the current image (I(k)) and a histogram (H(k-l)(ul)) for the previous image (I(k-l)), the difference being considered as substantial if the cross-section histogram (HC(k, k-l)(ul)) comprises a number of pixels that is below a threshold value.

7. A video apparatus (PVA) for processing a sequence of images (1(1), .., 1(N)), the video apparatus comprising: - an image analyzer (SPL, HIB) for analyzing a current image (I(k)) so as to obtain an analysis result (H(k)) for the current image; a difference detector (PHC) for comparing the analysis result (H(k)) for the current image (I(k)) with an analysis result (H(k-1)) for a previous image (I(k-1)) so as to detect whether there is a substantial difference between the respective analysis results (H(k), H(k-l)) or not; and a processor (IMP) for processing the current image (I(k)) in a manner (PP(k-i)) that is similar to the manner in accordance with which the previous image (I(k-l)) has been processed if there is not a substantial difference between the respective analysis results (H(k), H(k-1)), the processor (IMP) processing the current image (I(k)) in a different manner (PP(k), PP(k-i-j)) if there is a substantial difference between the respective analysis results (H(k), HQc-I)).

8. A computer program product for a video apparatus (PVA), the computer program product comprising a set of instructions that, when loaded into the video apparatus, causes the video apparatus to carry out: an image analysis step (SPL, HIB) in which a current image (I(k)) is analyzed so as to obtain an analysis result (H(k)) for the current image; a difference detection step (PHC) in which the analysis result for the current image (HQc)) is compared with an analysis result (HQc-I)) for a previous image (IQc- I)) so as to detect whether there is a substantial difference between the respective analysis results or not; and a processing step (IMP) in which the current image (IQc)) is processed in a manner (PP(k-i)) that is similar to the manner in accordance with which the previous image (IQc-I)) has been processed if there is not a substantial difference between the respective analysis results (HQc), HQc-I)), or in which the current image (IQc)) is processed in a different manner (PPQc), PPQc-i-j)) if there is a substantial difference between the respective analysis results (HQc), HQc-I)).