WO2002023469A1

WO2002023469A1 - A robust and fast method for colour image size reduction with fine structure preservation

Info

Publication number: WO2002023469A1
Application number: PCT/SG2000/000174
Authority: WO
Inventors: Ruihua Ma; Vinod V. Vasudevan
Original assignee: Newstakes, Inc.
Priority date: 2000-09-13
Filing date: 2000-09-13
Publication date: 2002-03-21

Abstract

This document describes a method and a system that is concerned with reducing the size of images and preservation of fine structure in it. The method introduces the generic 'thin pixel' rather than some ad hoc thin lines. In plain words, thin pixels are those that bear critical information (e.g. stock charts) but tend to be lost, either partially or completely by conventional reduction methods. By classifying pixels into thin and non-thin pixels and holding a small scale (≤ 2), a number of simple and robust core reduction algorithms (CRA) are devised. By applying the CRAs iteratively, images can be reduced by arbitrary scales. The method handles colour images.

Description

A Robust and Fast Method for Colour Image Size Reduction with Fine

Structure Preservation FIELD OF INVENTION

The current invention applies to the general field of size reduction of images that contain fine structures. In particular, the invention relates to the field of the reduction of images that contain relatively thin lines. The present invention also has application to related fields, such as web browsing through cellular phones, videophones, PDAs, handheld PCs and the like and MPEG-7 applications. BACKGROUND ART

Image resizing is a common image processing operation that finds applications in many domains. One basic requirement for such an operation is to retain visually essential information in the original image. While magnifying an image seldom causes quality degradation, shrinking the image usually leads to loss of information. This problem becomes particularly serious and annoying with images containing mainly lines, alphanumeric characters and drawings. Examples are scanned images, faxes, maps, charts, such as those shown in Figure 1 , where lines or outlining contours become discontinuous or even lost as the image is reduced in size by conventional methods, making the image incomprehensible. This is a significant problem, particularly for financial charts, where critical information resides in free-shape fine structures. Early work addressing this problem has various other disadvantages such as handling only binary images, imposing restrictions to the types of lines and reduction scales, as well as often high computational complexity.

With the fast development of Internet and communication technology, content is increasing exponentially and is being accessed by devices with drastically different display capability, such as cellular phones, PDAs (Personal Data Assistant), hand-held PCs; basically any device other than desktop PCs, as most Internet content is being created for desktop PCs. There is thus a need for adapting (reducing) images of Web content to different display sizes suited to these different types of devices. The task is more challenging in that the images generally contain multiple colours, line objects are richer in shape and width, and processing speed is more demanding. Therefore, fast, general and robust methods need to be developed to address these problems. Existing Methods for Image Size Reduction

A simple method of image reduction is the decimation method, in which one pixel is retained every M pixels in one direction and every N pixels in another. M and N are integer reduction scales. A similar method is the nearest neighbour method in which each pixel is projected back into the original image and the pixel nearest lo the projected position is chosen as the pixel in the reduced image (see Figure 2). Still another method is the interpolation-then-thresholding method. This method computes a weighted value based on the pixels in the neighbourhood defined around the projected position and then submits the value to thresholding to obtain 1 or 0. All these methods are considered not to guarantee preservation of thin lines.

An improved reduction method is the logical OR method, in which a logical OR operation is conducted on the pixels in a neighbourhood to determine the value of the reduced pixel. However, although thin lines can be preserved in the reduced image, they sometimes result in widened lines (Figure 3). This creates image distortion.

Other prior art techniques conduct analysis in a local neighbourhood or on the entire image to obtain a parameterised representation of the lines, and use templates for thin line detection. This approach has been found to work satisfactorily with images containing only simple line types (fixed width, straight, no crossings or intersections, etc.). But this approach does not have robustness with more complex graphical objects in real-world applications. In addition, line detection is extremely computationally expensive if all the scenarios are to be retained and processed in a proper way. Often simplification is made and quality and robustness is compromised. Therefore, the methods of the prior art are considered to still have several disadvantages. The most notable of which is that they still fail to adequately reduce images containing relatively thin lines. Review of Prior Art Patents

United States Patent 5,054,099 (October 1 , 1991 ) teaches a method for overcoming loss of thin lines though the introduction of an explicit thin line detection step. The operation is conducted in a window of which the size depends on the reduction scale. It involves 2 thresholds: one for line width and one for line length. Line width threshold is implemented in such a way that any width less than the threshold is considered a valid thin line segment, in order to make the method more robust to uneven thin lines which are often seen in scanned images such as fax. One problem is its incapability of coping with free form lines. For example, there is pixel loss with thin lines at L- or T-junctions. Another problem is high computational complexity.

United States Patent 5,121 ,222 (June 9, 1992) discloses a method that detects thin lines by matching original image and a predefined set of pattern masks. The method copes only with binary images and handles only a few configurations in order to minimise computation.

United States Patents 5,138,672 and 5,309,524 teach a similar method. First the original image goes through a low-pass filter (LPF) in order to get a smoothed image. Sub-sampling is then applied to obtain a reduced image, and finally pixel values in the reduced image are corrected depending upon the configuration of the neighboring points surrounding the sampling location of each reduced pixel. Since intensity variance in the neighborhood is used for determining whether correction is needed, the method can only apply to binary images. The method is not general enough to perform reduction at any scale.

United States Patent 5,335,295 (August 2, 1994) discloses a method of magnifying-then-shrinking to achieve free-scale image reduction. The shrinking scale is a power of 2 to enable the use of a same 2x2 kernel (average) in the iterative shrinking. The magnification scale is computed from the desired scale and the shrinking scale. While this invention has the advantage of being relatively simple to implement and taking into consideration all pixels in the image, it does not handle the issue of loss of thin lines in chart images.

United States Patent 5,351 ,137 by Kato describes "logical OR'ing" in which, if a two-dimensional block of pixels (e.g., a 2x2 block for a 4-to-1 reduction) contains at least one black pixel, the pixels in the block are replaced by one black pixel. If the block contains no black pixels, the pixels are replaced with one white pixel. The image is reduced block by block. Kato also teaches the "majority" method in which a two-dimensional block is reduced to a single black pixel if the block contains a majority of black pixels; otherwise, the block is reduced to a white pixel. These methods, once again do not guarantee the preservation of these lines.

United States Patent 5,539,534 Hino, et al. (July 23, 1996) teaches a resampling position correction method. The 'thin line variation' problem refers to the case where a line wider than 2 pixels can get a width in its reduced version having different values, depending the location in the original image and the reduction scale. The invention successfully solves two problems: loss of thin lines and line width variation. It has, however, two limitations. One is the reduction scale must be less than 2, and the other is that it handles only binary images. United States Patent 5,946,416 (August 31 , 1999) describes a method for detecting pixels that are part of thin lines, but does not disclose techniques relative to whole images. The disclosure relates to an image processing device and method for detecting and removing fine lines in an image pattern, but not for preserving them. Therefore, overall it is considered that a number of problems exist with these prior art methods and apparatus, including:

• No method is capable of dealing with free-shape thin lines.

• No method is capable of performing arbitrary scale reduction while retaining robustness. • No method is capable of handling colour chart images.

The present invention seeks to overcome or ameliorate at least one problem of the prior art. SUMMARY OF INVENTION

One aspect of the present invention accordingly provides a method of reducing the size of an image to a desired size by iteratively applying appropriate reduction algorithms in order to achieve a desired size such that the reduction algorithms only reduce the image by factors between one and two, inclusive.

Another aspect of the present invention provides a method of reducing the size of an image to a desired size, including at least one of the following steps of: (a) consecutively applying a first algorithm in relation to a plurality of objective pixels to perform a reduction of the image in the horizontal and/or vertical direction by a first factor, where the first factor is an integer; (b) applying a second algorithm to a plurality of objective pixels to perform a reduction of the image in the horizontal and/or vertical direction by a second factor, where the second factor is a non-integer; and if further reduction of the image is required, reiterating steps (a) and/or (b) until the desired size is achieved. Therefore, in this aspect, an iterative reduction scheme is performed. In fact, the inventors learnt from previous work that direct arbitrary scale reduction is complex and not robust, whereas it is much easier to devise a simple and robust method for small scale reduction. Therefore, the inventors have devised a number of algorithms, called 'core reduction algorithms' (CRAs), that are capable of robust reduction for a scale of 2 or smaller. The core algorithms are applied in an iterative manner to achieve arbitrary reduction scale. In this way, a robust reduction method for arbitrary reduction scales is obtained.

The core algorithms preserve 'thin structures' through preservation of 'thin pixels'. A 'thin pixel' is defined as one that tends to be lost after reduction but needs to be preserved to maintain structural information in the image. A 'thin structure' is a group of pixels of which at least one is a thin pixel. 'Thin structure' is a concept more general than 'thin line'.

A 'thin line' is a relative concept, i.e., relative to the desired reduction scales. More specifically, any line that can pass between 2 sampling positions can be considered 'thin'. That is, the width of a thin line must not be larger than the sampling step, i.e., the reduction scale. Figure 4 shows an example where S = 1.2. The letter 'L' is 1 pixel wide. Its vertical part happens to go between the sampling positions. The body of the number '1 ' is 2 pixel wide. It will fall on 1 or 2 vertical columns of sampling positions, regardless of its position in the original image. When the scale S becomes bigger, say, S = 4 , 2- or even 3-pixel wide lines should be considered 'thin' and need appropriate processing in line detection based methods. This relativity of 'thinness' is not considered by prior inventions up to date.

Figure 5 shows some examples of thin structures, containing simple lines, line intersections (T and cross) and connecting structures, that cannot be handled properly by existing line detection based methods. The core algorithms of the present invention however, are adapted to accommodate these cases. That is, restriction on the shape of image objects is overcome via the present invention. In addition to thin structure preservation, the resulting reduction algorithms are very simple and fast, because to tell whether a pixel is thin is much simpler than detecting a line that is thin, as will be illustrated below.

It is an inherent feature that the algorithm applies not only to 1 -bit image (e.g. black and white), but also to colour images. In another aspect, the present invention provides a method of determining the background colour to an image, including the following steps: (i) determining a proportion of the image for which each colour in the image accounts; and (ii) classifying the background colour as the colour which accounts for more than half of the image. This aspect of the invention enables the background pixels to be distinguished, as thin background pixels are not subject to preservation. Thus, an overall method, background colour is determined before starting the iteration BRIEF DESCRIPTION OF THE DRAWINGS

To assist the further understanding of the invention, reference is now made to the accompanying drawings which illustrate preferred embodiments of the present invention. It is to be appreciated that these embodiments are given by way of illustration only and the invention is not to be limited by this illustration.

In the drawings:

Fig. 1 illustrates examples of images seen on the Internet. Fig. 2 illustrates a known method of image size reduction, being the nearest neighbour method using a scale of 1.6.

Fig. 3 illustrates a second known method of image size reduction, being the logical OR method.

Fig. 4 illustrates an example of a thin line. Fig. 5 illustrates some examples of thin structures.

Fig. 6 illustrates a method of pixel classification according to one embodiment of the present invention.

Fig. 7 illustrates a block diagram of a first embodiment of an image reduction method according to the invention. Fig. 8 illustrates a block diagram of a second embodiment of an image reduction method according to the invention. Fig. 9 illustrates references blocks and reference pixels according to the present invention.

Fig. 10 illustrates an example of a reduced image created using the present invention. Fig. 11 illustrates another example of a reduced image created using the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 1. General Diagram

Let S_H be the desired reduction scale in the horizontal direction and S_v the desired reduction scale in the vertical direction.

Figure 7 shows a general block diagram of a first embodiment of the image reduction method according to the present invention. It is an iterative method. First the image to reduce in size is read in. It is followed by a background colour determination step. Background Colour Determination

We assume that input chart images' background is monochrome, which holds for most of the time. The background colour is the most populous and the most widely distributed in the images. To determine the background colour of an image, we proceed as follows. First the colour histogram is computed. The bounding box or colour bin for each colour is also computed, preferably at the same time. Then the colour bins of the histogram are sorted in the ascending order. Finally, the colours are examined one by one, the first one of which the bounding box is more than half of the image size is assigned the background colour of the image.

Then the process comes to the iterative reduction using the core reduction algorithms (CRAs). The core reduction algorithms fall into two categories, according on the actual reduction scales (s_H ,s_v ) to be applied. The first category consists of 3

CRAs, performing reduction by a scale of 2 (integer): CRA2 in both horizontal and vertical directions, CRA2H only in the horizontal direction, and CRA2V only in the vertical direction. The second category also consists of 3 CRAs, performing reduction by a scale smaller than 2 but bigger than 1 (non-integer): CRA1x in both horizontal and vertical directions, CRAIxH only in the horizontal direction, and CRAIxV only in the vertical direction. See Table 1. Table 1 Core Reduction Algorithm versus reduction scales ( s_H ,s_v

Pixel Classification

According to Figure 7, after the background colour is determined, the pixels at the original image are classified as 'thin pixels' or 'non-thin pixels'. The process creates a map indicating the type of all the pixels of the input image. This operation is performed before proper reduction.

Referring now to Figure 6, for each pixel to classify, a 3x3 block is defined, with the pixel at the centre. Further, four sub-blocks, each being 2x2 and containing the centre pixel, are defined. If the centre pixel is not a foreground pixel, it is not a thin pixel. If it is a foreground pixel, in each sub-block, a counter C is established that counts the total number of pixels that have the same colour as the centre pixel to classify. The centre pixel is labelled as a 'thin pixel' if the following conditions are satisfied: • all of the 4 counters are less than 4;

• at least one of counters is not 1 .

The second condition is to prevent an isolated pixel from being classified as 'thin' and preserved over iteration. In images containing numbers such as statistical charts, however, this condition should be removed because the points in numbers bear critical information and need to be kept.

All the background pixels are not thin by default. Reference Block and Reference Pixels

Next, the objective pixels are defined. The objective pixels are the intended pixels of the reduce image, as shown by the crosses on Figure 4. Given the location (., j) of an objective pixel in the reduced image and the reduction scales s_Hand s_v , the corresponding sampling position is computed as follows

[x = s_H * (i - 1) + 1 [y = s_v Hj -ϊ) + l where we assume the origin of the image coordinates system is (1 , 1). The sampling position position (x , y ) defines a so-called 'reference block' which contains at most

4 pixels.

The shape and size of the reference block depends on the reduction scales. If s_H>{ and s_v > 1 , the reference block is a 2x2 block. If s_H =l and s_v > 1 , the reference block is a 1x2 block (vertical). If s_H>l and s_v = 1 , the reference block is a 2x1 block (horizontal). See Figure. 9. The top and/or left pixel is called 'primary reference pixel', expressed as (i_μnmαn , j_pnιmr>. )\ the rest of the pixels in a reference block are called

'secondary reference pixels', expressed as (._sec„„_rfπ , ;_sec, _nn.). (i_pnmαn , i_p,_mαry ) and computed as follows: | =W _{Eq 2}

[_J primary ~ L-' J where |_»J is a floor operator. Therefore, if both s_H and s_v are integers, so are _{re ι} and ;^' _re/., , and the primary reference pixel is the pixel at the sampling position itself. If one of s_H and s_v is not an integer, the primary reference pixel is the nearest pixel at the top-left of the sampling position. The 'reference blocK is called so because the value of the objective pixel of the reduced image depends totally on it. In other words, objective pixel determination is entirely a local operation.

The algorithm applied to the reference block to determine the value of the objective pixel depends upon whether or not the horizontal reduction scale Sβ or the vertical reduction scale s_v are non-integer. The procedure for integer reduction scales is shown in Figure 7, while the procedure for at least one non-integer reduction scale is shown in Figure 8. The Core Reduction Algorithms with Integer Reduction Scales (e.g. CRA2, CRA2H, CRA2V)

Referring to Figure 7, after the computation of the reference block and reference pixels comes the assignment of each of the objective pixel in the reduced image.

To assign a value to the objective pixel, two cases are considered separately: • If the primary reference pixel is a thin pixel, then the objective pixel is assigned the value of the primary reference pixel. Here the simple nearest- neighbor rule is applied. • If the reference pixel is not a thin pixel, then logical OR operation is performed with respect to the secondary reference pixel(s) in the reference block. In other words, if all of them are non-thin pixels, then the objective pixel is assigned the value of the primary reference pixel. If one of them is a thin pixel, however, the objective pixel is assigned the value of that secondary reference pixel (the first one in some chosen order in case of

2x2 reference block). The Core Reduction Algorithms with at Least One Non-Integer Reduction Scales (e.g. CRA1x, CRAIxH and CRAIxV)

Figure 8 shows a diagram of an image reduction method with at least one non- integer reduction scale.

Compared to the core reduction algorithms with integer reduction scales, a supplementary check is added for the case where the primary reference pixel is non- thin but one of the secondary reference pixels is thin. If the (thin) secondary reference pixel is also the primary reference pixel of the next reference block (explained below), the current objective pixel will be assigned the value of the primary reference pixel (non-thin). If it is not the case, the current objective pixel will be assigned the value of the secondary reference pixel.

If a secondary reference pixel is at the right of the primary reference pixel, the next reference block is the neighbouring reference block on the right of the current reference block. If a secondary reference pixel is below the primary reference pixel, the next reference block is the neighbouring reference block just below the current reference block. If a secondary reference pixel is diagonal to the primary reference pixel, the next reference block is the neighbouring reference block diagonal to the current reference block.

Examples of Image Reduction Results Using the Proposed Approach

Figure 10 and Figure 1 1 show examples of reduction results with images in Figure 1 , using the approach proposed in this invention.

For the chart, the reduction scales are 1.3 and 2.2, respectively. At 5 = 1.3 , only CRA1x is called. At S = 2.2 , first CRA2 is called with s = 2 and CRA1x is called with s = 1.1 . For the map of Paris, the reduction scales are 1.3 and 2.0, respectively. At S = 1.3 , only CRA1x is called. At S = 2.2 , only CRA2 is called. From Figures 10 and 1 1 it can be seen that fine structures in the images are well preserved, at integer or non-integer reduction scales.

Although preferred embodiments of the invention are described herein in detail, it will be understood by those skilled in the art that variations may be made thereto without departing from the spirit of the invention or the scope of the appended claims.

Claims

CLAIMS:

1. A method of reducing the size of an image to a desired size by iteratively applying appropriate reduction algorithms in order to achieve a desired size such that the reduction algorithms only scale the image by factors between one and two, inclusive.

2. A method of reducing the size of an image to a desired size, including at least one of the following steps of:

(a) consecutively applying a first algorithm in relation to a plurality of objective pixels to perform a reduction of the image in the horizontal and/or vertical direction by a first factor, where the first factor is an integer;

(b) consecutively applying a second algorithm to a plurality of objective pixels to perform a reduction of the image in the horizontal and/or vertical direction by a second factor, where the second factor is a non-integer; and if further reduction of the image is required, reiterating steps (a) and/or (b) until the desired size is achieved.

3. The method of claim 2 wherein the first and second factors are equal to one or two or between one and two.

4. The method of claim 2 further including the step of classifying each of the pixels of the input image as either a thin pixel or a non-thin pixel.

5. The method of claim 4 wherein the step of classifying further includes the steps of:

(A) defining a 3x3 block of pixels with the pixel to be classified being in the centre of the block;

(B) defining four 2x2 sub-blocks of pixels, each containing the centre pixel; (C) counting the number of pixels within each sub-block that are of the same type as the centre pixel; and if all of the sub-blocks have a count less than four, then classifying the centre pixel as a thin pixel.

6. The method of claim 5 wherein, in addition to all of the sub-blocks having a value less than four, at least one of the sub-blocks must not be equal to one for the centre pixel to be classified as a thin pixel.

7. The method of claim 5 wherein if the pixel to be classified is not a foreground pixel, then it is classified as a non-thin pixel.

8. The method of claim 5, wherein the pixels are of the same type if they are the same colour.

9. The method of claim 2 wherein the steps of applying a first algorithm or a second algorithm include the following steps:

(i) defining a reference block of pixels in relation to an objective pixel, such that the number of pixels in the block depends upon the reduction factors in the horizontal and vertical directions;

(ii) defining one of the pixels in the reference block as a primary reference pixel and the remaining pixels as secondary reference pixels;

(iii) if the primary reference pixel is a thin pixel, then the first objective pixel is assigned the value of the primary reference pixel;

(iv) (A) if the primary reference pixel is not a thin pixel, and the reduction factors in the horizontal and vertical directions are integers, then ascertaining whether the one or more secondary reference pixels in the reference block are non-thin pixels, and (I) if all of the secondary reference pixels are non-thin pixels, then the objective pixel is assigned the value of the primary reference pixel; or (II) if a secondary reference pixel is a thin pixel, then the objective pixel is assigned the value of the secondary reference pixel; (B) if the primary reference pixel is not a thin pixel and at least one of the reduction factors in the horizontal and vertical directions is a non-integer, then ascertaining whether the one or more secondary reference pixels in the reference block are non-thin pixels, and

(I) if all of the secondary reference pixels are non-thin pixels, then the objective pixel is assigned the value of the primary reference pixel; or

(II) if a secondary reference pixel is a thin pixel, ascertaining whether that secondary reference pixel is a primary reference pixel of the next reference block, and

(x) if the result is affirmative, the objective pixel is assigned the value of the primary reference pixel; or

(xx) if the result is negative, the current objective pixel is assigned the value of the secondary reference pixel.

(v) repeating steps (i) to (iv) for all of the objective pixels.

10. The method of claim 9 wherein the reference block is a 2x2 block of pixels if the reduction factor in both the horizontal and vertical directions is greater than 1.

1 1. The method of claim 9 wherein the reference block is a 1x2 vertical block of pixels if the reduction factor in the horizontal direction equals one and the reduction factor in the vertical direction is greater than one.

12. The method of claim 9 wherein the reference block is a 2x1 horizontal block of pixels if the reduction factor in the horizontal direction is greater than one and the reduction factor in the vertical direction equals one.

13. The method of claim 9 wherein the primary reference pixel is the upper and/or left pixel in the reference block.

14. A method of determining the background colour to an image, including the following steps:

(i) determining a proportion of the image for which each colour in the image accounts; and (ii) classifying the background colour as the colour which accounts for more than a predetermined proportion of the image.

15. The method of claim 14, wherein the proportion of the image for which each colour accounts is determined using a histogram.

16. The method of claim 14 wherein the predetermined proportion is half.

17. A method of identifying thin pixels within an image to be reduced where thin lines and structures are preserved including the steps of:

(a) defining a block of pixels with the pixel to be classified being substantially central within the block;

(b) defining a plurality of sub-blocks of pixels each containing the central pixel;

(c) counting the number of pixels within each sub-block that are of the same type as the central pixel;

(d) classifying the central pixel as a thin pixel if all of the sub-blocks have a count less than a predetermined number.

18. A computer program product for implementing any one of the methods in claims 1 to 15.