WO1997031337A1

WO1997031337A1 - Method for color palette design and look-up

Info

Publication number: WO1997031337A1
Application number: PCT/US1997/002662
Authority: WO
Inventors: Karl L. Denninghoff
Original assignee: Denninghoff Karl L
Priority date: 1996-02-23
Filing date: 1997-02-21
Publication date: 1997-08-28
Also published as: AU2053297A

Abstract

A method of image quantization is provided which includes the following steps: creating an optimal palette (3) for an image using full three-dimensional deviation and capable of assigning less colors to smooth, transitional regions of the images; constructing a color look-up table (4) to efficiently map arbitrary colors to colors of the optimal palette using a sweep-list method; and mapping (5) an image to a color quantizing image using an over-compensated and adaptively damped error diffusion method.

Description

TITLE: METHOD FOR COLOR PALETTE DESIGN AND LOOK-UP

This application claims the benefit of U.S. Provisional Application No. 60/012,197, filed February 23, 1996.

BACKGROUND OF THE INVENTION

1. Field of the Invention:

This invention relates to methods for producing color images on displays and, more particularly, to such methods which use quantization of images.

2. Description of the Related Art:

In general, quantization of images suffers in quality and poor performance. The problem is sufficiently acute that high quality in 256 or fewer color images has often been obtained routinely by touching up the images using artists after some tool has performed an initial quantization. Thus, since human artistic talent is often required, the algorithmic solutions have produced poor quality quantized images regardless of speed.

Especially difficult to reproduce in quantized images have been lustrous or smooth, shiny surface effects with no contouring artifacts. Most commercial applications, such as Debabelizer, have these quality problems and, as a result, do not attempt to produce quantized images at "on-the-fly" speeds. Heretofore, no attempt has been made to address the problem of lost lustre in quantized images. While artists can usually deal with the problem of contouring artifacts by hand, they are generally unable to recreate the lustre of smooth, shiny surfaces in quantized images.

Further, the well known quantization artifact called contouring, where the human eye perceives transition lines between quantized colors that do not appear in the original image, has been dealt with by counter-productive means. The literature for contouring solutions involves attempts to repair the deficiency by adjusting the color palette selection, which causes poor color fidelity; or it uses randomness heavily in the dithering or error diffusion transformation, which further dulls lustrous surfaces in the image. However, the main reason that the contouring problem has not been solved better in the past is that existing dithering and error diffusion schemes have not been able to eliminate it without a too severe loss of resolution, i.e., the quantized images become relatively fuzzy or grainy. This usually occurs by using too much randomness in the transformation. Randomness breaks up artifact contour lines but it also breaks up features that actually appear in the original image. As a result, at least a partial remedy has in the past been sought in the palette selection step.

Solving contouring in the palette selection step involves allocating more colors to areas where contouring is most apparent, such as slowly transitioning smooth areas of an image. However, this is exactly opposite to what should be done based on human color perception characteristics. There are highly non-linear characteristics of the human eye in detecting differences in color. It increases considerably faster than the difference squared. Thus, smoothly transitioning areas of an image requires fewer colors to give the lowest overall error where the error takes into account these characteristics of human color perception. A quantization method is needed that allows the selection of color palettes solely on the basis of human color perception and which eliminates contours, maintains a sharp image, and preserves lustre.

A color bitmap for computer display typically uses far more than 256 distinct colors, but the most commonly used PC display modes are restricted to only 256 distinct simultaneously displayed colors. Thus, images are necessarily distorted when displayed on these color restricted machines. Image processing for display on such machines attempts to minimize or even eliminate the human perceivable distortion. The image that the user sees is called a quantized version of the original image. Usually between 200 and 236 of the 256 colors can be selected arbitrarily from about 16 million colors; the selection of precisely which of those 16 million colors are used to display a particular image has pronounced effects on the amount of distortion seen in the displayed image. A set of colors selected to minimize distortion for a particular image is called an optimal palet te for that image. Problems with existing optimal palette selection include slow speed and poor quality of the final image due to optimal palette selection on the basis of criteria that are neither matched well to human color perception capabilities nor to the mechanism used to map the image to a quantized version using the optimal palette.

Once a palette has been selected for the quantized image, perhaps an optimal palette for the image in question, the image must be transformed into a quantized version using that palette. The best quantized image to the human eye is usually not the image obtained by replacing each original pixel color with the nearest color from the palette. Indeed, two pixels having the same color in the original image will often have different colors in a well quantized version of the image. Dithering and error diffusion are two broad classes of schemes for performing such transformation. All such schemes only partially correct for the unavoidable color distortion in a pixel by correcting for that distortion in nearby pixels. Such schemes can only be partially effective since pixels are sufficiently large that the human eye does not always perceive the average of the nearby colors and may perceive loss of resolution or a relative fuzziness in the image. It is because of this inherent partial effectiveness that there was so much room for improvement in the perceived quality of quantized images. Problems with the quantized images include the appearance of artifacts such as contouring and dithering patterns that the human eye can easily see, lack of lustre or dulling of the image when smooth shiny surfaces are quantized, and poor perceived color fidelity.

Lustre has not been maintained in the existing dithering and error diffusion transformations because both are relatively clumsy in correcting for color error. A smooth surface has a texture that needs to be maintained. Simple attempts to correct for color error can destroy that texture and this is especially true of lustrous surfaces. To the human eye, it is often better to maintain the texture with perhaps a slight loss of color fidelity. Dithering simply averages colors on a local basis over pre-determined blocks of pixels without regard to texture and thereby dulls lustrous surfaces. Error diffusion schemes do not take into account the fact that it is very important to correct for a color error only near the error itself, i.e., existing error diffusion schemes propagate error an unlimited distance from the pixel whose quantization created the error. This results in gratuitous matting of lustrous surfaces and actually reduces the human perceived color fidelity. When randomness is used too heavily to break un contour lines, it also mattes lustrous surfaces.

It can speed up image quantization greatly to use a color look-up table to quickly identify the nearest color in the restricted palette. A color look-up table is a solution to the more general problem of finding the nearest point in a target set of points to each of many points in a multidimensional space. For colors, three dimensions are usually used. Computing nearest points, in general, can often be greatly accelerated by using a color look-up table which reduces each nearest point computation to simply indexing into an array. A color look-up table is typically an array where the indexes into the array represent a grid of points in the space and the entries in the array are the nearest points to the corresponding grid point from the target set. These are called Voronoi labeled grids after the elegant Voronoi techniques for finding nearest points. The difficulty with using such a table is the time taken to construct them. Since use of optimal palettes greatly improves image quality, each image may have its own unique optimal palette and resulting color look-up table. Thus the speed with which a color look¬ up table can be built determines a critical overhead cost for the display of each image. Such tables have been used with fewer entries than would otherwise be desired to speed construction. In this way the cost of building such tables has also affected the quality of quantized images.

One reason that constructing color look-up tables, which is equivalent to constructing a Voronoi labeled grid in three dimensions, has been so costly is that standard analysis techniques have proven to be useless in deciding the best method between a pair of methods . There are known methods for constructing such tables and other related problems that by standard analysis techniques appear to be equivalently good for the task. However, when actually coded and tested on a computer against real problems, their performance can differ by multiple orders of magnitude. Thus, the only really useful comparison technique between a pair of candidate methods is to code and test them. From a reserach perspective, this is inefficient. The best algorithm for constructing a color look-up table in the literature runs in just over 11 seconds on a SPARC1. This is much too slow for "on-the-fly" quantization of images even if all other tasks required no time.

An advantage of "on-the-fly" image quantization is that only high color resolution images need be kept with an application. The high resolution images can be quantized whenever necessary for display on the low end machines. This allows the application to serve both the high end market and low end market with the same set of images. It therefore also reduces storage space for the application and production costs for the application creators. Note that it is much better to sell applications that work in several color modes. Modern machines usually have modes that allow high color resolution and other modes, usually with higher numbers of pixels, that have limited palettes. Applications normally must work in all such modes.

Another advantage of "on-the-fly" quantization stems from the fact that scaling an image gives far better results if it is done prior to quantizing the image. Consider a slow link over which an image is to transmitted, perhaps across the internet and then across a modem connection locally. The image can be transmitted in a compressed format for full-color images but the decompressed image can be smaller than the displayed image because at the users' machine it will be scaled up to the required size. Thus, if the stretching algorithm is of sufficiently high quality, the image can be stretched or scaled after transmission and prior to quantization. Of course, this only works if the quantization method is sufficiently fast and reliable. In this way, the highest quality images can be displayed in a fraction of the time required by any other technique over slow-link internet or other slow link connections. Thus, a high quality and fast software stretching method is needed that can be applied before or, even better, during image quantization. The

advantage of intermingling stretching with \mage quantization is that speed improves and the need for full intermediate images can be eliminated to save large memory allocations. This could be accomplished by stretching only part of the image and then quantizing that part of the image, perhaps row by row of the image pixels.

Heretofore, "on-the-fly" quantization of images has not been done for still images due to either slow performance or unreliable quality of quantized images. Since both speed and image quality are important, tradeoffs have been made in the past to achieve speed at the expense of image quality. Although some investigation has been made into fast quantization for video images, they typically compromsie quality for speed by finding a way to get around the need for a color look-up table. No dithering or error diffusion transformations are performed either. As a result, the quality in video images is poor by still image standards.

SUMMARY OF THE INVENTION

It is an object of the invention to provide a method of color quantization of images.

It is another object of the invention to provide such a method which creates an optimal palette for one or more images .

It is a further object of the invention to provide a method for efficiently building a color look-up table from the optimal palette or optionally from an arbitrary palette.

It a still further object of the invention to provide such a method of error diffusion which uses the color look-up table to efficiently create an output image having minimal distortion.

These and other objects are met by providing a reliable method of color quantization of high quality images that is sufficiently fast enough for "on-the-fly" use. The method is designed so that human intervention is totally unnecessary to achieve the highest quality quantized images.

Overall, the method of color quantization of images has incorporated therein a novel method for high speed color palette selection, a novel method for creating at high speed color look-up tables, and a novel method of high speed error diffusion. The method includes provisions for the intergration of a method of stretching. More specifically, the method of image quantization includes the following steps: a) creating an optimal palette for an image using full three-dimensional deviation and capable of assigning fewer colors to smooth transitional regions of the images; b) constructing a color look-up table to efficiently map arbritrary colors to the nearest colors of the optimal palette using a sweep-list method; and, c) mapping an image to a color quantized image using an over-compensated and adaptively damped error diffusion method.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a control flow diagram of the method of color quantization with closely integrated stretching.

Fig. 2 is a control flow diagram of the method of color quantization with loosely integrated stretching.

Fig. 3 is a control flow diagram of the method of color quantization with stretching performed prior to the step of quantization.

Fig. 4 is a control flow diagram of a method for optimal palette selection.

Fig. 5 is a control flow diagram of a method for creating a color look-up table.

Fig. 6 is a control flow diagram of the sweep list method referred to in Fig. 5. Fig. 7 is a control flow diagram of a method for over- compensating and adaptively damping color error diffusion to quantize an image.

Fig. 8 is a diagram of the error proprogation pattern of the error diffusion method disclosed herein.

DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

Shown in the accompanying Figs. 1-8, there is shown a general method of color quantization. Incorporated in the general method are several novel methods for high speed color palette selection, efficiently creating a color look-up table, and high speed error diffusion.

Figs. 1-3 show control flow diagrams of the overall method of image color quantization. In each control flow diagram, a decompressed image is used. It should be understood that any image may be used in the method. Also, in each control flow diagram, an optional stretching step is also provided. Further, palettes that are not optimal may be used.

The method of image color quantization includes the following steps: a) creating an optimal palette for an image using full three-dimensional deviation and capable of assigning fewer colors to smooth transitional regions of the images steps 3, 9 and 16; b) constructing a color look-up table to efficiently map arbritrary colors to colors of said optimal palette using a sweep-list method steps 4, 10 and 17; and, c) efficiently mapping an image to a color quantized image using the color look-up table and an over-compensated and adaptively damped error diffusion method steps 5, 12 and 18.

The high speed optimal palette selection is matched both to human color perception capabilities and to the error diffusion method disclosed herein. Further, contouring is not corrected in the palette selection step. In fact, the optimal palettes selected by this method would increase contouring artifacts if used with existing dithering or error diffusion schemes. The quality of the optimal palettes selected is accomplished by making color palette selection solely on the basis of fundamental characteristics of human color perception. So in particular, this palette selection method does not use more colors for smoothly transitioning areas of an image to control contouring artifacts. This method actually does the opposite and allocates fewer colors to gently transitioning flat areas of the image than any pre¬ existing technique. However, this exacerbates and passes the problem of eliminating contour artifacts to the error diffusion method. The preferred embodiment of the optimal palette selection method uses the widely used YUV color representation scheme which is related to the standard RGB scheme by a linear transformation. A region R of a histogram of YUV colors is a three dimensional portion of the histogram defined by three ranges, one for each dimension. Thus the boundaries of a region are perpendicular to the axes of the YUV space. The entire YUV space is the largest possible region.

A region of the histogram has an associated value that is called the deviation of that region, which is determined by one of a family of formulas, the simplest of which is a three- dimentional standard statistical deviation of the color occurrences appearing in a region of the histogram. In the preferred embodiment, the deviation is a function of the sum and the sum of squares of colors in the region for each dimension as well_> as the count of pixels in the region. A pl anar cut is a region of a histogram such that for one of the dimensions Y, U, or V, all colors in the planar cut region have the same value. Where a planar cut consists of all grid points of a region with an index value of j for the dimension X, the value j is the index of that planar cut for the dimension X in that region. The following steps, as shown in Figure 4, are performed for selecting an optimal color palette of n colors: a) building a YUV histogram of colors appearing in the image to create an initial region step 20; b) selecting a region of the histogram having the maximum deviation of all the regions step 21; c) calculating for each of the primary colors Y, U, and V, the sum and the sum of the squares of each dimension of all the pixel color values for each planar cut of the region and storing them in an array for fast retrieval using the index of the planar cut, and simultaneously generating for each of the primary colors Y, U, and V, the minimum and maximum color values and the count of pixels in the planar cut and then storing them in the array step 22; d) constructing an empty high child region and an empty low child region for one of the color dimensions Y, U, or V step 27; e) adding the unused planar cut having the lowest index for this dimension to the low child region and mark that planar cut as used if the low child region has a lower deviation value than the high child region, otherwise adding the unused planar cut for this dimension having the highest index to the high child region and marking that planar cut as used steps 28, 29 and 30; f) repeating step (e) until all of the planar cuts for the dimension have been added to either the high or the low child regions step 31; g) repeating steps (d) through (f) until an optimal partition of the region is generated for each of the dimensions Y, U, and V step 32; h) selecting one of Y, U, or V for partitioning the region based on the one that has the least maximum value for the deviation of the two potential child regions from steps (d) through (f) and then partitioning the region using the child regions for this chosen component step 24; i) repeating steps (b) through (h) until there are n regions step 25; and, j ) calculating the average colors of pixels in each partition region of the histogram to obtain n YUV colors to represent the image step 26.

In step (e), the deviation value is determined by the following equ

where p is the number of pixels in the region R and if p is 0 or 1 then dev (R) is defined to be 0; W_y, W_u, and W_v are weights (to correct for perceptual non-uniformity) of each of the dimensions of the Y, U, V space respectively and in the preferred embodiment are 5, 3, and 2 respectively; r is preferrably greater than or equal to 2 and not necessarily an integer; ¥_{i f} U_i , and V are the Y, U, V values of the pixel i; and Y, U, and V are the average Y, U, V values, respectively, of the p pixels in the region R.

The formula is, without the correction factor f (p) and where r=2 , a weighted statistical standard deviation in three dimensions. Setting r to 2 is not optimal for quality of color secection but this non-optimality is substantially corrected by using an appropriate function for f. The particular deviation function preferred for efficiency and used for on-the-fly purposes is given as follows:

The correction factor function for a pixel count of p currently preferred in this formula is ln(l+4p) where In is the natural logarithm. Thus, the importance of variance increases more than the importance of large numbers of pixels in a region. Several correction factor functions were tested, notably the square root function. Functions which are the most useful as correction factors lie substantially in the region of the graph bounded by the constant function f (p) =l and the identity function f (p) = p. Since p is always integral and usually fairly small, it can effectively be evaluated using a look-up table.

By using the identity

.=0 P which can be found in most mathematical statistics texts in some form, it is possible to efficiently calculate the deviation. Notice that the right side of the equation is a function of the sum of color values, the sum of squares of

p color values, and the pixel count. The values of 2^(^υ _ι"^)²

-t=U

p and ^(^i-^^{2 are} similarly calculated.

1 * 0

Using the above described method, an optimal color palette is selected. In order to efficiently find the nearest color from the optimal palette to an arbitrary color, it is useful to use a color look-up table.

In the method disclosed herein, a color look-up table of 32K entries is constructed which is produced in under 150 milliseconds on a pentium 90 which is grossly faster than any previous method. Given that this table is used by the quantization algorithms described, the usefulness of this technique for constructing a table is determined solely by its speed. The speed is accomplished at the highest level by a divide and conquer technique to cut the problem down recursively and then by a very fast method, hereinafter called the sweep-l ist method for finding nearest points. The sweep list method finds the nearest color to a point in the three dimensional color space, then it finds the nearest color for an adjacent point. This continues in a straight line until the boundary of the grid is reached. This is called a sweep. A list of possible nearest colors is searched for each grid point. Each color in the list has a partial distance pre- calculated for it in the two dimensions whose difference is the same for all grid points in the sweep. The same list is searched for each point in the sweep but during the sweep the list gets smaller by using position information and the partial distance information to eliminate colors from the list that cannot be closest to any point yet to be processed in the sweep. The search stops for a point when the fill or three- dimensional distance to the nearest point so far found is less than the two-dimentional partial distance to the next color in the list. This works because the list is sorted on the basis of this partial distance before the first search in a sweep.

The nearest color to a grid point is a color that minimizes the formula: W_r

^C3) where W_y , W_u, W_v are weights for the dimensions of Y, U, and V, respectively, and have preferred values 5, 3, and 2 respectively, and where (cl, c2, c3) , and (gl, g2 , g3) are the color and grid point YUV vectors, respectively. This is the distance between a grid point and a color. Note that the distance function given is only used to compare between distances and no use is made of the actual differences in distance. Thus it is an optimization to use the above given formula rather than a formula that has a square root. Also, as with the color selection method previuosly given, it would give better quality to use a perceptually uniform color representation and an exponent for the differneces greater than 2 that is not necesarily an integer. However, this is the preferred embodiment for efficiency reasons.

The partial distance between a grid point and a color is determined by the above formula with only two dimensions - Y and U, Y and V, or U and V.

The steps of the color table construction are given in two separate sequences shown in Figs. 5 and 6. The general sequence is shown in Fig. 5, while the second sequence, known as the sweep-list method, is shown in Fig. 6. The divide and conquer or region partitioning technique is given only for the outermost dimension to improve clarity; but the same techniqu should be used for all but the innermost dimension. In other words, partitioning three dimensional regions with planar cuts is .described but the analogous and simpler splitting of planar cuts with line cuts is not described. A line cut partitions a planar cut just as a planar cut partitions a three dimensional region and similar speed benefits are possible. Line cuts should be used for the same reasons as planar cuts and in substantially the same way. In principle, splitting can be used for all of the dimensions in constructing such a look-up table, but a 32K cube of grid points is small enough that using the region splitting technique is of limited usefulness for the innermost dimension, especially when the innermost dimension is labeled by the highly efficient sweep-list method. The innermost dimension is called the sweep-list dimension, all other dimensions are called partition dimensions since partitioning or dividing regions is performed using planar cuts perpindicular to those dimensions. The split parameter for a region is a measure of the usefulness to the speed of the overall labeling process of the partitioning that created the region. Smaller values for the split parameter are better than larger values and values less than 1 are not possible. If the split of the parent region gave a good value for the split parameter then the child region is also split; if not then splitting of the child region is not performed. When splitting is not performed for a region it is labeled as quickly as possible using the sweep-list method. The method for creating a color look-up table includes the following steps: a) selecting a region of the grid to be labeled with Dl being the partition dimension with the greatest range in the region, D3 being the sweep-list dimension, and L being the list of target points for the region step 39; b) marking the region for partitioning and selecting a grid value (gl) of the Dl dimension by estimating the median of Dl values in the optimal palette colors of L, if the split parameter is less than 1.5 and there are at least 3 indexes of Dl in the region steps 40 and 41; c) selecting a grid value (gl) of the Dl dimension for a planar cut of the region that has not yet been labeled, if gl is yet unselected step 33; d) calculating the partial distances to the optimal palette colors of L for dimension Dl step 34; e) selecting a grid value (g2) of the region and from the D2 dimension and computing the partial distances to the optimal palette colors for the Dl and D2 dimensions combined step 35; f) sorting and copying the optimal pallette colors into a linked list on the basis of the partial distances in the two dimensions, said lists being designated as LD and LU step 45; g) assigning the median of dimension D3 values, from a few initial optimal palette colors in the linked list, to g3 step 46; h) labeling the grid points (gl, g2 , g3 ) with the nearest optimal palette color using the sweep-list method with no initial guesses and list LU for UP and list LD for DOWN sweep and assigning that nearest color to minLabel step 47; i) labeling successively the nearest optimal palette color to (gl, g2,g3+l ) , (gl ,g2 ,g3+2 ) , etc., until the boundary of the region is processed using the sweep-list method with list LU for the UP sweep and minLabel as the guess and labeling successively the nearest optimal palette color to (gl, g2 , g3-l ) , ( gl,g2 ,g3-2 ) , etc., until the boundary of the region is processed using the sweep-list method with list LD for the DOWN sweep and minLabel as the guess steps 48 and 49; j) repeat steps (e) through (i) if some grid values of D2 for gl in this region have not yet been labeled step 42; k) if the region is marked being partitioned, construct two sub-regions of the region, said sub-regions being grid points of the region on each side of the gl planar cut with each sub-region using the optimal palette colors of the parent region that are (i) on the same side of the planar cut as the grid points of the sub-region, (ii) labels of the planar cut itself, or (iii) closer to the continuous extended plane of the planar cut (from the opposite side) than any of the labels of the 4 nearest planar cut grid points step 44.

1) repeat steps (c) through (k) if all of the planar cuts of the region for Dl have not been processed step 36; m) repeat steps (a) through (1) , if all regions have not been labeled, if so, then process is complete step 37. As shown in Fig. 6, a sweep-list method for labeling an increasing index line of grid points is also provided including the following steps: a) assigning a color to minLabel which is the initial element of list L, if there is no guess, otherwise assigning the guess to minLabel step 64; b) computing the distance between g and minLabel, let minDist be the distance between g and minLabel and let nextLabel be the first optimal palette color in the list L step 64; c) label grid point g with minLabel if the minDist is less than the partial distance to the nextLabel steps 65 and 66; d) assign to nextNextLabel the optimal palette color following nextLabel in the list L step 67; e) if the distance between g and nextLabel is less than the minDist, then go to step g step 68; f) remove nextLabel from the list L, if the D3 component of minLabel is greater than or equal to the D3 component of nextLabel and g3 is greater than or equal to the D3 component of nextLabel, then go to step (h) steps 69 and 70; g) assign nextLabel to minLabel and assign the distance between g and nextLabel to minDist step 71; and, h) assign nextNextLabel to nextabel, then repeat steps (b) through (g) step 72.

To achieve the speed of the preferred embodiment, it is important to use a regular traversal of points in the higher two dimensions. Thus when a region is no longer being split, each successive sweep for labeling a line of grid points is performed on a line adjacent to the line previously labeled. When the sweep is on a line adjacent to the previous sweep and in the same region, then the sort step (f) above is performed using insertion sort starting with the sorted order used for the previous sweep; otherwise it is performed using the quicksort algorithm. While quicksort is more efficient on randomly ordered lists, insertion sort is extremely fast and much better than quicksort for only slightly out of order lists. Because there has been a change of only one unit in some dimension between the adjacent lines, the previous list is nearly in order for the next sweep.

In step (k) above, where it is tested whether a color is "closer to the continuous plane of the planar cut (from the opposite side) than any of the labels of the 4 nearest planar cut grid points", it should be understood that the continuous extended plane includes all points between the grid points as well as outside of the grid points in the plane of the planar cut. It should further be understood that a color for the region may have a component dimensional value outside of the range of the corresponding dimensional values of the grid points of the planar cut; this may occur for either or both of the two dimensions whose values are not fixed in the planar cut. If it is out of the range for one of these dimensions only, then it is compared for being closer to the continuous plane of the planar cut with only the labels of the two nearest grid points of the planar cut; if it is outside the range of both such dimensions, then it is compared for being closer to the plane than the label of the nearest grid point. To obtain the efficiency described, the partial distances for colors are kept in a sorted array and the lists are implemented as a separate array whose entries are indexes of the next element in this sorted list. A color is removed from the list simply by setting the index of the previous color in the list to the index of the next color in the list. Since the color look-up table is a YUV table and the input colors are normally in a different representation such as RGB, the YUV color look-up table is converted to a RGB color look-up table. In the preferred embodiment a 32K entry RGB to YUV look-up table is used with 6-bits for each dimension in conjunction with the YUV optimal color look-up table to construct an RGB optimal color look-up table by simply indexing into the arrays. Then the RGB color look-up table is used in the overcompensated error diffusion step described below. Using the above described method for selecting an optimal palette color including the sweep-list method it is possible to efficiently perform the following method of error diffusion which eliminates contour artifacts, maintains lustre in the image, maintains a sharp image, and is fast. It does this by over-compensating the error when a particular pixel color is chosen and by damping the quantization error as it propagates further from the pixel in which it occurs. Contouring is

- /. ' mostly handled by overco pensation but some residual contouring is dealt with by a very sparing use of randomness. The fact that the overcompensation and damping technique allows randomness to be used very sparingly preserves the sharpness and lustre of the image. The speed of this step is accomplished primarily by use of a color look-up table and by use of a table to achieve pseudo-random behavior in the over¬ compensation step with high performance. The randomness is actually simulated in a table so that the effects on the quantized image appearance are the same as true randomness but the speed of a look-up table is also achieved.

In the following error diffusion method it is assumed that the image colors are represented in standard RGB format but many other representation schemes would work as well, such as YUV. Let input stand for the unquantized image array of pixels and input ( i , j ) (R) stand for the value of the red component of the pixel at row j and column i in the image. Let output likewise stand for the quantized array of pixels that is constructed by the following steps: a) constructing two error arrays errorA and errorB having the same width as the quantized image plus one. The elements of the array are errors for each of the three color dimensions and may be either positive or negative. Initialize the error arrays to have zero entries step 51; b) assign 0 to j step 51; c) Switch errorA and errorB arrays so that the array that used to be errorB is now errorA and set the errors array to have only zero values step 52; d) If scaling is to be done, scale the next row of the image and use it in place of the input row in what follows step 53; e) assign 0 to i step 53; f) overcompensate the error that has propagated to the (i,j) pixel and add it to the input pixel value such that for each dimension X, which is one of R, G, or B, calculate the desired color as

4224errorA(i)(X) +33000 tar(X) = input(i,j)(X)+ errorA(i)(X) randQ

4224errorA(i)(X) +12000 where rand( ) is a uniformly distributed random value between

0.5 and 1.5. It is possible that tar(X) is out of range so if it is out of range adjust the value so that it is in the range 0 to 255 step 57; g) use the color look-up table for nearest palette colors to find the nearest palette color to (tar(R), tar(G) , tar(B) ) and set the color of output ( i, j ) to that nearest color step 58; h) let newerror(X)=input(i,j) (X)-ferrorA(i) (X) - output (i,j) (X) for each dimension X to calculate the difference between the target color for the pixel and the actual color assigned to that pixel step 59; i) adaptively dampen the propagated error in each dimension independently for each dimension X, if errorA(i)(X) and newerror(X) have the same sign then subtract ^■ error A(i)(X) errorA(i)(X) from newerror(X) , if this changes the sign of newerror(X) then set newerror(X) to 0 step 60; j ) propagate the error by adding three eighths of newerror(X) to each of errorA(i+1) (X) and errorB(i) (X) and one fourth of newerror(X) to errorB(i+1) (X) for each dimension X step 61; k) increment i and repeat steps (f) through (j) if there are more pixels in the row steps 62 and 63; and,

1) increment j and repeat steps (c) through (k) if there are more rows in the image steps 54 and 55.

In step (f) above there is considerable improvement in speed to be obtained by evaluating the formula using look-up tables. In the preferred embodiment this is done with two bits for the random function portion of the index and 11 bits for the errorA(i)(X) input. The table thus has 8K entries. Likewise, it is faster to combine the overcompensated error with the input value using another look-up table to obtain tar(X) .

As discussed above, the literature describes using more colors for smoothly transitioning areas of an image to control contouring artifacts. This is absolutely backwards to what the method disclosed herein does to to achieve the best image quality. This method almost always allocates fewer colors to gently transitioning flat areas of the image than any pre- existing technique. The simplest explanation for this is that the error in each quantized pixel in a smoothly transitioning region can be relatively minimal since the colors of the region are all close together. Thus, allocation of more colors to such a region necessarily reduces colors in more varied regions and increases the average error of a pixel when measured over the whole face of the image. It is an important part of the preferred palette selection method that one actually reduce the number of colors allocated to such regions.

A more mathematical justification for this is that the ease with which the human eye picks up differences in color is highly non-linear. It appears to increase considerably faster than the difference squared. Thus a smoothly transitioning region of an image should require fewer colors than existing methods provide to give the lowest overall error based on human perception. However, the human eye is also excellent at picking up contour lines so reducing the number of colors there improves the image only if the error diffusion method can eliminate contours.

The error diffusion method given here does eliminate contours. It also maintains lustre in the image and is fast. It does this by simultaneously over-compensating and adaptively damping the error as it is used and propagated.

In compliance with the statute, the invention, described herein, has been described in language more or less specific as to steps used. It should be understood, however, the invention is not limited to the specific features shown, since the means and construction shown comprised only the preferred embodiments for putting the invention into effect. The invention is, therefore, claimed in any of its forms or modifications within the legitimate and valid scope of the amended claims, appropriately interpreted in accordance with the doctrine of equivalents.

Claims

CLAIMSI claim;

1. A method of color palette selection for display of a digital image represented by a set of pixels on a display limited to n colors, comprising:

(a) representing said set of pixels as a histogram occupying a space;

(b) subdividing said space into smaller regions, each of which includes a subset of said pixels; (c) repeatedly selecting the region with the largest deviation among the included pixels and subdividing said region into two regions with roughly equal deviations, until the number of regions is equal to n; and

(d) applying a formula to the color values of the pixels within each region to calculate a single color value for each region.

2. The method of claim 1 wherein the deviation is calculated with factors of number of pixels in the region, range of the pixels, and an exponential weighting of distance of each pixel from the center.

3. A method of dithering a digital image represented by a set of pixels comprising the steps of:

(a) quantizing the color values of each pixel,

(b) for each pixel, calculating the error from the quantization and allocating the error, multiplied by a factor, to two or more adjacent pixels to be quantized, where the factor overcompensates for the error, and

(c) adaptively damping the effect of such error on pixels beyond the immediately adjacent pixel.

4. A method of image color quantization including the following steps: a) creating an optimal palette for an image using full three-dimensional deviation and capable of assigning fewer colors to smooth, transitional region of the images; b) constructing a color look-up table to efficiently map arbritrary colors to colors of said optimal palette using a sweep-list method; and, c) efficiently mapping an image to a color quantized image using an over-compensated and adaptively damped error diffusion method.

5. A method of image color quantization as recited in claim 4, wherein said method of creating an optimal color palette includes the following steps: a) building a YUV histogram of colors appearing in the image to create an initial region; b) selecting a region of the histogram having the maximum deviation of all the regions; c) calculating for each of the primary colors Y, U, and V, the sum and the sum of the squares of each dimension of all the pixel color values for each planar cut of the region and storing them in an array for fast retrieval using the index of the planar cut, and simultaneously generating for each of the primary colors Y, U, and V, the minimum and maximum color values and the count of pixels in the planar cut and then storing them in the array; d) constructing an empty high child region and an empty low child region for one of the color dimensions Y, U, or V; e) adding the unused planer cut having the lowest index for this dimension to the low child region and mark that planar cut as used if the low child region has a lower deviation value than the high child region, otherwise adding the unused planar cut for this dimension having the highest index to the high child region and marking that planar cut as used; f) repeating step (e) until all of the planar cuts for the dimension have been added to either the high or the low child regions; g) repeating steps (d) through (f) until an optimal partition of the region is generated for each of the dimensions Y, U, and V; h) selecting one of Y, U, or V for partitioning the region based on the one that has the least maximum value for the deviation of the two potential child regions from steps (d) through (f) and then partitioning the region using the child regions for this chosen component; i) repeating steps (b) through (h) until there are n regions; and, j) calculating the average colors of pixels in each partition region of the histogram to obtain n YUV colors to represent the image.

6. A method of image color quantization as recited in claim 4, wherein said method of constructing a color look-up table to efficiently map arbritrary colors to colors of said optimal palette using a sweep-list method includes the following steps: a) selecting a region to be labeled with Dl being the partition dimension with the greatest range in the region, D3 being the sweep-list dimension, and L being the list of target points for the region; b) marking the region for partitioning and selecting a grid value (gl) of the Dl dimension by estimating the median of Dl values in the optimal palette colors of L if the split parameter is less than 1.5 and there are at least 3 indexes of Dl in the region; c) selecting a grid value (gl) of the Dl dimension for a planar cut of the region that has not yet been labeled, if gl is yet unεelected; d) calculating the partial distances to the optimal palette colors of L for dimension Dl; e) selecting a grid value (g2) of the region and from the D2 dimension and computing the combined partial distances to the optimal palette colors for the Dl and D2 dimensions combined; f) sorting and copying the optimal pallet colors into a linked list on the basis of the partial distances in the two dimensions, said lists being designated as LD and LU; g) assigning the median of dimension D3 values, from a few initial optimal palette colors in the linked list, to g3; h) labeling the grid point (gl, g2, g3) with the nearest optimal palette color using the sweep-list method with no initial guesses and list LU for UP and list LD for DOWN sweep and assigning that nearest color to minLabel; i) labeling successively the nearest optimal palette color to (gl,g2,g3+l) , (gl,g2,g3+2) , etc. until the boundary of the region is processed using the sweep-list method with list LU for the UP sweep and minLabel^• as the guess and labeling successively the nearest optimal palette color to (gl,g2,g3-l) , (gl,g2,g3-2) , etc. until the boundary of the region is processed using the sweep-list method with list LD for the DOWN sweep and minLabel as the guess; j) repeat steps (e) through (i) if some grid values of D2 for gl in this region have not yet been labeled; k) if the region is being partitioned, construct two sub- regions of the region, said sub-regions being grid points of the region on each side of the gl planar cut with each sub- region using the optimal palette colors of the parent region that are (i) on the same side of the planar cut as the grid points of the sub-region, (ii) labels of the planar cut itself, or (iii) closer to the continuous plane of the planar cut (from the opposite side) than any of the labels of the 4 nearest planar cut grid points;

1) repeat steps (c) through (k) if all of the planar cuts of the region for Dl have not been processed; ) repeat steps (a) through (1), if all regions have not been labeled, if so, then process is complete.

7. A method as recited in claim 3, wherein said sweep-list method includes the following steps: a) assigning a color to minLabel which is the initial element of list L if there is no guess, otherwise assigning the guess to minLabel; b) computing the distance between g and minLabel, let minDist be the distance between g and minLabel and let nextLabel be the first optimal palette color in the list L; c) label grid point g with minLabel if the minDist is less than the partial distance to the nextLabel; d) assign to nextNextLabel the optimal palette color following nextLabel in the list L; e) if the distance between g and nextlabel is less than the minDist, then go to step g; f) remove nextLabel from the list L, if D3 component of minLabel is greater than or equal to the D3 component of nextLabel and g3 is greater than or equal to the D3 component of nextLabel, then go to step (h); g) assign nextLabel to minLabel and assign the distance between g and nextLabel to minDist; and, h) assign nextNextLabel to nextLabel, then repeat steps (b) through (g) .

8. A method of image color quantization as recited in claim 4, wherein said method of mapping an image to a color quantizing image using an over-compensated and adaptively damped error diffusion method includes the following steps: a) constructing two error arrays errorA and errorB having the same width as the quantized image plus one. The elements of the array are errors for each of the three color dimensions and may be either positive or negative. Initialize the error arrays to have zero entries; b) assign 0 to j; c) Switch errorA and errorB arrays so that the array that used to be errors is now errorA and set the errorB array to have only zero values; d) if scaling is to be done, scale the next row of the image and use it in place of the input row in what follows; e) assign 0 to i; f) overcompensate the error that has propagated to the (i,j) pixel and add it to the input pixel value such that for each dimension X, which is one of R, G, or B, calculate the desired color as 4224errorA(i)(X) + 33000 tar(X) = input{i, j)(X) + error A(i)(X randi

4224errorA(i)(X) +12000 where rand () is a uniformly distributed random value between 0.5 and 1.5. It is possible that tar ( X ) is out of range so if it is out of range adjust the value so that it is in the range 0 to 255; g) use the color look-up table for nearest palatte colors to find the nearest palette color to (tar(R) , tar(G), tar(B) ) and set the color of output (i ,j) to that nearest color; h) let newerror(X)=input(i,j) (X)- errorA(i) (X) - output(i,j) (X) for each dimension X to calculate the difference between the target color for the pixel and the actual color assigned to that pixel; i) adaptively dampen the propagated error in each dimension independently for each dimension X, if error(i) (X) and newerror(X) have the same sign then subtract — errorA(i)(X) from newerror(X) , if this changes the sign of newerror(X) then set newerror(X) to 0; ) propagate the error by adding three eights of newerror(X) to each of errorA(i+1) (X) and errorB(i) (X) and one fourth of newerror(X) to errorB(i+1) (X) for each dimension X; k) increment i and repeat step (f) if there are more pixels in the row; and, 1) increment j and repeat step (c) if there are more rows in the image.