WO2002019269A2

WO2002019269A2 - A method for encoding and decoding image dependent watermarks

Info

Publication number: WO2002019269A2
Application number: PCT/CA2001/001146
Authority: WO
Inventors: Shelby Pereira
Original assignee: Shelby Pereira
Priority date: 2000-08-28
Filing date: 2001-08-14
Publication date: 2002-03-07
Also published as: WO2002019269A3

Abstract

A method for generating signal dependent wa-termarks is described. The method is based on a novel coding technique based on the use of absolute values which renders the watermark dependent on the image. Various embodiments are described in which 1 bit or a group of several bits are encoded by using a function containing one or more levels of absolute value. The de-coding is straightforward and consists of calculating the function used at embedding and comparing it against a threshold to determine if a 1 or 0 was embedded. The method is described for the particular case of images and individual video frames, but is easily applicable to any digital signal including audio signals.

Description

A method for encoding and decoding image dependent watermarks

Technical Field

The present invention relates to methods of generating and decoding watermarks in images according to the preamble of the independent claims.

Background Art

The present invention relates to methods of generating and decoding image dependent watermarks in a novel way which simultaneously addresses one or more critical problems not solved by current methods .

The idea of using a robust digital watermark to detect and trace copyright violations has stimulated significant interest among artists and publishers in recent years. Podilchuk (Podilchuk & Zeng 1998) gives three important requirements for an effective watermarking scheme: transparency, robustness and capacity. Transparency refers to the fact that we would like the watermark to be invisible. The watermark should also be robust against a variety of possible image processing attacks by pirates. These include robustness against compression such as JPEG, scaling and aspect ratio changes, rotation, cropping, row and column removal, addition of noise, filtering, cryptographic and statistical attacks, as well as insertion of other watermarks (Petitcolas & Anderson 1998) and the watermark copy attack proposed by Kutter (Kutter, Voloshynovskiy & Herrigel 2000) in which a watermark is estimated from one image and added to another one.

The third requirement is that the watermark be able to carry a certain amount of information i.e. capacity. In order to attach a unique identifier to each buyer of an image¹, a typical watermark should be able to carry at least 60-100 bits of information. However few publications deal with 60 or more bits.

Watermarking methods can be divided into two broad categories: spatial domain methods such as (Bender, Gruhl & Morimoto 1996, Pitas 1996) and transform domain methods which have for the most part focused on DCT (Podilchuk & Zeng 1998, Barni et al. 1998), DFT (Pereira & Pun 1999, Barni, Bartolini, Rosa & Piva 1999) and most recently wavelet domain methods (Podilchuk & Zeng 1998, Barni, Bartolini, Cappellini, Lippi & Piva 19999, Zhu, Xiong & Zhang 1999) . Transform domain methods have several advantages over spatial domain methods. Firstly, it has been observed that in order for watermarks to be robust, they must be inserted into the perceptually significant parts of an image. For images these are the lower frequencies which can be marked directly if a transform domain approach is adopted (Cox, Killian, Leighton & Shamoon 1996) . Secondly, since compression algorithms operate in the frequency domain (for example DCT for JPEG and wavelet for EZW) it is possible to optimize methods against compression algorithms. Thirdly, certain transforms are intrinsically robust to certain transformations. For example, the DFT domain has been successfully adopted in algorithms which attempt to recover watermarks from images which have undergone affine transformations (Pereira & Pun 1999) .

While transform domain watermarking clearly offers benefits, the problem is more challenging since it is more difficult to generate watermarks which are adapted to the human visual system (HVS) . One possibility which has recently appeared is the attempt at specifying the mask in the transform domain (Podilchuk & Zeng 1998) . Podilchuk and Zeng have accurately modeled the masking in both the wavelet and discrete cosine transform (DCT) domains where it is shown how to obtain the allowable distortion at a given coefficient as a function of all other coefficients. The existing technologies exhibit at least one of the following problems:

1. Less than 60 bits are encoded.

2. sub-optimal spatial domain modulation is applied to reduce visibility.

3. The watermark is not image dependent and in particular does not resist against the watermark copy attack which estimates the watermark from one image and adds it to another.

4. Uses an additive watermark which is easily copied, or attacked by denoising and perceptual re odu- lation as proposed by Voloshynovskiy (Voloshynovskiy, Herrigel, Baumgartner, Pereira & Pun 2000) .

5. At embedding the image is treated as noise.

It is the object of the present invention to provide a method of embedding a watermark which simultaneously is capable of dealing with the 5 stated problems.

In one aspect, the invention consists of formulating the problem as a constrained optimized problem, in which the optimization takes place over the watermarking domain with constraints on visibility posed in (possibly) another domain. Furthermore, the image is not treated as noise, but as a sequence of known values which leads to a much better performance.

In another aspect, in order to render the watermark image dependent, a coding scheme is described in which coding 1 or more bits depends explicitly on the values of one or more transform or spatial domain coefficients. Since these coefficient vary from image to image, copying of the watermark will not result in a successful detection. In fact this coding scheme renders the watermark non-additive which is essential in resisting the copy attack. The non-additive and highly adaptive nature also makes the watermark extremely robust.

In yet another aspect, it is shown how to incorporate the knowledge of JPEG quantization tables or any other quantization tables such as MPEG, LZW or others in order to render the watermark more resistant to compression.

In another aspect, we indicate how to apply the algorithm to video watermarking and music watermarking.

Disclosure of the Invention

It is the object of the present invention to provide a method of the type mentioned above that is capable of dealing with at least some, preferably all of these problems .

According to the present invention, the problem is solved by the method of the independent claims .

Preferred embodiments are described in the dependent claims .

The present method is suited for watermarking still images or video data or music signals. While the primary goal of watermarking is copyright protection, the method is also suited for other applications such as steganography where we are interested in embedding information in a medium.

Modes for Carrying Out the Invention

Formulation of preferred embodiments :

We formulate the embedding process as a constrained optimization problem. We assume that we are given an image to be watermarked denoted I. If it is an RGB image we work with the luminance component though the same methodology can be applied to other color spaces where one or more of the color componenets are being watermarked. We are also given a masking function V(J) which returns a matrix of the same size of X containing the values Δj__# j corresponding to the amount by which co- efficient (i,j₎ can be changed without being noticed. Δ can be computed in either a transform domain or the spatial domain by noise visibility functions NVF as proposed by Voloshynovskiy (Voloshynovskiy, Herrigel, Baumgaertner & Pun 1999) or other visual models such as those proposed by Osberger (Osberger, Bergmann & Maeder 1998) or Podilchuk (Podilchuk & Zeng 1998) .

In the general case, the function V can be a complex function of texture, luminance, contrast, frequency and patterns. We wish to embed a binary message m where M is the number of bits in the message. In general, the binary message may first be augmented by a checksum and/or coded using error correction codes such as BCH or turbo codes to produce a message m_c= (mι,m2...mjjt) of total length Nt- Without loss of generality we assume the image J is of size 128X 128 corresponding to a very small image. For larger images the same procedure is adopted for each 128x 128 large block.

To embed the message, we first divide the image into 8x8 blocks and calculate the DCT of each block. In each 8X8 DCT block we embed N^ bits from m_c . For each bit m-j_ we select 2 coefficients ci and C2 based on a key, in which we will embed the information bit. For better performance each coefficient in a block should only be used once so that in general it is understood that for a different πy_^ c and C2 are different. We recall that in order to ensure the watermark remains invisible, we must insist that C]_ and c2 do not change more than the allowed Δj_^j for the coefficient as determined by the function V(J) .

From this basic setup, several strategies can be adopted to embed the message. In one embodiment, we encode a 1 by maximizing |cι_C2l while to encode a 0 we minimize |c]__C2|. The key advantage arises from the use of the absolute value. The main idea is that to maximize l^cl-^c2l ^we will increase c]_ and decrease c2 if c >C2 otherwise we will increase C2 _an^ decrease ^cl. ⁱⁿ order to minimize ~i-C2 we will move el and c2 so that they are as close as possible to being equal. It is also possible to move them both towards 0 although this will typically cost more energy. Whether we maximize or minimize we note that the embedding depends on the original values cl and c2 which vary from image to image . This is the key to rendering the scheme image dependent and thereby resistant to the copy attack.

As is obvious to a person skilled in the art the maximization and minimization may be inverted at embedding. The compensation at decoding is straightforward. Once the coefficients have been modified in the DCT domain, the inverse DCT block by block is computed to obtain the watermarked image in the spatial domain.

In order to decode the watermark we simply calculate |c --C2| associated which each bit and then compare it to a threshold T. If it is larger than T we assign 1 to the bit otherwise we assign 0. The error correction codes are then decoded and the checksum tested if necessary. In a superior embodiment, rather than assigning 1 and 0 after comparison to a threshold (known as hard decoding) we can retain the values |c]__C2|-T and use them directly in the soft decoding of error correction codes which may yield a gain of more than 3dB in some cases . Typically the threshold T is calculated empirically by testing the algorithm and choosing a T which yields the best performance.

In the above simple embodiments we have used the function |cι_C2| for embedding. In other embodiments functions of the form |f(cι_ΛC2 03,04...)! may be interesting. In particular, rather than using just a difference of two coefficients, we may calculate a linear combination of several coefficients prior to taking the absolute value. Another possibility is to multiply coefficients. Yet another embodiment consists of using several levels of absolute values . In other words , we would calculate I I C]_ I _ I C2 I I or the absolute value of a linear combination of absolute values in general. In all cases the principle used for decoding remains the same. That is we calculate the function used at embedding and then use soft or hard decoding of error correction codes . Preferred embodiments would use turbo codes which approach optimal performance in Gaussian channels.

In the above embodiments we have chosen to encode each bit separately by modulating a set of coefficients as determined by a function. In a fundamentally different embodiment we can encode several bits by the maximization and minimizations of functions of the form |f (c₁,c₂,c_3/c₄...) I .

Table 1: Encoding of multiple bits

Bits associated function

00 |ci-C2|

01 |c₃_c |

10 ]c5_cg|

11 |c7_c₈|

As an example of the encoding scheme, table 1 associates pairs of bits with a given function which in this embodiment is just the absolute value of the difference between 2 coefficients . If we would like to encode 00 we would maximize \ c -C2 \ and minimize | 03-04), \ cζ- Cζ I and | 07.03 |. Clearly in other embodiments we may use more general functions of the form | f(cι 02,03 C | as described previously. The decoding for this embedding strategy is straightforward. We must simply calculate the associated functions and choose the bit pair which corresponds to the maximum. We also note that more than two bits can be used however the number of functions and therefore the encoding and decoding complexity required goes up exponentially. In other embodiments Reed-Solomon codes or low density parity check codes (LDPC) are used when bits are grouped together at embedding since these tend to perform better in these situations .

In what has been described, the DCT domain has been used, however any other domain may also be used for the embedding. In the case of spatial domain embedding, typically the local mean would first be removed. Although this is not necessary, if we do this, we obtain coefficients whose expected value is 0. This is the case for most transform domains since the mean is represented by one coefficient in a given block. In the case of the wavelet domain, the coefficients representing the local means, are contained in the lowest subband. Using zero mean coefficients considerably simplifies the embedding since the mean must no longer be accounted for at decoding.

We note that it is now well known that a synchronization pattern can be added to the watermark. At decoding the synchronizing pattern is searched for. If the image has undergone geometrical transformations, these are compensated for and then the watermark is decoded. An example of a synchronization pattern commonly used is a set of peaks in the Fourier transform domain as done in (Pereira & Pun 1999) . Consequently in all the embodiments, it is to be understood that synchronization patterns can be done with little or no effect on the watermark itself since the energy used in the pattern is typically much less than the energy of the watermark.

While there are shown and described presently preferred embodiments of the invention, it is to be distinctly understood that the invention is not limited thereto but may be otherwise variously embodied and practiced within the scope of the following claims . References :

Barni, M., Bartolini, F., Cappellini, V., Lippi, A. & Piva, A. (1999), A DWT-based technique for spatio-frequency masking of digital signatures, in P. W. Wong & E. J. Delp, eds, 'Security and Watermarking of Multimedia Contents', Vol. 3657, The Society for Imaging Science and Technology (IS&T) and the International Society for Optical Engineering (SPIE), SPIE, San Jose, California, U.S.A., pp. 31-39. *http://lci.die.unifi.it/Pub-ications/swmc99-04.ps.gz

Barni, M., Bartolini, F., Cappellini, V., Piva, A. & Rigacci, F. (1998), A M.A.P. identification criterion for DCT-based watermarking, in 'EUSIPCO'98', Rhodes, Greece.

Barni, M., Bartolini, F., Rosa, A. D. & Piva, A. (1999), Capacity of the watermark- channel: How many bits can be hidden within a digital image ?, in 'SPIE: Security and Watermarking of Multimedia Contents', Vol. 3657, San Jose, California, pp. 437-448.

Bender, W., Gruhl, D. & Morimoto, N. (1996), 'Method and apparatus for data hiding in images', U.S. Patent # 5689587 .

Cox, I. J., Miller, M. L. & McKellips, A. L. (1999), 'Watermarking as communications with side information', Proceedings of the IEEE 87(7), 1127— 1141.

Cox, I., Killian, J., Leighton, T. & Shamoon., T. (1996), Secure spread spectrum watermarking for images, audio and video., in 'Proceedings of the IEEE Int. Conf. on Image Processing ICIP-96', Lausanne, Switzerland, pp. 243-246.

Kutter, M., Voloshynovskiy, S. & Herrigel, A. (2000), Watermark copy attack, in P. Wah Wong & E. J. Delp, eds, 'IS&T/SPIE's 12th Annual Symposium, Electronic Imaging 2000: Security and Watermarking of Multimedia Content IP, Vol. 3971 of SPIE Proceedings, San Jose, California USA.

Osberger, W., Bergmann, N. & Maeder, A. J. (1998), An automatic image quality assessment technique incorporating higher level perceptual factors, in 'IEEE ICIP-98', Chicago,USA.

Pereira, S. & Pun, T. (1999), Fast robust template matching for affine resistant watermarks, in '3rd International Information Hiding Workshop', Dreseden, Germany.

Pereira, S. & Pun, T. (2000), A framework for optimal adaptive dct watermarks using linear programming, in 'Tenth European Signal Processing Conference (EUSIPCO'2000)', Tampere, Finland.

Petitcolas, F. A. P. & Anderson, R. J. (1998), Attacks on copyright marking systems, in '2nd International Information Hiding Workshop', Portland, Oregon, USA, pp. 219-239. Pitas, I. (1996), A method for signature casting on digital images, in 'Proceedings of the IEEE Int. Conf. on Image Processing ICIP-96', Lausanne, Switzerland, pp. 215-218.

Podilchuk, C. I. & Zeng, W. (1998), 'Image-adaptive watermarking using visual models', IEEE Journal on Selected Areas in Communications 16(4), 525-539.

Swanson. M. D., Zhu, B. & Tewfik, A. (1996), Robust data hiding for images, in '7th IEEE Digital Signal Processing Workshop', Loen, Norway, pp. 37-40. G-.WM1-A23.

Swanson, M. D., Zhu, B. k Tewfik, A. H. (1998), 'Multiresolution scene-based video watermarking using perceptual models', IEEE Journal on Selected Areas in Communications 16(4), 540-550.

Vetterli, M. & Kovacevic., J. (1995), Wavelets and Subband Coding, Prentice Hall.

Voloshynovskiy, S., Herrigel, A., Baumgaertner, N. & Pun, T. (1999), A stochastic approach to content adaptive digital image watermarking, in 'Third International Workshop on Information Hiding', Dresden, Germany.

Voloshynovskiy, S., Herrigel, A., Baumgartner, N., Pereira, S. & Pun, T. (2000), A generalized watermark attack based on stochastic watermark estimation and perceptual remodulation, in P. Wah Wong & E. J. Delp, eds, 'IS&T/SPIE's 12th Annual Symposium, Electronic Imaging 2000: Security and Watermarking of Multimedia Content II', Vol. 3971 of SPIE Proceedings, San Jose, California USA. (Paper El 3971-34).

Voyatzis, G. & Pitas, I. (1999), 'The use of watermarks in the protection of digital multimedia products', Proceedings of the IEEE 87(7).

Zhu, W., Xiong, Z. & Zhang, Y. Q. (1999), 'Multiresolution watermarking for images and video', IEEE Transactions on Circuits and Systems for Video Technology 9(4), 545-550.

Claims

1. A method for embedding a watermark in a still or non-still image or music signal or other digital signal I comprising the steps of: selecting coefficients CJ_ from a set c = {ci, ... c^} of coefficients, wherein said set of coefficients corresponds to said signal I or a transform thereof, and wherein said coefficients c^ are selected to be used for encoding a message m_c or a message derived from m_c by use of error correction codes, and modifying each of said selected coefficients CJ_ to encode one or more bits of said message by minimizing or maximizing functions of the form |f (01,02,03,04._,.) I .

2. The method of claim 1 wherein said function f is of the form |kl*cl+k2*c2+k3*c3+...kn*cn| where the values k are constants and at least 2 coefficients kl and k2 are non zero.

3. The method of claim 2 where kl is 1 and k2 is -1.

4. The method of claim 1 where the function is |kl I ci I -k2 I C2 I I where the values k are constants.

5. The method of one of the preceding claims applied to the spatial domain, where the local mean has first been removed.

6. The method of one of the preceding claims wherein said coefficients CJ_ are constrained by values Δf, j as determined by a masking function of the data set V(J) .

7. The method of claim 6 where bits are first grouped together and each possible bit group is assigned a function and the function corresponding to the bit group to be embedded is maximized while the others are minimized.

8. The method of claim 7 where Reed-Solomon codes or LDPC codes are used to encode the original message prior to embedding.

9. A method for decoding the watermark generated by the method of one of the preceding claims (except claims 7 and 8) comprising the step of calculating the function | f (c , C2 , C3 , 04...) | and comparing it to a threshold value T, and assigning a value of 1 if greater than T and 0 otherwise and then decoding error correction codes if necessary.

10. The method of claim 9 where soft decoding is used based on the difference |f(cι 02,03 C4._,.) | - T.

11. A method for decoding a watermark generated by the method of claims 7 or 8 consisting of calculating all the functions used at embedding and choosing the bit sequence associated with the function yielding the maximum value, and decoding the error correction codes if necessary after the entire bit sequence has been retrieved.

12. A method for watermarking video data comprising a plurality of consecutive video frames, wherein the method of one of the claims 1-11 is applied to at least some of said video frames .

13. A method for watermarking a ID music signal wherein the method of one of the claims 1-11 is applied to at least some of said music signal.