WO2008005406A2 - Watermarking movies to easily recognize who is the owner - Google Patents

Abstract

A new watermark (4901, 4902, 4903) for motion pictures utilizes the fact that defects of isolated still images that are clearly identifiable as such can be tuned so that the effect on the motion picture is invisible (we say that they are frame-by-frame visible or F-visible). This fixes the deadlock problem on watermark proof of ownership. F-visible watermarks can be added to the uncompressed or to the compressed movie. The frames will be cut into one or more zones and the succession of watermarking zones will be determined so that some minimal number of frames separates two consecutive utilizations of the same zone. To mark in the zones one can use a library of shapes or a generator of shapes that will produce watermarks that are human readable and recognizable as defects or recognizable images on still frames, and non-machine readable to the best of technologies to this effect.

Description

IS-WTMRK-PCT

WATERMARKING MOVIES TO EASILY RECOGNIZE WHO IS THE OWNER

CROSS-REFERENCE TO RELATED APPLICATIONS

[000] This application claims priority under 35 U.S.C § 119(e) from U.S. Provisional Patent Application Nos. 60/817337, filed June 29, 2006 the subject matter of which is herein incorporated by reference in full.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

AND DEVELOPMENT

Not Applicable

NAME OF PARTIES TO A JOINT RESEARCH AGREEMENT

Not applicable

INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A COMPACT

DISC

Not Applicable

BACKGROUND

FIELD OF THE INVENTION IS-WTMRK-PCT

[001] The present invention relates to a system and method for watermarking digital motion picture documents representing news, artistic creations, or any other type of content. The present invention offers a watermarking solution to an attack on the utilization of watermarks as proof of ownership, known as "the IBM attack" or "the deadlock attack" in the field of so called "invisible robust digital watermarks". It should be noted that non- watermarking defenses against this attack, involving, for instance, copyrights and/or the imposition of using only so-called "non-invertible watermarks", have been offered in the past. The present invention offers a solution that does not require the use of external agencies to prove creatorship or ownership of a digital motion picture, and does not prevent other parties from using whatever watermarking scheme they choose.

BACKGROUND ART

[002] The proliferation of motion pictures has increased with the number of television channels and has considerably grown further with the development of the Internet. Digital motion pictures are relatively easy to create, handle, store, and transmit. This ease of use contributes to the rapid increase of the number of documents in the digital world. The number of motion pictures (of all lengths) created or adapted into digital formats each year may soon be at a point at which most (if not all) copyright systems will be unable to fully satisfy the needs of all, and even of most, users. Watermarking techniques known as invisible robust digital watermark techniques have been proposed as a substitute for copyrighting various, if not all, digital materials. These watermarking techniques were initially meant (also if not only or for the least mostly) to be used to help one identify the legitimate creators or owners (we will often use the term owner to mean "owner or creator" in the sequel) of a digital image document (moving or still). IS-WTMRK-PCΓ

[003] Unfortunately these marks of ownership have been the object of many attacks, beginning with what we call here the "expected attacks". What we call "expected attacks" are attempts either:

- At erasing or making sufficiently illegible the watermarks so as to prevent one's ability to prove convincingly the identity of the creator of the watermarks.

- Or more particularly at guessing what algorithms producing the watermarks so that they can be fully or mostly erased to come back to the image as it was before watermarking.

Over the years, these expected attacks have been enhanced greatly, and are now sophisticated enough so that they can defeat many of the proposed watermarking schemes (see e.g., the review in "Attacks on Copyright Marking Systems" by F.A.P. Petitcolas, R.J. Anderson, and M.G. Kuhn, in D. Aucsmith, Ed., Second workshop on information hiding, in vol. 1525 of Lecture Notes in Computer Science, Portland, Oregon, USA, 14-17 April, 1998, pp. 218-238. ISBN 3-540-65386-4). In addition to the expected attacks (where one tries to erase, imitate, mask,...), there is an attack on invisible robust digital watermarks called by some the IBM attack (because of the affiliation of most of the authors when the paper describing the attack was published) and by others the deadlock attack (the name that we will use in the sequel). This attack has been discussed as far back as 1996 and defeating it in a way that neither requires imposing any sort of watermark nor uses any external (copyright) agency is the subject of the preferred embodiment of the current invention. One can say that the deadlock attack has been so devastating that the proof of ownership has by 2007 been removed by many practitioners from the list of (important) applications of invisible robust watermark, a state of affair that this invention should help change dramatically.

[004] Before we go any further let us define some basic concepts and list references on watermarking with an emphasis on robust invisible watermarks and attacks on such watermarks. IS-WTMRK-PCT

[005] The Deadlock Attack (informal presentation: a formal presentation will be provided for the sake of completeness):

• Setting: The Image (or movie) has two (or more generally n>i) watermarks.

• Question A: Which of two (or n>1 ) persons has rightful ownership?

• Question B: Which watermark was inserted first?

- Brief description: The deadlock attack consists in the realization that because Question B can be hard to answer (at least if the watermarking others than the first one are generated by invertible transformations, so is question A, hence the difficulty in using watermarks to prove ownership.

• Remark: The inability of most current techniques to deal with deadlock is independent of the insertion method or how robust it is.

• State of the art: no scheme can unambiguously determine ownership if it does not require the use of external agencies to prove creatorship or ownership of a digital motion picture nor impose the exclusive use of so- called non-invertible watermarking techniques whose actual non-invertibility can usually not be proven if one does not make some assumptions generally believed to be true but that have yet to be proven.

BASIC NOTATIONS:

[006] If I denotes a document, let Iw denote a watermarked version of the same document. We set:

a relation that defines the chosen watermarking algorithm (or watermarking operator) F_w that associates Iw = F_w(l) to the document I. We then say that the watermark algorithm F_w is non-invertible if extracting I from F_w (or more generally computing the the inverse IS-WTMRK-PCT map (Fw)-¹) is computationally hard. For the purposes of this discussion, but without limitation, we define computationally hard to mean that the number of arithmetic operations in the computation does not grow any faster than a polynomial function of the size of the document for a still picture, and on the size of the frames or block of frames being mark as a single entity for movies. Other notions of computationally hard can be used as well.

[007] A watermark is defined as a relatively minor alteration of a data set that is not meant to inhibit the ability to "recognize" the intended content of a document that is usually audible, viewable, or a combination of both. We will sometimes say "mark" (respectively "marking", etc.) instead of watermark (respectively "watermarking", etc.) when no confusion should result. In the case of movies, which is the only kind of documents that we will consider here, one can watermark the video signal, the audio signal, or a combination of both: the present invention is only concerned with marking the video signal although audio marks can be used as auxiliary marks to strengthen the overall watermarking process according to this invention. If there are several sets EW1 , EW2 EWm of marks to be used, we will denote by SW1 , SW2, ..., SWm the sets of frames that carry these sets of marks. The sets EWi and EWj for i≠j need not be disjoint and even less so for the sets SWs and SWt for s≠t, since one may find advantages to using marks from different sets, say SWs, SWt, etc. on the same frame, some of which are designed according to the present invention, and possibly also some of which are designed according to prior art.

[008] An imperceptible watermark is defined as an alteration of a data set that is, for the most part, imperceptible to a human, or at least imperceptible during normal use of the data. By this we mean (in the case of visual perception such as for a still picture or a movie, the only case to be considered here), that the watermark should be invisible or almost invisible, but still be able to be recognized (or at least detected) by a machine such as a computer. The general principles for developing watermarks are disclosed, for IS-WTMRK-PCΓ example, in "Digital Watermarking, Principles and Practice" by I. Cox, M. Miller, and J. Bloom (Morgan Kaufmann Publisher, 2001) that provides many references to the literature.

[009] A robust watermark is a watermark that is designed to survive modifications of the image, and more precisely, a watermark that remains detectable even after the image has been modified to some reasonable extent Example modifications include:

* Attacks

• Normal image processing such as compression, modest cropping, etc.

[0010] Requirements of robust watermarks usually include:

- That they be non-removable by an adverse party, at least not without visible distortion to the image.

- Some degree of cryptographic security (so that an attacker could not remove parts of the watermarks by obtaining some of the keys by cryptanalysis and/or by attacking the system that generated the marks).

[0011] A description of cryptographic techniques and applications can be found for instance in "Handbook of Applied Cryptography" by AJ. Menezes, P.C. van Oorschot and S. A. Vanstone, (CRC Press, 1997).

[0012] Initially, robust invisible watermarks usually (and perhaps mainly) were designed to establish ownership or to help protect copyrights. However, the deadlock attack mentioned in the introduction (and is described and discussed below) has seriously compromised that objective so that the initial role of being a copyright substitute has mostly been modified (and some would say downgraded) to other roles such as (without intent of limitation):

- Easy location on the Internet, of watermarked documents, IS-WTMRK-PCT

- Marking who is the customer (to allow the detection of customers who break contracts by making unapproved third party sales, for instance),

- Recognizing who steals or otherwise mishandles documents in a company that produces and/or uses digital documents,

- Recognition of frauds whereby the owning company attempts to improve its statistics, but with no serious intent of use for legal actions,

[0013] In most cases of interest to us here, one expects the robust invisible watermarks still to be detectable by machines (but not by humans, except perhaps if they pay particular attention in particular circumstances that exclude normal use, these precisions in wording being designed to address both prior art and the present invention) when the image is transferred to the analog world, for instance when printed or transformed into analog signals such as required for display on a TV or on a movie screen. Such watermarks cannot be replaced by digital signatures, known in this context as signature schemes, with no visible trace on the image do not resist transfer from the digital to the analog world.

[0014] Robust watermarks detection is usually based on statistical analysis, in part because image processing and attacks generally have some effect on the watermark: Individual components of a mark can be altered by attack and/or by digital/analog conversion. Thus, one is reduced to checking that traces of the overall mark persist.

[0015] Basic means of watermarking:

In general, on a page or frame, a robust watermark consists of either:

- An array of elements M(K) , for a cryptόgraphically designed key K. Thus,

M (K) (h, v). is such that h and v stand respectively for the horizontal and vertical coordinates of a pixel (other units can be used for marking but people versed in the art would readily adapt what is recalled here to other units), and where for each fixed set of the IS-WTMRK-PCΓ parameters, M(K)(h,v) indicates what to do at pixel (h,v) in terms of modification or replacement of the original signal, - Or a sequence S (K)(i,j)(H,V) of changes to be made in the OCT (Discrete Cosine

Transform) or other transform of the macro-block (H₁V) in the (i,j) coefficients of the transform, that depends on some key K.

Here as in the rest of this disclosure, the absence of any key is interpreted as an example of a key that does not require separate treatment.

[0016] In a movie, one can mark all the frames or a subset SW of the frames that is chosen to carry watermarks; SW is in fact the union of the sets SW1, SW2, ... . SWn that were mentioned before as carrying different sorts or different sets of marks.

[0017] Approaches toward defeating the deadlock attack began to be discussed as far back as 1996, published, for instance in "Resolving Rightful Ownerships with Invisible Watermarking Techniques: Limitations, Attacks, and Implications" by S. Craver, N. Memon, B.-L Yeo, and M. M. Yeung, (IEEE Journal On Selected Areas In Communication, Vol. 16, N° 4, May 1998 pp.573-586). In that paper, Craver et al. also proposed that non-invertible watermark algorithms would enable the defeat of the deadlock attack. We will review that point and review also the deadlock attack before we provide a list of problems to be solved by the present invention. Before doing that we assume some familiarity with the field, for now building only on concepts that are discussed later in this text (see in particular the background section devoted to watermarks) or for which we provide references. Later, we will occasionally make some allusion to more complex elements that experts in the fields of image processing, artificial intelligence, and cryptography will know how to use. These elements are not central to the core ideas of this invention, which is an invention in which several modules can be realized using a choice of technologies, with a perspective that future technologies should further improve the performances of the present invention and IS-WTMRK-PCT allow one, in particular, to overcome attacks that may be mounted against it since progress in technology enable even stronger attacks. For now, we go directly to the points that we need to make in order to describe the problems that our invention addresses.

[0018] Finding non-invertible watermarking algorithms is not difficult when using typical tools from cryptography, as long as one does not require robustness from the watermark, but one does require that the watermarks that are generated be robust to:

- Attacks other than the deadlock attack (what we have called expected attacks above: in fact, attacks on the mark itself rather than attacks like the deadlock attack, which aim at the usage that one can make of the watermark). Non-adversarial image processing.

[0019] (Examples of provably non-invertible algorithms that generate watermarks that were claimed to be robust were provided for instance (with no intent of limitation) in:

- Ex1 : S. Craver, N. Memon, B.-L Yeo, and M. M. Yeung, Cited above.

- Ex 2: "Watermarking methods for MPEG encoded video: Towards resolving rightful ownership," L Qiao and K. Nahrstedt in International Conference on Multimedia Computing and Systems (ICMCS: Austin, Texas, USA, 1998), pp. 276-285, (IEEE Washington Brussels Tokyo, 1998).

- Ex 3: "Image watermarks and counterfeit attacks : Some problems and solutions," by M. Ramkumar and A. Akansu, In Symposium on Content Security and Data Hiding in Digital Media, pp. 102-112, 1999.

[0020] While the solutions proposed in Ex 1) were invalidated in Ex 2) and Ex 3), the claims of success in Ex 2) were in turnβ challenged by the work in: IS-WTMRK-PCT

- Ex 4) "Advanced techniques for dispute resolving and authorship proofs of digital work," by A. Adθlsbach and A.R. Sadeghi, In Proceedings of the SPIE vol. 5020, Security and Watermarking of Multimedia Content V, pp. 67-688 (2003).

- Ex 5) "Watermarking schemes provably secure against copy and ambiguity attacks," by A. Adelsbach, S. Katzenbeisser, and H. Veith. In DRM'03, October 27, 2003, pp.111-119, (2003).

- Ex 6) On the Insecurity of Non-invertible Watermarking Schemes for Dispute Resolving," by A. Adelsbach, S. Katzenbeisser, and A.-R. Sadeghi, In Springer- Verlag Lecture Notes in Computer Science, Vol. 2939, ρp.355-369 (2004).

[0021] These last papers (Ex 4 to Ex 6) proposed to solve the deadlock attack problem by using more sophisticated cryptographic techniques such as recourse to zero-knowledge verification, a process that allows one to check the fact that someone has a secret without learning any useful information that would allow one to know that secret. Not surprisingly, over the years, more and more sophisticated, but also sometimes more exotic methods often imported from cryptography, have been used, but sometimes with disputable impact on the solution of the main issues. As for zero-knowledge proofs that are also used in the following example of paper on the subject (see below, Ex 7), we notice that it could be used in our own invention, for instance (with no intent of limitation) through the incorporation of the papers on watermarks that use zero-knowledge like Ex 4 to Ex7.

[0022] For reference to zero knowledge proofs, see for instance:

- The knowledge complexity of interactive proof-systems," by S. Goldwasser, S. Micali, and C. Rackoff.in Proceedings of 17th Symposium on the Theory of Computation, Providence, Rhode Island, (1985).

- "Proofs that yield nothing but their validity," by O. Goldreich, S. Micali, A. Wigderson, Journal of the ACM, vol. 38, issue 3, pp. 690-728. (1991). IS-WTMRK-PCT

- "Everything provable is provable in zero-knowledge," by M. Ben-Or, O. Goldreich,

S. Goldwasser, J. Hastad, J. Kilian, S. Micali, and P. Rogaway In Advances in Cryptology-CRYPTO '88, Proceedings, pp. 21-25. August 1988 (S. Goldwasser, editor) vol. 403 of Lecture Notes in Computer Science, pp.37-56. Springer-Verlag, 1990.

- On Defining Proofs of Knowledge," by M. Bellare and O. Goldreich In Advances in Cryptology - CRYPTO '92 Proceedings, pp. 390-420, Lecture Notes in Computer Science vol. 740. Springer-Verlag, Berlin, 1992.

As a last example of paper related to the deadlock attack, the following paper has already been mentioned above:

- Ex 7). On the Possibility of Non-invertible Watermarking Schemes," by Q. Li, and E.-C. Chang, In Springer-Verlag Lecture Notes in Computer Science, Vol. 3200, pp.13-24 (2004).

This paper also uses zero-knowledge techniques and offers the following particularity: that scheme does not use the original document I to derive the key nor the watermark. « Hence the watermark is statistically independent from the original* the authors point out. It remains to verify that this is an advantage, but anyway we will not suggest using this type of independence from the original, at least not for all marks, in the context of our own invention.

[0023] We find that all these works provide a solution to the deadlock attack that fail to have been universally accepted for several reasons, some of which we will summarize later in the section entitled "problems to be solved."

EXAMPLES OF WOULD BE ROBUST INVISIBLE WATERMARKS.

[0024] Examples of watermarks presented by their creators as being both robust and invisible have been disclosed, for instance, in:

- U.S. Pat. No. 5,825,892 issued to F.C. Mintzer and G.W. Braudaway, IS-WTMRK-PCΓ

- U.S. Pat. No. 6,154,571 issued to IJ. Cox, M.L Miller, and R. Oami

- U.S. Pat. No. 6,278,792 issued to IJ. Cox , M.L. Miller, and R. Oami,

- U.S. Pat. No. 6,724,911 issued to IJ. Cox and M.L Miller

- U.S. Pat. No. 6,738,493 issued to IJ. Cox and M.L Miller,

- Cox, J. Kilian, T. Leighton, and T. Shamoon, "Secure spread spectrum watermarking for multimedia," IEEE Transactions on Image Processing, vol. 6, no. 12, pp. 1673-1687, (December 1997).

- J. Chou, S. Pradhan, and K. Ramchandran, "A robust blind watermarking scheme based on distributed source coding principles," in Proc. of ACM multimedia conference, Los Angeles, CA, USA, October 2000.

- H.S. Malvar and D.A.F. Florέncio, "Improved spread spectrum: A new modulation technique for robust watermarking," in A. Akansu, E. DeIp, T. Kalker, B. Liu, N. Memon, P. Moulin, and A. Tewfik, "Special issue on signal processing for data hiding in digital media and secure content delivery," IEEE Trans. Signal Processing, vol. 51 , no. 4, Apr. 2003., pp. 868-905.

- "Watermarking methods for MPEG encoded video: Towards resolving rightful ownership,"by L Qiao and K. Nahrstedt in International Conference on Multimedia Computing and Systems (ICMCS: Austin, Texas, USA, 1998), pp. 276-285, (IEEE Washington Brussels Tokyo, 1998).

• "Image watermarks and counterfeit attacks: Some problems and solutions," by M. Ramkumar and A. Akansu, In Symposium on Content Security and Data Hiding in Digital Media, pp. 102-112, (1999).

- "Advanced techniques for dispute resolving and authorship proofs of digital work," by

A. Adelsbach and A. R. Sadeghi, In Proceedings of the SPIE vol. 5020, Security and Watermarking of Multimedia Content V, pp. 67-688 (2003).

- "Watermarking schemes provably secure against copy and ambiguity attacks," by A.

Adelsbach, S. Katzenbeisser, and H. Veith. In DRM'03, October 27, 2003, pp. 111-119, (2003). IS-WTMRK-PCT

- On the Insecurity of Non-invertible Watermarking Schemes for Dispute Resolving," by A. Adelsbach, S. Katzenbeisser, and A.-R. Sadeghi, In Springer-Verlag Lecture Notes in Computer Science, Vol. 2939, pp.355-369 (2004).

- "On the Possibility of Non-invertible Watermarking Schemes," by Q. Li, and E. -C. Chang, In Springer-Verlag Lecture Notes in Computer Science, Vol. 3200, pp.13-24 (2004).

[0025] Attacks on watermarks or on their assumed utilizations:

Several attacks on robust invisible watermarks or their use as ownership proofs were reviewed for instance in:

- Craver, N. Memon, B.-L Yeo, and M. M. Yeung, "Resolving rightful ownership with invisible watermarking techniques: limitations, attacks, and implications," IEEE Journal of selected areas in communications, vol. 16, no. 4, pp. 573-87, May 1998, Special issue on copyright and privacy protection.

M. Kutter, S. Voloshynovskiy, and A. Herrigel, "Watermark copy attack," in Security and Watermarking of Multimedia Contents II, P. W. Wong and E. DeIp, EdS₇ San Jose, CaI., USA, January 2000, vol. 3971, SPIE Proceedings.

- Cox and J.-P. Linnartz, "Some general methods for tampering with watermarks," IEEE Journal on selected areas in communications, vol. 16, no. 4, pp. 587-93, May 1998, Special issue on copyright and privacy protection.

- "Image watermarks and counterfeit attacks: Some problems and solutions," by M. Ramkumar and A. Akansu, In Symposium on Content Security and Data Hiding in Digital Media, pp. 102-112, 1999.

- "Advanced techniques for dispute resolving and authorship proofs of digital work," by A. Adelsbach and A.R. Sadeghi, In Proceedings of the SPIE vol. 5020, Security and Watermarking of Multimedia Content V, pp. 67-688 (2003). IS-WTMRK-PCT

- "Watermarking schemes provably secure against copy and ambiguity attacks," by

A. Adelsbach, S. Katzenbeisser, and H. Veith. In DRM'03, October 27, 2003, pp. 111-119, (2003).

- On the Insecurity of Non-invertible Watermarking Schemes for Dispute Resolving," by A. Adelsbach, S. Katzenbeisser, and A.-R. Sadeghi, In Springer-Verlag Lecture Notes in Computer Science, Vol. 2939, pp.355-369 (2004).

[0026] We notice that many robust invisible watermarks have eventually been proven to either be not robust enough, or not secure, or a combination of both. As attacks get more and more sophisticated, better and better marks are required, just as in cryptography where codes need to be more and more secure because of progress made in cryptanalysis. There is also a related but more subtle issue that we call "Problem P" and that links cryptography issues to perception issues:

Problem P: In the watermarks context there is, on top of the cryptographic security issue, a perceptibility issue that has no real analog in the context of pure cryptographic security and that may well prevent forever any form of provable full security.

In fact, because human perception is so complex and seems hard to fully model, it seems hard to (mathematically) prove simultaneously that a mark is not perceptible and that it is secure. This is a problem that we will illustrate in Figure 1 in relation with the invention disclosed here. Problem P has its place by the side of the others "Problems to be solved" listed next.

PROBLEMS TO BE SOLVED. IS-WTMRK-PCΓ

[0027] We only consider watermarking of movies, and the problems to be solved as well as the invention, are all in that context, which is what we always assume henceforth, except if otherwise specified.

Current methods for robust watermarking of motion pictures with no perceptible quality deterioration often use the same basic concepts and assumptions that are used for still pictures. The robust invisible watermarks must be both:

• Invisible to the naked eye,

• And robust,

Robustness means that the mark is robust to:

• Removal without very visible traces,

• Compression,

• Weak cropping,

• Small linear and non-linear distortions, and finally:

• Secure against the attacks aimed at destroying the watermark, as long as these attacks only provoke acceptable image deterioration.

[0028] We claim that as a result of the invisibility requirement (as will be made clear as a by-product of our invention), robust invisible watermarking fails to fulfill its primary objective which was to serve as an effective proof of authorship, good enough to be defended in most courts. This is because it is not possible to prove that a second watermark was actually the second to be inserted (this is the so-called deadlock attack that we have already mentioned and that we will next define more formally), except possibly if all watermarks that are used are non-invertible. We notice that even if mandatory non- invertibility is accepted and can be enforced, in order to solve that way the deadlock attack we still have to assume that all problems associated to non-invertibility could be solved up IS-WTMRK-PCΓ to, for instance, the fact that the jury would have to believe mathematical proofs of non- invertibility to be correct despite the above mentioned Problem P.

[0029] We now give a formal enough presentation of the deadlock attack in order to formulate problems that persist despite the example of papers (Ex 1 to Ex 7) cited previously. If P1 and P2 are two parties, and P1 creates document I and releases:

IW1 = FWI (I). where W1 is the watermark of P1 , then P2 can keep:

l' = Gw2^'1dWi). and pretend that I' is the original while presenting lwi now understood as

IW1 = GW2 (GW2~¹OW1)) = GW2 (0. as being the W2-watermarked version of the would be original I'.

[0030] Under some circumstances, it was shown (see the papers Ex1 to Ex7 for instance) that this abuse of property pretense could be avoided only if all parties use non-invertible

watermarking algorithms, so that Gyv2 ^and GW1 ^{are not} computationally accessible functions, and if some extra precautions are taken, that may involve:

• Using a Trusted Third Party (a practice which, in our opinion and in the opinion of many users, seems to defeats almost entirely the advantage of finding a substitute to watermarking, as it would replace a well established copyright system in some countries by one or many new ones, that all would need to establish some form of mutual trust),

• Or using very advanced cryptographic techniques that may be hard to explain to the jury in case of a trial.

[0031] We next list 5 problems (problems A to E, a list that we do not propose as being complete) that come up with solutions to the deadlock attack as discussed in the previous IS-WTMRK-PCT paragraph. These five problems support our claim that, using only prior art, "invisible watermarks fail to emerge as a universally accepted candidate for an efficient proofs of ownership":

• Problem A) Non-invertibility may be hard to prove, especially if one does not rely on commonly accepted, but still unproved, assumptions, such as the fact that some problems whose solution can be checked in polynomial time can have no polynomial time solution.

• Problem B) Some problems that once were widely considered to be computationally hard, eventually have found some reasonable solutions by using some unexpected tricks or progress, or by some seemingly -insignificant results that were not expected to be accessible or to be of any interest In cryptography for instance, the barriers of security need from time to time to be pushed further up so that protections by using the computational hard criterion may sometime be deceptive. As an example with no intent of limitation, the RSA (Ribet-Shamir- Andelman) public key encryption and signature need now 1024 bits while 512 bit where considered as secure for several years.

^• Problem C) One would have to force the other parties to also use non-invertible watermark algorithms.

^• Problem D) Many of the proposed solutions require a trusted third party or a form of authority on watermark, which could be acceptable if not easily avoidable, as we shall demonstrate, as a by-product of our invention. Furthermore, the acceptability would depend on the authority to be international in order to be good enough to be considered as a good replacement of copyright. IS-WTMRK-PCΓ

• Problem E) If enough of the watermark can be erased (and techniques for such attacks are making a lot of progress), one might lose all the benefits of imposing non-invertible watermarking algorithms.

[0032] This state of affairs is well known by some in the industry and in the academic world. As indicated by elements of the literature, has too often been hidden to customers, often because the deadlock attack is not universally known and even less understood. One may estimate that this state of affairs has caused great damage to the commerce of watermarks since the proof of ownership could have probably created more revenue for the watermarking industry than the applications of

^• Invisible fragile watermarks,

• Visible robust watermarks,

^• And robust invisible but with no proof of ownership all taken together.

[0033] It is thus a goal of the present invention to present a novel method for watermarking motion pictures that, among other advantages, allows one to defeat the deadlock attack without suffering of any of the Problems A to E listed above (nor of problem P): in fact the new method according to the present invention does not require to impose non-invertibility to every acceptable watermark technique. As a consequence, solving Problems C to E and Problem P are the main remaining problems that are solved by the present invention that simply disposes of Problems A and B by not requiring one to impose non-invertibility. IS-WTMRK-PCT

SUMMARY OF THE INVENTION

[0034] The new watermark for motion pictures that is the object of this invention utilizes the fact that defects of isolated still images that are clearly identifiable as such can be tuned so that the effect on the motion picture is invisible: such marks that are "frame-by- frame visible" but invisible in the movie are called F-visible. The invention described herein not only fixes the deadlock problem for watermarks, but also has properties such as resistance to automatic detection and to correction and also offers further benefits. For instance, our invention provides other advantages of watermarks by simply modifying parameters such as which prior art techniques are incorporated in the mark that is made according to this invention. Our watermark can be executed on either the uncompressed or compressed format. The frames that will carry marks according to the present invention (W-frames), will be cut into one or more zones and if needed, the succession of zones that carry watermarks according to the present invention (W-zones) will be determined (by arithmetical or statistical means for instance) so that some minimal number of frames separate two consecutive uses of the same zone (or successive blocks of consecutive frames where one uses approximately the same mark on the same zone or about). Such precautions are more important when many of the frames are W-frames, in which cases one will preferably choose to have many candidate W-zones per W-frame and take care of the succession, while this problem could be moot if only a small proportion of all the frames are W-frames. To mark in the W-zones according to the present invention one can use either:

(1 ) A combination of a library of shapes (i.e., sets of pixels or of macro-blocks in the sense of the usual techniques of compression such as MPEG-2) along with some specified set of operations to be performed either on a pixel or macro-block level within the shapes,

or IS-WTMRK-PCΓ

(2) A random generator of shapes (as defined above) along with some specified set of operations to be performed either on a pixel or macro-block level within the shapes.

Either (1 ) or (2) will produce sorts of shadows on the W-frames that are (i) Human-readable and either recognizable or recognizable as defects (without intent of limitation of what human-readable may mean) on still frames. (ii)-Non-machine readable to the best of technologies to this effect, (iii) Non-perceptible in the motion picture. The items (i) and (iii) together define what we call "F-visible."

[0035] Let us be a bit more specific, leaving fine details to the section on the description of preferred embodiments.

A) If the watermarking is performed in a non-compressed format, to be used in non-compressed format (a rare situation nowadays) several or all frames will carry few recognizable or strange shaped (with no intent of limitation) zones with luminance and/or chrominance excess or deficit, taking care that the zones where such spots appear do not repeat too frequently (except possibly for blocks of consecutive frames where one purposely uses a mark and attenuations of it: we will often not mention this provision that experts in the arts of video would known how to handle). Such alterations of some or all frames of the movie can be done by either modifying the analog or modifying the digital signal, or modifying both signals.

B) The modifications on the non-compressed format as discussed above would transport by themselves through compression to compressed formats such as the well-known MPEG (acronym of Moving Picture Expert Group) formats (without IS-WTMRK-PCT limitation on the high quality compression method). Resistance to compression is a necessary property and can be achieved by a variety of old or new techniques, and even more so in the "visible on still frames only," or F-visible arena. C) One may prefer to add the watermark at compression time or after compression. In MPEG for instance, one distinguishes 3 types of frames, called the l-pictures, the P— pictures, and the B~pictures, the first ones with no reference to other frames, the second ones computed with some reference to previous frames, and the last ones computed with some reference to previous frames and with some reference to following frames.

In the sequel, we also say l-frames, P-frames, or B-frames for frames that are respectively l-pictures, P-pictures, or B-pictures.

[0036] Among all frames, only the l-frames would not need to contain the erasure of what would be done on the previous frame. Therefore, one can either: a) Watermark many or all of the I-, P-, and B-pictures, taking care of all erasures (or other required forms of visual correction) on the following frame, as needed or b) Only watermark the frames that come just before i-frames.

[0037] In any case, we will determine a set of W-frames (for instance all the frames, or a subset of the l-pictures, or a variety of other possible sets of W-frames), that will carry watermarks according to this invention, except in some special cases such as frames of uniform color that makes it better to avoid them or frames on which no zone is judged to be far enough from all the zones that have been used close by previous W-frames to watermark. We need next to determine the W-zones, i.e., the parts of the W-frames will carry our marks. IS-WTMRK-PCΓ

[0038] For some N_max, the set SW of W-frames can be written as the union

SW= SW1 » SW2 » ... » SW N_max.

Here, for any iOE{1 , 2, .... N_maχ}, Swi represents a set of frames that carry F-visible marks (or F-marks) of some sort indexed by i according to the present invention, while there may be some number Ntotal ^≥ Nmax ^sucn that for any j(E{N_max+1 , N_max +2 _Ntotal). SWj represents a set of frames that carry robust invisible marks according to some chosen method such as (for instance, with no intent of limitation) methods from the cited literature on robust invisible watermarks. Some of the invisible marks can be attenuated versions of the F-marks, as will be discussed in the context of some preferred embodiment.

Recall now two examples of video formats in pixel sizes to provide an idea on orders of magnitude:

^• "VHS video" quality: 352x288 pixels per frame,

* Standard Definition Television (SDTV) quality: 704x576 pixels per frame.

[0039] In terms of size the most delicate case for the implementation of our invention is that in which the size is small, so we will consider only in this summary the more difficult case of the VHS format: 352x288 pixels per frame. Progressive or interlacing should not matter in an essential way: we will consider that we are in the progressive case where individual frames are treated as whole units. It will be obvious as to how the invention needs to be adapted to the interlaced case where each frameβ comes as a succession of fields (usually two fields: one made up of the odd numbered lines and the other composed of the even numbered lines).

[0040] Since 352/8=44 and 288/8=36, there are 44x36 DCT (Discrete Cosine Transform), that uses 8 by 8 blocks of pixels macro-blocks in the VHS video format that we consider as possibly the most challenging embodiment to explain the invention. We will separate the IS-WTMRK-PCΓ frame into 11 vertical strips (each 4 macro-blocks wide) and 6 horizontal bands (each 6 macro-blocks high) for a total of 66 watermarking zones, each watermarking zone being constituted of 4 by 6 rectangular groups of DCT macro-blocks. One could instead use complicated shapes (i.e., not necessarily rectangular) for the watermarking zones, and these zones need not be all of the same shape nor of the same size (for instance, one may use bigger zones away from the centre of the screen to have less interaction with the main content of the movie).

[0041] The possible orderings in which the watermarking zones are marked can be achieved by known methods of arithmetic, random sampling, or a combination of both to guaranty the rare repetition of any watermarking zone from one frame to a nearby one. The basic principle is that consecutive regions be as far as possible from all recently visited regions. Examples of systematic, mathematics-based methods will be discussed next (with no intent of limitation):

1 ) One intertwines the zones for dark and light spots (or more generally of any two families of marks), assuming that N_max=2 and one uses arithmetic progressions modulo the number of zones of each type to organize the numbering of the zones.

2) One takes a random sampling of where the next zone will be chosen out of neighborhoods of the two (or more) previous choices. For previously marked sites, the size of the forbidden neighborhood may decrease with time until the site is no longer forbidden.

[0042] We proceed to the next element of the invention that concerns what is actually done in a watermarking zone. Basically, one will have a (possibly movie-dependent) library of shapes or a (possibly movie-dependent) pseudo-random generator of shapes that will produce shadows that are:

1) Human readable on still frames (or F-visible) IS-WTMRK-PCΓ

2) Non-machine readable (i.e., the shadows are not recognizable by a machine to the extent of current technologies to this). We are helped here by the fact that an ugly spot is hard to recognize as such by a machine, harder than a number or letter or word hidden so that the machines are defeated.

3) Non-perceptible in the motion picture.

[0043] The number of frames used to watermark may be adjusted to the objectives of the marking and can depend on many considerations such as the tradeoff between security and time to watermark, or be conditioned by the requirement of better invisibility in the motion picture mode. There are several watermarking techniques that can be applied to get a visible watermark on still pictures. If anyone were to recognize and suppress a W- zone using image restoration techniques, then any court or jury should be able to easily recognize (with minimal equipment such as special glasses with no intent of limitation) that the true original is better than any would-be original obtained by doctoring the watermarked version. The process that we describe can be implemented in many ways on various computer systems, and we will provide examples in the preferred embodiments section, with no intent of limitation, if desired, all the steps of the process will be done using the security tools that have been developed over the years on the basis of modern cryptography, and preferably this will be reflected in the system used to implement the watermarking method that is the object of this invention.

IS-WTMRK-PCT

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0044] The principles and main advantages of the present invention will be better understood by illustrating some concepts and in the examples illustrated by the figures to follow, where:

^• Figure 1 represents the three sets of video documents that are either visually non- distinguishable from the original I, visually distinguishable from the original I using frame-by-frame careful human examination, or visually distinguishable from the original I during normal video viewing: these three sets allow a presentation of the new invention compared to the use of robust invisible watermarks.

^• Figure 2 represents a frame in the 352x288 pixels per frame format of VHS video, divided into 66 watermarking zones, with a numbering of the zones that is split into two sets of 33 zones, each set carrying numbers 1 to 33, according to one of the possibilities for the handling of zones offered by the present invention.

^• Figure 3 represents a frame in the 352x288 pixels per frame format, divided into 66 watermarking zones as in Figure 2, but now with a reordering of the zones that were enumerated in Figure 2 to define one possible order in which the zones can be used for watermarking with F-visible marks, according to the present invention.

^• Figure 4 represents an alternate way, using random draws, of ordering the zones that can be used for watermarking with F-visible marks, according to the present invention.

^• Figure 5 represents a further alternate way using random draws, of ordering the zones, that can be used for watermarking with F-visible, according to the present invention.

^• Figure 6 represents still a further alternate way, using random draws, of ordering the zones that can be used for watermarking with F-visible marks, according to the present invention.

^• Figure 7 represents a flow chart to watermark as suggested in Figures 4 to 6, when the watermarking is done before video compression. IS-WTMRK-PCΓ

Figure 8 represents a typical succession of frames in a group of pictures according to the MPEG-2 compression method.

• Figure 9 represents a flow chart to watermark as suggested in Figures 4 to 6, when the watermarking is done after video compression.

• Figure 10 illustrates a first type of watermarks to be used on watermarking zones according to the present invention.

• Figure 11 illustrates a second type of watermarks to be used on watermarking zones according to the present invention.

• Figure 12 illustrates a third type of watermarks to be used on watermarking zones according to the present invention.

• Figure 13 illustrates a fourth type of watermarks to be used on watermarking zones according to the present invention.

^• Figure 14 represents a flow chart that teaches the high level method to watermark according to the present invention in the case when the invention is implemented on the non-compressed video.

• Figure 15 represents a flow chart that teaches the high level method to watermark according to the present invention in the case when the invention is implemented on the compressed video.

• Figure 16 represents two methods to protect better against attacks on the F-visible watermarks to be inserted in chosen watermarking zone in chosen watermarking frames according to the present invention.

^• Figure 17 illustrates an exemplary hardware configuration that is adapted to be an information handling system for incorporating the present invention. IS-WTMRK-PCT

DEATAILED DESCRIPTION OF THE INVENTION

[0045] With reference now to Figure 1 , we see a graphical representation of the main idea of the present invention as compared to utilization on robust invisible watermarks according to prior art. With the same notation previously used to present the deadlock attack, P1 and P2 are two parties, and P1 creates document I at 10 and releases lwi = Fwi (I) at 11 , where Fyv 1 is the watermark operator of P1. Instead of proceeding as in the standard deadlock attack as we have presented it, we generalize somewhat the kind of attacks that P2 may attempt in the spirit of the basic deadlock attack. Hence P2 now deforms strongly lyyi to (lwi)' ^at 22» which is an image as different as possible from lwi . but with no visible difference from I, by which is meant that only a machine can help humans distinguish between the two elements of any of the pairs:

(Uwi). (l,( IWi)'). and (lwi,( »Win- Next, P2:

* creates the watermarked document

OW1)'W2 = GW2((IW1)') at 23, (where Gw2 stands for the watermarking operator of P2) and keeps (lwi)' ^as the would-be original un-watermarked document.

[0046] Given the progress made on attacks on watermarks (as discussed for instance in references Ex4 to Ex6), the worst case where (lwi)' does not contain a convincing trace of W1 does not seem improbable. It would be difficult to prove that (lwi)' should be ruled out as an original since we cannot formulate an efficient and faithful formalization of the concept of visibility and recognizability by human agents. This is a very hard artificial intelligence problem, at least for a set of images large enough to cover what one uses in most movies. In any case, even if W1 has not been sufficiently erased and if one does not enforce the use of non-invertible watermarks, one is still fighting the standard deadlock IS-WTMRK-PCT attack by simply using (lwi )' = lwi ■ Keeping in mind Problems A to E as they have been expounded and we see in Figure 1 that the weakness of the prior art comes from the fact that all the documents that are involved (i.e., I, lwi. (lwi)'_> ^and OW1)'W2) pertain to the space 70 of images that are perceptually not-distinguishable from the original document. Notice that in order to simplify some statements, we consider a non-watermarked document as a document watermarked with an empty or null watermark. This convention allows us for instance to say that I belongs to the space of documents for which the watermarks are invisible even under frame-by-frame examinations by human agents (or, as we will say for short Irame-by-frame-invisible", that we can further shorten to "fully invisible").

[0047] Still in reference to Figure 1, in contrast to the prior art the present invention uses watermarks that are visible to the human eye by a careful frame-by-frame examination (or F-visible), i.e., people with good vision (i.e., people who are used to watch movies and have normal vision) will be able to recognize the mark by looking carefully at the watermarked still frames of the document INW created at 12 using the new watermark NW according to the present invention.

[0048] NW will be conceived as will be explained so that:

* The detection of NW using a machine will be difficult, if at all possible, and at any rate, possible only with some rate of success that is significantly smaller than 100%,

^• The best correction using available technology, (INW)' at 60, will remain in the space 80 (containing INW) of documents distinguishable from I by a careful frame-by-frame examination but which stay out of the space at 90 of documents for which the watermark can be perceived while the movies is being played. IS-WTMRK-PCΓ

[0049] With reference now to Figure 2, we see a typical frame in the 352x288 pixels per frame format, the format for VHS (Video Home System) video, which is one of the smallest formats of concern for us. We use VHS because it is a hard case to treat because the format is small and because its name is well known, even to the public at the time of filing, but this should not entail any lack of generality of the invention since.

[0050] Some basic concepts on formats:

The HDTV (High Definition Television) size standards are much bigger than the VHS format:

^• 1080-line (1920x1080) or

^• 720-line (1280x720)

To give a typical example in traditional TV, the American NTSC (National Television System Committee) works in maximal resolution of 720x486 pixels per frame, a size of the same order of magnitude as the European standards:

- SECAM (Sequentiel Couleur avec Mόmoire, French for "sequential color with memory") and

- PAL (Phase-Alternating Line).

The 352x240 format optical DVD (Optical Video Disk)), for the NTSC standard, is smaller than the VHS format, but the 352x288 that works for the PAL standard, is the same size as VHS and is almost as small as the DVD format. For the purpose of our discussion, we chose the VHS format as it gives us more room in our diagrams to illustrate our point. The technique can be applied to the DVD format without problem.

[0051] Back to Figure 2, since the DCT (Discrete Cosine Transform) macro-blocks are 8x8 pixels-large, there are 44x36 DCT macro-blocks in the 352x288 frames (there would be IS-WTMRK-PCT

44x30 for the 352x240 frames). We will separate the 44x36 DCT macro-blocks of the

352x288 frames into 11 vertical strips and 6 horizontal bands for a total of 66 watermarking zones, each watermarking zone being constituted of 4x6 macro-blocks. We would get the same number of strips and bands for the 352x240 frames, but now with 4x6 macro-blocks for each watermarking zone. So what we discuss below, for the relatively small example coming from VHS, would work in the same way but with slightly smaller watermarking zones for the even smaller 352x240 frames. As for the larger formats, one can expand the zones, or even more efficiently, get more zones so that anyone versed in the art of video would readily know how to adapt to any other formats. As was mentioned in the summary of the invention, instead of regular zones that are all identical to each-other, one can use any partition into zones, including having a single zone. A combination of different splits into zones that would respectively be used in sets of frames SW1, SW2, ..., SW N_max is another preferred possibility.

[0052] Whether one uses a progressive mode or an interlacing mode should not matter in an essential way, as the experts in video will immediately recognize, so that this distinction will not be considered here: we will consider that we are in the progressive case where successive frames are treated in integrity. How the invention needs to be adapted to the interlaced case (where each frames come as a succession of fields, usually 2: the odd numbered lines and the even numbered lines) is rather trivial: one splits the watermarking zones defined for the invention into the parts that fall in the two fields, and each of the two parts will contains its share of what is conceived for the full zone seen as part of a full frame.

[0053] Coming back now to the exemplary 352x288 frames as represented in Figure 2, we see there the frame at 100 decomposed into the 11x6 watermarking zones that are split into two groups (in the Figure, as an example but with no intent of limitation as one single IS-WTMRK-PCT group or more than two groups): one group carries black-colored numbers from 1 to 33, and one group carries grey-colored numbers from 1 to 33 for the total of 66 zones.

[0054] Since these zones will be used to carry watermarks that are F-visible, it will be useful if not important that:

^• ZS1 : zones be chosen in a succession such that any zone is not used again too early, and

^• ZS2: successive zones to be used are not too dose together.

[0055] We will provide some means to satisfies both Zone Succession criteria ZS1 and ZS2 with reference to Figures 3 to 8, and anyone versed in relatively elementary mathematics will easily recognize that there are many further means to satisfy these two criteria. We will first discuss the satisfaction of ZS1 and ZS2 with no reference to the security of the watermarking scheme according to our invention, and only revisit security issues later on. In fact,

^• Sed : On the one hand we believe that with the proper care on another aspect of the invention that is the choice and the precise handling of the watermarks to be discussed below, the protocol that we propose would be resistant enough to attacks to offer proper protection in terms what it would cost for a successful attack, even if the choice of the successive zones is known to the attacker without inspection of the document. In fact, since the watermarks that are used in the invention are frame-by-frame-visible under careful inspection, the secrecy of the order would only (at best) add to the time needed for a successful attack.

^• Sec2: On the other hand, anyone versed in elementary methods of modem cryptography could readily build a plethora of means to define secret successions of zones that could only be read by locating the successive marks: we will discuss some examples below for the sake of completeness. IS-WTMRK-PCΓ

[0056] We recall that the problem of the choice of successive W-zones is only important for movies with dense W-frames, or in parts of movies where the W-Frames are dense: when W-frames are separated enough from each other, one need not even have more than a single zone and more generally, the location of successive zones is less and less important when the density of W-frames gets smaller, with quantitative specifications that depend on the technical details of the movie and of the amount of visibility of the F-visible frames.

[0057] With reference now to Figure 3, we see an order of zones to be visited that prevents too early repetition of any given W-zoneβ, or too close occurrences of contiguous zones. The example of order of zones displayed in Figure 3 can be described in the following terms that invite to easy generalization: Order-a) We first consider the 11 first zones marked with black numbers in Figure 2, and mark the zone that is marked NCE{1,2 11} in Figure 2 using the number (1+6(N-

1))mod.11, i.e., using the modulo symbol "mod." that is standard in arithmetic, we write in black characters the rest of the division of (1+6(N-I)) by 11 in integers on frame 200 of Figure 3 the zone that is initially marked NGE{1 ,2 11} in frame 100 of Figure 2. Hence:

We keep a 1 where there was a 1 in Figure 2,

We get a 7 where there was a 2 since 1 +6(2-1 )=1 +6=7,

We get a 2 where there was a 3 since 1+6(3-2)=1+12=13, and 2 is the rest of the division of 13 by 11 in integers, and so on as represented by comparing Figures 2 and 3.

Order-b) We next permute the successive black numbers from 12 to 22, from 23 to 33, and the successive grey numbers from 1 to 11, from 12 to 22, and from 23 to 33 in the same way as described in Order-a) for the successive black numbers from 1 to 11 , as can be read precisely by comparing the frames 100 in Figure 2 and 200 in Figure 3. IS-WTMRK-PCΓ

[0058] Many other forms of permutations would be possible. Here we have separated the grey numbers from the black numbers to offer the possibility of using two types of visible watermarks, say EW1 for the zones marked with black numbers and EW2 for the zones marked with grey numbers for instance. As a specific example with no intent of limitation, one can use for instance rather dark marks in EW1 and rather clear marks in EW2, or we can use two different sorts of marks of the types to be discussed below with reference to Figures 9 to 12.

[0059] We come back to the fact that not all frames must carry watermarks, and in particular not all frames must be W-frames and carry watermarks according to the present invention. For instance, in the case when the invention is used after compression, we can choose to use only l-pictures (or frames that are intra-coded, i.e., coded with no reference to other frames). Whatever the total set of frames that are susceptible to carry F-visible watermarks according to the present invention, one can use a subset of the set of frames, say the W-frames, that can be kept secret. For instance, one can chose a set of gaps between two successive frames being marked (remembering that in MPEG-type compression, if an I picture, is marked in a frame-by-frame-visible way, the next frame -or some later mark if we want to come closer to the limit of visibility while watching the movie - should carry an erasing mark, but we do not count that as a mark): as an example the gaps between usable frames being actually used can be in the range 4 to 7 and one can use a pseudo-random generator with a secret key K to generate the succession of the lengths of gaps to be chosen in the chosen possible set of gaps. Of course, if a frame is designated, either by a regular or a non-regular succession, that cannot carry any mark, for instance because it is all white, that frame will be omitted in the marking process. It is with these provisions that the exemplary order proposed in Figure 3 will be used. IS-WTMRK-PCΓ

[0060] Other methods to define the succession, with the same provision on the set of frames a priori usable and the actual subset of those determined by fixed or secret random gaps, will be discussed next with reference to Figures 4 to 9. More precisely, Figures 4 to 6 illustrate some new methods that we offer as an alternative to what was proposed in Figure 3, and the corresponding algorithms will be discussed using the flow charts in Figures 7 and 9.

[0061] With reference now to Figure 4, we see in panels A to F six successive frames that will carry frame-by-frame-visible watermarks, or W-frames. In A is the first W-frame, and we see in B, C, D₁ E, and F the 5 following W-frames, all like the frame 100 from Figure 2. On W-frame A, we see that the location of the first watermarking zone (W-zone or zone when there is no confusion) marked by a white number 1 on a black background. This first zone to be used is accompanied by a neighborhood at 110 where the next zones cannot be chosen as long as the forbidding neighborhood is active. On frame B, the second zone to have been drawn, by a fully deterministic or pseudo-random process that can be either:

Publicly known

Or secret and depend on a key K is represented by the white number 2 on a black background. We use K even in the case when that key is known publicly, hence not secret, and we call PsRand(K.U) the process that has drawn zones 1 and 2 and that will draw the following zones (we use PsRand as name of the drawing process, with the common idea that a fully deterministic process is a particular case of a pseudo-random one).

[0062] Coming back to Figure 4 and more precisely to frame B, we see around zone 2 a neighborhood that is forbidden for the next zone to be drawn and we see also that the neighborhood 110 of zone 1 is still figured on frame 2 to mean that the same neighborhood around zone 1 that was forbidden for zone 2 is also forbidden for zone 3. On frame C, zone 3 has been drawn out of both neighborhoods 110 of zone 1 and 120 of zone 2, while IS-WTMRK-PCT a third neighborhood, 130, now around zone 3 is represented, while 110 and 120 are still present meaning that zone 4 will have to be out of both 110, 120, and 130. Passing now to frame D, the fourth W-frame, we begin to get in the cycle of general W-frames, since now neighborhood 110 is no longer presented, while a new forbidden neighborhood 140, now around zone 4 has appeared. Now, for the process being represented by Figure 4, and with no intent of limitation, there will always be exactly three forbidden neighborhoods, all square and of the same size, that will be forbidden for the following zone starting with zone 4: the examples of W-frames 5 and 6 are represented as frames E and F, with new forbidden neighborhoods 150 and 160.

[0063] With reference now to Figure 5, we see a process very similar to what is represented in Figure 4, except that the square neighborhood 110, 120, 130, 140, 150, and 160 from Figure 4 are now replaced by round forbidden neighborhoods 210, 220, 230, 240, 250, and 260 in Figure 5.

[0064] In the case of a round forbidden neighborhood (with a richness of choices that is more obvious), we can decide in many way what is really forbidden: it can be all the zones that intersect the forbidden disks, or the zones whose center are inside the forbidden disks, with no intent of limitation of the exclusion rules (similar specification is needed with square neighborhoods if they do not exactly fully cover some number of neighborhoods). In symbols, for square or circular neighborhoods, or for other shapes of forbidden zones that can be useful (and recall here that zones may be strangely shaped), we will declare that "a zone is not forbidden as W-zone for W-frame N because of the neighborhood around the previously used W-zone on W-frame number N-I" by writing that d_a(Z{X,N}, R(N-I)) > r (I) where: d_a refers to the distance being used by the decision algorithm called a, IS-WTMRK-PCΓ

Z{X,N} refers to some point chosen in the zone tentatively chosen at location X for W-frame N₁ (typically one would choose the center of that zone),

R(N-I)) refers to some point chosen in the zone that has been chosen for the W-frame N-I (typically one would choose the center of that zone), r (I) refers to some radius depending on I (and possibly also on other variables, e.g., if the shapes of the zones are not all the same) around the point R(N-I)) according to the chosen distance d_a-

[0065] In the processes illustrated in Figures 4 and 5, the radius r(l) is the same for all the three forbidden neighborhoods (in effect, up to three since we consider the cases when N=1 ,2,3 for which respectively no I, only 1=1 , and only IOE{1 ,2} make sense) , i.e., we have there a case where: r (1) = r (2) = r (3) > 0 ; and r (I) = 0 for I > 3.

This type of all or nothing dichotomy where the same forbidden neighborhood around all the recently used zones are forbidden is not the only possibility, nor even probably the best possibility: the size of the neighborhood can be constant for some number of steps and then decrease with some number of steps with lengths that may or may not be context dependent For larger formats like for HDTV, the decrease may be done in several steps each lasting some number of values of I. Such general behavior of the radius r (I) of as a function of I will be easily understood from the simple example that we describe next. With reference now to Figure 6, we see that we have more forbidden neighborhoods that can co-exist and be active at once since after three steps with the same radius, as for instance for forbidden neighborhood 310 of zone 1 in frames A, B, and C, we see the forbidden neighborhood 310 of zone 1 be replaced by neighborhoods of 1 that get smaller and smaller at 311 and 312 in frame O, and E, and disappear completely in frame F. The beginning of the same evolution around zone 2 can be seen at 320, 321 , and 322 (but the IS-WTMRK-PCΓ evolution around 2 could be different from the evolution around 1 , for instance because the visibility of the marks at 1 and at 2 are different, with one visible by most people after 15 minutes of examination, while the other mark is only seen by about 50% of people after an average of 30 minutes of examination (the time of examination may comprise a time of training, or can be preceded by a time of training).

[0066] One could imagine that some conjunction of choices of the function r(l) and some instances of the draw of successive W-zones would lead to a candidate W-frame that would be fully covered by the forbidden neighborhoods: such a possibility should not necessarily condemn the particular choice of the function r(l): more precisely, one could just not use such a W-frame with no room for a W-zone, and then let the decrease of the function r(l) leave room in the next candidate W-frame or in any case, in some following candidate W-frame. Here a candidate W-frame is a frame that is drawn to be a W-frame by some system. To be more specific on the drawing mechanism, in some case, all frames or all I pictures are drawn, or every 5 frame is drawn, or there is a pseudo-random draw: in general, the drawing mechanism is a pseudo-random number generator

N=PsRandFRAME(Kf,N-1 ,P) that depends on N- 1 (the order number of the previous W-frame among all W-frame or within a subset of the W-frames, on a cryptographic key Kf for the frame drawing, and possibly on a set P of other parameters such as:

^• The intensity of the colors of successive frames,

^• Or the nature (I,P, or B picture) of the frame,

^• Or a distribution of successive gaps among W-frames etc, ...

This is general enough since we can consider a deterministic sampler to be pseudorandom, (with a trivial probability distribution) and no key as special case of a choice of a key. It may occur that a frame drawn as a candidate W-frame has a uniform color that might make any mark too visible, or it may be the case that there would be no possible W- zone available because of the coverage of that frame by neighborhoods: anyway that IS-WTMRK-PCT frame expected to be a W-frame is not used as such and the candidate W-frame is not a

W-frame, while most candidate W-frames are also W-frames.

[0067] Having so illustrated some means to draw the successive W-zones and some means to deal with the successive draws of successive W-zones in a way that would be sufficient for anyone versed in the arts of video processing and mathematics, we nevertheless describe next the algorithms that have been suggested, distinguishing what needs to be done depending on whether the W-frames are chosen in an uncompressed or compressed format.

[0068] We need to emphasize here that (some or all of) the F-visible watermarks to be used according to the present invention can also be incorporated as analog signal if one prefers to deal with an uncompressed analog video signal (then some further processing can be done in a digitize format if needed).

[0069] This being said we now consider successively the cases of uncompressed and compressed videos, where compression is supposed to resemble somehow MPEG-2. Here the resemblance is defined by the fact that what we assume of the compression method in the description of the invention for the post compression case makes sense for the compression being used. By convention, here and in what follows, watermarking at compression time is considered as a particular case of post-compression watermarking: this convention is inspired by the technological similarities and the fact that watermarking after compression may induce some further compression steps to finish-up the overall or watermarking after compression.

[0070] With reference now to Figure 7, we see at 701 the initialization of the number N corresponding to the frame to be considered (drawn as N=PsRandFRAME(Kf,N-1,P), as discussed previously). Then, for a counter U that will serve in the (zone) drawing process IS-WTMRK-PCT

PsRand(K.U) that depends on the number of draws U and on a key K that may be secret or be a publicly known number (if K is a publicly known number, there will be no secrecy on the zones once the W-frames are known, which is something that we do not recommend). That drawing process, as was said, can be very simple, such as just PsRand(K,U)=U mod M, i.e., counting modulo the number M of zones (hence 66 for the VHS format for the size of zones that we have used so far as an example with no intent of limitation), or modulo the number M of zones in some set of zones as in Figure 2 where two sets of zone are displayed, each with three groups with M=11. In such cases where there are sets of possible W-zones (like the black and grey marked zones in Figures 2 and

3) that can carry marks from several sets EW1, EW2 a process as illustrated by Figure

7 or by Figure 9 will be run as many times as there are sets of zones dedicated to sets S of marks EWi, EWj,... that can carry W-zones on the same frame. Of course counting may not be a good choice but there are many drawing processes (Pseudorandom Number Generators or PRNGs) available and documented on the Internet, some of which are considered as cryptographically secure (Cryptographically Secure Pseudorandom Number Generators or CSPRNGs): for references, see the discussion of CSPRNGs below. One can also cut the set of W-frames into subsets where different arithmetic means are used, and one reach some secrecy by keeping secret this repartition into subsets.

[0071] Coming back to Figure 7, the index N of the frame is increased at 702: as was suggested previously, N is not necessarily the number of the frame but can be the number among an ordered set of frames, called W-frames, that have been selected as candidates to carry W-zones (a set that was said to be either public or secret and depending on the key Kf (that was discussed above) that may be the same as K or different). Then at 203, the counter U is increased by 1 and one can enter the drawing algorithm PsRand (K₁U) at 710. One verifies at 711 whether the zone Z (X,N) with number X in frame N₁ that is a candidate W-zone, has already been considered for this value of N. If the answer is YES at 715, one gets back to the U-adder at 703. If the answer is NO one follows to 716 after IS-WTMRK-PCT which one checks at 720 whether or not N is bigger than the number P of steps during which a previously drawn W-zone keeps an active forbidden neighborhood of some shape as illustrated in non-limitative examples in Figures 4 to 6: in Figure 4 and 5 we chose P=3 while we had an example with P=5 in Figure 6. The value of P will often be chosen bigger when dealing with frames bigger than the VHS frame. In the flow chart in Figure 7, the effective number of zones whose neighborhoods are considered is denoted by Q: we need another name than "P" since the number P can only be used as the number of neighborhoods being used to reject choices of X when N is large enough (in fact when N>P). [0072] Coming back to Figure 7 and the test:

N>P? at 720 (i.e., one asks there: "is N>P or not?"), if the answer is NO at 721 , the effective number Q of zones whose neighborhoods are considered is set to be N-1 at 723. Then at 730, one tests whether Q>0.

- If the answer is NO at 731 , then N=1 and one sets R (1)=X at 799 to get back to the adder 702: notice that R(N) denotes the name of the W-zone chosen at W-frame N.

- If the answer is YES at 732, then one initializes an index counter I=O at 734, and then start to increase I in the adder by 1 at 736 (I acts by defining with N the value of the quantity N-I). This gets to a further test box at 740 where one tests if

"t>Q?-, i.e., one asks: "is b>Q or not?".

- If the answer is NO at 741 , we test if the draw value X is out of the forbidden neighborhood of the previous W-zone R(N-I) on W-frame N-I by asking:

"da(Z{X,N>, R(N-I)) > (I) ?" at 750. If the answer to that is NO at 751 , one gets back to the U-adder at 703: the previous value X proposed at 710 is indeed refused by the test at 750. If the answer is YES at 752, one gets to the former W-zone by adding 1 to I, back at 752. IS-WTMRK-PCΓ

- If the answer to the question at 740 is YES at 742, then all previous W-frames neighborhoods that need to be considered have been considered with no one of them containing the proposed new W-zone X for W-frame N. Thus we set R(N)=X at 799 and proceed at 702 to the N-adder.

Coming back to the test "N>P?" at 720, if the answer is YES at 722, then one is in the general case, and the effective number Q of zones whose neighborhoods are considered is set to be P at 724. From there one initializes the l-loop at 734, beginning a piece of path that we have already discussed when Q was smaller.

[0073] Before we consider the case when the watermarking according to the present invention is made on a compressed movie, we recall some basic concepts from MPEG-2- like compression techniques. We try to be general enough in recalling basics from compression to cover the compression techniques that can be handled by the invention applied after such compression (one could always implement the invention on the uncompressed movie, and even in analog form as was mentioned, but this may cost more time or space, especially at the coder and watermarking end). We use here as a basic reference the paper MPEG-2 VIDEO COMPRESSION by P.N. Tudor that can be found (in January of 2006) at: • (http7/www.bbc.co.uk/rd/pubs/papersypaper_14/paper_14.shtml).

[0074] Referring to this paper as a handy short summary of MPEG-2, and given the fact that experts in video know such matter well beyond the rudiments that we recall here, we recall some basic vocabulary to keep the text readable by others and to fix some notation and terminology. Thus we start this summary at the level of defining Picture types. IS-WTMRK-PCT

PICTURE TYPES.

[0075] In MPEG-2 and other compression schemes such that the invention can be implemented after compression, three so-called picture types are defined. The picture type defines which prediction modes (including «no prediction*) may be used to code each macro-block (blocks of the size that is used as elementary unit for compression as it is done for still image, e.g., 8x8 pixels blocks in the case of MPEG-2 that uses DCT on blocks).

[0076] «lntra pictures'_* (l-pictures) are the frames compressed without reference to other frames, otherwise speaking all the macro-blocks of l-pictures are intra-coded (with DCT (Discrete Cosine Transform) in the case of MPEG-2 but other transforms could be used such as wavelet transforms), l-pictures can be used periodically to provide access points in the bit stream where decoding can begin, or used in a non-periodic way: there will be some positive density of l-pictures. One can choose to apply the invention only to some or all I- pictures, but we will see that this restriction on the pictures being used is not a necessity.

[0077] "Predictive pictures» (P-pictures) are frames whose compression can use the previous l-picture or P-pictures for motion compensation and may be used as a reference for further prediction. Each block in a P-picture either can be predicted using former frames or intra-coded.

[0078] «Bidirectionally-predictive» pictures (B-pictures) are frames whose compression can use the previous and next I- or P-pictures for motion-compensation, and offer the highest degree of compression. Each block in a B-picture can be forward, backward or bidirectionally predicted or intra-coded. IS-WTMRK-PCΓ

[0079] Since there are intra-coded blocks in P-pictures and B-pictures, one may have zones, filled with such blocks, which can be candidates to be W-zones: an inventory of which P-pictures and B-pictures contain candidate W-zones, and of the candidate W-zones in such pictures has to be established in the case when one does not want to use only I frames to carry W-zones when implementing the invention after compression. The invention can also be implemented by using blocks that are not intra-coded: one then distorts the predictions that would otherwise be made to make this blocks imitate best the original movie. For definiteness, we shall mostly assume that only intra-coded zones (zones whose blocks are all intra-coded) are used, but people versed in the art of video compression will know how to use other blocks as well, along the ways that we have indicated.

[0080] The different picture types typically occur in sequences, called "Groups of Pictures" or GOPs: a typical example is displayed in Figure 8 with the GOP IPPBPPBPPI. GOP's can be regular, but neither MPEG-2 nor the use of our invention after compression requires that. In fact, it is for instance better to have an l-picture at each scene change: a scene change is hardly well compressed as a P-picture. By reducing both spatial and temporal redundancies, P-pictures offer increased compression rates (by a factor about 3 in MPEG-2) as compared to l-pictures, and the compression rate is even about twice as strong for B-pictures: hence one cannot expect to have only l-pictures for good compression rates.

[0081] Other compression schemes: The essential point recalled here apply as well to other compression schemes such as for instance (with no intent of limitation):

• MPEG-4: a standard for multimedia applications.

• H. 261 : Developed by CCITT (Consultative Committee for International Telephone and Telegraph) in 1988-1990, designed for videoconferencing, video-telephone applications, etc. IS-WTMRK-PCΓ

* H. 263: This is a new improved standard for low bit-rate video, adopted in March 1996. As H. 261, it uses the transform coding for intra-frames and predictive coding for inter- frames.

[0082] Beside the short review by Tudor that we have mentioned, there are several good books and in particular textbooks on video (and accompanying audio-compression). See for instance:

-The Data Compression Book," by M. Nelson,M&T Books, (1995).

-"Introduction to Data Compression", by K. Sayood, (Morgan Kaufmann Pub, 1996; 2005).

-"Multimedia: Computing, Communications and Applications." by R. Steinmetz and K.

Nahrstedt, (Prentice Hall. 1995).

[0083] With reference now to Figure 9, we see the parallel in compressed format of what was displayed for uncompressed format in Figure 7. The only serious difference when it comes to choose W-zones, is due to the fact that we assume here that only intra-coded frame can carry marks as defined by the invention. We have mentioned above that such a restriction on the coding of the zones being used is not necessary: it is an easier case, and the case that will be discussed most, leaving out the details of what to do otherwise, details that would be easily filled in a variety of ways by people versed in the art of video processing and in particular video compression: basically in other blocks that are treated using motion vectors, one would decrease or increase some of these vectors to deform the resulting decoded image in the corresponding zones.

[0084] If all zones designated are declared to be intra-coded (which requires the watermarking to be integrated with the compression scheme, possibly a bad choice), or if one only uses l-pictures as W-frames, then one uses in effect the flow chart of Figure 7: Figure 9 comprises the extra-complication that is due to the fact that zones needs to be checked to be intra-coded (at 8100) and one also needs to check (at 8108 and 8020) if IS-WTMRK-PCT there remains possibilities for W-zones in the frame under consideration: this all is done using the list that is built at 8001 of the intra-codβd zones in all frames (or at least in all the W-frames). The rest of Figure 9 works like in Figure 7: the flow chart in Figure 7 is a strict subset of the flow chart in Figure 9, and we have kept the same number for similar flow functions.

[0085] In the comments of Figures 4 to 7 and 9, we have alluded to the fact that the set of W-frames, and the drawing function can be both random, and securely so. The techniques to achieve that sort of security are well known by people versed in the art of cryptography. A pseudorandom number generator (PRNG) is an algorithm that generates a sequence of numbers, the elements of which are approximately independent of each other. A Cryptographically Secure Pseudo-Random Number Generator (CSPRNG) is a (PRNG) with properties that make it suitable for use in cryptography. A number of CSPRNGs have been standardized. They can be found in:

FIPS 186-2

ANSI X9.17-1985 Appendix C

ANSI X9.31 -1998 Appendix A.2.4

ANSI X9.62-1998 Annex A.4

FIPS stands for Federal Information Processing Standards, the NIST (National Institute for Standards and Technology) computer security, ANSI stands for American National Standards Institute.

A good reference is maintained by NIST.

See also « Practical Cryptography, » by N. Ferguson and B. Schneier (Wiley ; 2003.) ISBN 0-471-22357-3.

[0086] One can use CSPRNGs to generate: IS-WTMRK-PCΓ

-the set of possible W-frames (the frames that are W-frames if no reason such that uniform color or a full covering by forbidden neighborhood of previously used W- zones prevent these frames from being indeed W-frames) and/or the set of W-zones in W-frames.

Such uses of CSPRNGs may be preferred to using known successions of frames or zones to make the localization of human readable marks on still frames more difficult. The difference of time generated by all sorts of slowing-down mechanisms such as this one would be particularly significant for documents that keep a high price only for a short time. In fact, even the price of premium movies drops fast in modem economy, but there, the big revenue is expected on the multitude of elements such as DVD or rights to many small TV stations, so that the marks should resist longer. The way our invention could help in the market of movies that get sold as DVD is that the visibility of marks in the frame-by-frame mode could be allowed to be known so that at least some honest people would help track anomalies: it would be important then to have marks that also indicate the intended use of a copy and/or a serial number (possibly in coded form) so that people would know that the copy that they have in their hands might be marked as being illegitimate, and possibly carry a unique serial number that has been associated by some means as for instance with cryptographic security, to one legitimate customer.

[0087] We will not get here in a full discussion of DRM (Digital Rights Managements) as this would be too far from the scope of this invention. However, we notice that by tuning the visibility parameter, the invention can be used to generate movies marked in a visible way, objects that can be used in many aspects of DRM : one further interest of the present invention in this respect is that one may prefer to have a single watermarking technique that generates all or most needed watermarks (from visible to invisible through F-visible which is the main novelty here) by varying one or another small number of parameters. IS-WTMRK-PCT

[0088] It may seem from the comments of Figure 9 that only intra-coded zones are considered to carry F-visible watermarks according to the present invention. While this is the choice that we have presented so far except for some comments, we recall that people versed in the art of video compression, by supervising and modifying slightly what regular compression encoders would do to some movement vectors, can easily generate F-visible effects in non-intra-coded zones. People versed in the art of image processing and video compression will readily understand how one would modify what was presented so far by allowing some non-intra coded zones as W-zones, or even by allowing only non-intra coded zones as W-zones (for instance only blocks that are not referred to by other frames, but this may help locating the W-zones as a subset of the zones with blocks not referred to).

[0089] We propose here to revisit the main function of watermarking that we were aiming at: the proof of ownership. The way the invention would be used is that watermarked version lwi would be released for public viewing, and the non-marked version I would be kept securely (in many copies for more redundancies, but not too much to have a strict access control). One expects that assuming that the watermarks are all erased, the movies that can be produced by attackers are visibly not as good in quality as I. Otherwise, this would mean in particular that the compression methods are not good enough, and at least marks on I frames that correspond to scene changes should be hard to clean off the marks, if only those. Using secret and cryptographically secure draws of the W-frames and W-zones would clearly enhance even further the security, and this statement will become even more obvious once we describe the watermarks to be used, which we do next with reference to Figures 10 to 14.

[0090] Thus we proceed to the next element of the invention that concerns what one does in a watermarking zone. Basically, one will have a library of shapes (soft and/or recognizable, as examples with no intent of limitation) or a random generator of such IS-WTMRK-PCΓ shapes that will produce marks (merely shadows in most cases) that are (as was already discussed):

1) Human readable on still frames

2) Non-machine readable to the best of technologies to this effect (as used in machines to deter machine-access to some WWW sites): we are helped here by the fact that an ugly spot is hard to recognize by a machine, harder to machine-read than a hidden number, or a hidden letter, or a hidden word so that the machines are more easily defeated than with what is used for WWW- sites protection.

3) Non-perceptible in the motion picture.

[0091] Examples of images that are hard to identify even when they are not particularly faint have been proposed under the names CAPTCHAS, an acronym for Completely Automated Public Turing Test to Tell Computers and Humans Apart (on the Web at www.captcha.net), by L von Ahn, M. Blum, and J. Langford, see -Telling humans and computers apart automatically, » by L von Ahn, M. Blum, and J. Langford. Commun. ACM, 47(2):5β-60, 2004. (CMU Tech Report CMUCS-02-117, February 2002). A CAPTCHA is a program that can generate and grade tests that most humans can pass, yet current computer programs can't pass. As an example of classical CAPTCHAS, let us mention Gimpy. Gimpy picks seven words out of a dictionary, and renders a distorted image containing the words. Gimpy then presents a test to its user, which consists of the distorted image and the directions: "type three words appearing in the image". Given the types of deformations that gimpy uses, most humans can read three words from the distorted image, while current computer programs could not when Ahn, Blum and Langford started. Since, then Mori and Malik began solving some CAPTCHAS: in "Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA" by G. Mori, and J. Malik. CVPR (2003), Mori and Malik explore object recognition in clutter. They test their object recognition techniques on Gimpy and EZGimpy, examples of visual CAPTCHAs. They have developed IS-WTMRK-PCT efficient methods based on shape context matching that can identify the word in an

EZGimpy image with a success rate of 92%, and the requisite 3 words in a Gimpy image 33% of the time. Contrary to the intended application, such percentages are not deterrent to use CAPTCHAS or similar idea as watermarks according to our invention, inasmuch as we will only use very faint versions of such images.

[0092] In the line of CAPTCHAS, we also notice the work "Pessimal Print: a Reverse Turing Test," by H. S. Baird, A. L Coates, R. Fateman Int'l J. on Document Analysis and Recognition, vol. 5, pp-158-163, (2003).

There, the authors exploit the gap in ability between human and machine vision systems to craft a family of automatic challenges involving print on very noisy backgrounds that tell human and machine users apart via graphical interfaces including Internet browsers. It must be emphasized that the attack on watermarks that would be designed according to our invention would not properly need to "read" faint CAPTCHAS when they are used: it would seem to be enough to guess where the marks are located. However the fact that some writings appear in natural movies would cause dramatic effects if the attack would erase everything that resembles a letter. One could use another basis for our marks such as deformed versions of images that pertain to a movie (e.g., the faces of some of the actors: see below for more in that line of thoughts) to generate faint CAPTCHAS-like entities adapted to a movie being watermarked according to the present invention. Another big difference between the original CAPTCHAS program and the watermarks that we need is that human intervention is possible in our case: for instance the choice of watermarks for a given movie may depend on the title or on the content of the movie. For instance, if car license plates are seen in the movie, the same characters that are used in the plates can be also used to create watermarks by a combination of human and automatic means as used in CAPTCHAS creations. Also, some actors (or sportsmen and sportswomen in sport events movies) animal or monuments seen in the movie or alluded IS-WTMRK-PCT to in comments or any text associated to the movie (be it the title or the title of a manuscript that has inspired the movie) can be the source of watermark design.

[0093] We should also keep in mind that slightly visible (on a frame-by-frame basis) version of a preferred robust invisible watermark can be used as frame-by-frame-visible watermark according to the present invention: since it may help to also use robust invisible watermarks, one could then use purposely similar or purposely different types of watermarks, both as frame-by-frame-visible watermarks and as auxiliary robust invisible watermarks. Using many techniques among the most secure ones at the time of creation of marks on a movie is one very good strategy that we suggest. We will nevertheless present alternate methods with reference to Figures 10 to 13.

[0092] With reference now to Figure 10, we see a schematic view for a second means to create watermarks according to this invention (the first means being the one based on preferred robust invisible watermarks as discussed above).

In panels A and B, we have figured the pixel partitions (at 910) and the DCT-type partitions (at 920) of zones from the VHS format (recall that 66 such zones would cover the full VHS frame) according to the choice in Figures 2 and 3, but other zones shapes and numbers could be preferred, or used on other frames.

In panel A, the spots 930 have rather regular shapes for drawing convenience but they would rather be generated as fractal sets (and in particular not be necessarily connected) as can be done readily by a variety of methods. As a non-limiting example, fractals can easily be obtained by acting on a point P by a sequence of contractions CAj=Cont(Aj,ki) (or another Iterated Function System: see for instance Fractals Everywhere. By M F. Barnsley. (Academic Press Professional, Boston ; 1993.) ISBN 0120790610) that contact to different points Ai with ratios ki. It is well known that if the sum of the absolute values o the ki's is smaller than 1 , the sequences of points that are obtained by acting successively with IS-WTMRK-PCT longer and longer compositions of the contractions converges to a Cantor set (a set that, with enough generality for our purpose here, is closed and bounded in the plane, perfect (no point is isolated), and totally discontinuous (every connected piece is a single point)). Instead of uniform contractions, one can take linear maps that contract with different rates along different direction, and can be composed with rotations and symmetries, or nonlinear contractions: in both cases ki is the defined by the fact that for any two points x, y, one has for such a map fi: d(fi(x),fi(y))≤ki.d(x,y), and in particular for the fixed point Ai of fi: d(fi(x),Ai)≤ki.d(x,Ai). where d stands for the Euclidean distance.

[0093] It is also well known in mathematics that one gets other fractals when relaxing the condition on the ki's. One can either obtain shapes that resemble natural objects or just shadows to be used as frame-by-frame-visible marks: see, e.g.,:

- « Fractals Everywhere," by M. F. Barnsley cited above, as well as

- « Fractal Geometry: Mathematical Foundations and Applications. » By K. Falconer, (West Sussex: John Wiley & Sons, Ltd. ; 2003). ISBN 0470848618

- «The Fractal Geometry of Nature,* by B.B.Mandelbrot, (W.H. Freeman and Co., New York ; 1982) ISBN 0716711869.

[0094] We have used 4 panels, A to D and represented the marks as clearer and clearer at 930, 940, and 950 to indicate that the intensity of coloring has been exaggerated for visibility, as will be the case in Figures 11 to 13 as well. Beside, from panel A to panels B to D, we see that the number of zone has increased because we wanted to now fully cover the zone in panel A to exhibit the fact that many spots are used (or at least one). The final display of intensity of grey coloring in Panel D can be interpreted as follows. One of the base grey level is taken as zero in the sense that: IS-WTMRK-PCT

One darkens

Or lighten, where the grey level is different from the base grey level, with an amount of darkening (respectively lighten) proportional (or in particular equal) to the difference of the grey in the spot being considered and the base grey level.

[0095] With reference now to Figure 11 , one sees that the watermarks attached to the zones do not necessarily have the same shape than the zones, nor even need to fit in the W-zones: the numbers in panels A to C are the same as in Figure 10 to represent that the spots have size of the same order of magnitude as represented in panel A of Figure 10 and till panel C of Figure 11 (however with less visibility). The spots 1030 in panel D of Figure 11 are smaller and many more of them can be used to create the mark, which is here just like a big shapeless entity but that could as well represent recognizable object or even people or other familiar entities.

[0096] With reference now to Figure 12, we see a schematic view for a third means to create watermarks according to this invention, inspired by the CAPTCHAS program as we have recalled it above and related developments: the shapes that we have used are rather simple for drawing convenience and the grey intensities of the examples in panels A and B are to be interpreted as for Figures 10 and 11. At 1101 , 1102, 1103, and 1104, we have used 4 unusual shaped versions of the word Television", two copies per mark. The backgrounds of the marks (which like the rest of the marks come to modify the image as it is before watermarking) are rather regular and normal picture looking in both panel A and B, to the contrary of what one gets in Figure 13 by using the two words versions (1103 and 1104) from panel B of Figure 12, but now with a pseudo-noisy environment 1110 (background and foreground) that is like the mark in panel D of Figure 11. The overall structure of the watermarking process before compression is obvious, once one has the elements that have been explained so far, with reference to Figures 2 to 7 and IS-WTMRK-PCT

10 to 13. The case of implementing the invention on watermarked movies is harder so that we will explain what to do, in the cases when one uses only I pictures as W-f rames in Figure 14, and in the cases when does not one use only I pictures as W-frames in Figure 15.

[0097] With reference now to Figure 14, we see the flow chart for the overall structure of the watermarking process after compression when one uses I pictures only, after:

Preliminary- 1 ) A process to generate F-visible watermarks has been chosen.

Preliminary-2) A process (for instance of the form

N=PsRandFRAME(Kf ,N-1 ,P) as discussed previously) to generate W-frames among the I pictures has been chosen.

Preliminary-3) A set SW of W-frames has been generated using the process in Preliminary-2); the frames in SW will be counted by the index N. Instead of a set SW, what is selected can be a set Swi among a collection SW1, SW2 SWm (of sets that are not necessarily pair-wise disjoint since a frame can carry several marks, some of which may be invisible watermarks).

We notice that "Preliminary- 1 )" may include: B1 -1 ) making an actual human or machine choice of watermarks, B1-2) and/or making a human and/or machine construction of a set of watermarks or of models from which the actual marks will be built B1-2 in turn, is possibly done in a way: i. B1 -2-1 ) that depends on the details of the movie, ii. B1 -2-2) and that is possibly cryptographically secure.

Back to Figure 14, at 702 we have a N-adder that allows one to get to the next W-frame (or to the next W-frame in one of the Swi collections: in the sequel, "the next W-frame" will IS-WTMRK-PCT stand for either "the next W-frame" or "the next W-frame in one of the Swi collections"), once a frame has been treated (there might be a single collection SW as discussed above).

[0098] At 1701 we have the "action of the method to chose the next watermarking region:" that box represents what is accomplished by using the flowchart in Figure 7. Then we go to 2010 where the former F-visible watermark is read from the "last watermark repository" at 2020. Having captured that data, we get to 2100 where one cancels that mark on the next picture if needed: more precisely, if the W-zone Z(N-I) of frame N-1 from the set SW is a zone that is not intra-coded in following frames. On any frame where the zone Z(N-I) is defined by reference to the zone Z(N-I) in frame N-1 of SW, one has to cancel the effect of the mark to be sure that this mark will be projected on only one frame (very few frames instead of one single frame may work also in some cases). After 2100, we get at 2200 to the F-visible watermark library or F-visible watermark generator according to the discussion that we have given of this matter, in particular in conjunction with Figures 10 to 13. Next at 2300 the W-zone (and possibly part of nearby zones as we mentioned in conjunction with Figure 11 ) is marked using the watermark that is Either added to the non watermarked image, or modifies the non watermarked image as one does when watermarks are imposed in the DCT domain, or it defines a modulation of at least one of the luminance and the chrominance of the non-watermarked image.

[0099] All these examples are not meant to be limiting. In all cases the intensity of the marks is tuned so that it is visible under careful frame-by-frame examination but invisible in the movie, according to the teaching of the present invention.

After the watermark is incorporated at 2300 in the chosen part of W-frame N, the mark and all relevant data about how it was incorporated (so that the resulting picture can IS-WTMRK-PCT be reconstituted from the data) are deposited at 2300 in the "last watermark repository" at

2020. After that, one optionally (hence the doted arrows) makes auxiliary changes in nearby frames at 2310: these changes concern both visibility and security motivated actions that have already been alluded to above and will be discussed in detail in conjunction with Figure 16.

[0100] With reference now to Figure 15, we address the same issues that were discussed in conjunction with Figure 14, but now in the case when intra-coded zones can be chosen as W-zones even in P or B pictures rather that only in I pictures in the case of Figure 14. The description is easier and should be better understood in comparison with what was done for Figure 14. First 1801 has replaced 1701 because the chosen method to find the next W-zone is now the one presented in Figure 9 (instead of being the method that corresponds to Figure 7 that was used at 1701 in Figure 14). Next there is at 2050 a possible change of coding mode for the zone to make it intra-coded: we have already indicated that such change would interact with the coding of the compression and should preferably be avoided if simplicity is a parameter one cares about, but the possibility exist and the choice may need to be made on a case by case basis. We have also indicated that one could keep a non-intra-coded block as such and watermark it by distorting what would have been the optimal motion vectors: such difference are minute technical details that should be handled easily by people versed in the art of video compression. Thus, there is the possibility to either only take intra-coded zones, or use other zones to incorporate the watermark that would then be composed by taking pieces from the previous pictures in the P pictures case, and from both past and future pictures in the B pictures case. Although the specification of the watermark is then more delicate, one has the advantage that if the zone under consideration is not relied upon by other frames, then it is not necessary to compensate for the watermark. These differences (replacement by intra-coded zone, or treatment as such of intra-coded zones) motivate the change of numbering from 2100 in Figure 14 to 2101 in Figure 15, as well as the dotted line from 2050 to 2101 and from 2050 IS-WTMRK-PCΓ to 2010, which indicates that no call to 2010 needs to be done if no correction is made in the frame following N-1 if that frame was marked in an inter-coded zone, and more generally in a zone not called upon by other frames.

[0101] The process in Figure 14 or in Figure 15 will be run many times if there are several sets of W-frames that will receive different watermarks; if the same set is made of frames that receive several marks, one can either run the process many times or take care of the watermarking and compensations in following frames by parallel treatment of zones that need not even be disjoint.

[0102] With reference now to Figure 16 we let n stand for the rank among all frames of the frame with rank N among the set SW of W-frames. Then:

- (n+1) stands for the next frame among all frames and not necessarily for the frame

corresponding to the (N+1 ) frame among the set SW of W-frames (the exceptional case when n+1 corresponds to the (N+1 ) frame among the set SW of W-frames is when the

(N+1 ) frame among the set SW of W-frames is consecutive to the N frame among all the frames: this happens for instance when all frames are W-frames).

- (n-1 ) stands for the previous frame among all frames and not for the frame corresponding the (N-1) frame among the set SW of W-frames, (with provisions comparable to what was described just above)

- and so on.

[0103] However, the choice of N is exemplary among all W-frames and any of those frames could have been chosen so that it's order number M, or X represent the same frame as the frame with order number m or x, fixing another correspondence between the entirety of frames and the subset SW. With the choice we have made, there are numbers pN≥1 and fN≥1 such that: IS-WTMRK-PCT

- The frame preceding N among the W-frames be the (n-pN) frame among all the frames and

- The frame following N among the W-frames be the (n+fN) frame among all the frames. These notations are used in panels A and B of Figure 16. Still in Figure 16 at 1610, 1621, 1622, 1631, 1632, and 3254 we see watermarks of different kinds that we will now describe.

First 1610 stands for an instance of one of the frame-by-frame-visible watermarks according to the present invention, as was discussed in connection with Figures 10 to 13. Next 1621 and 1622 are still frame-by-frame-visible but less so than 1610 (in fact the mark would escape most people if not suggested by having seen the mark at 1610): one could have more panels that just one before and one after the panel n (or N among the SW frames) to contain weakened marks of that sort On both ends of the series of weakened marks, one has at 1631 and 1632 robust invisible watermarks of any desired kind, such as for example the marks from literature cited so far. On panel B, the weakened marks have disappeared before n. On the other side, at n+1 we have at 3254 the sum of the marks 1622 and 1632. If one prefers to consider frames with weakened but still perceptible marks, as W-frames, then one would replace the narration for Figure 16 by saying that the set SW contains consecutive blocks of frames where a most visible mark for frame by frame inspection would be accompanied by even less visible marks: there is no clear cut point between almost invisible and very difficult to see even with extended expert examination.

[0104] The description that we have given here with reference to Figure 16 is for non- compressed movies. Anyone versed in the art of image processing would understand, using also the teaching provided so far, how to adapt what was presented here for regular frames only to the combination of I, P, or B pictures. In fact the main difference is that cancellation must be used after a W-frame that is referred to by P or B-frames, as was already pointed out: the main difference is when we have weakened marks in non- IS-WTMRK-PCT compressed format: these may translate into partial erasure in P and B pictures, as anyone versed in the art would understand after the description given so far of the invention.

So the simplest case would not have any frame as accompanying W-frames except for erasure in B and P frames referring to a marked frame, the next level of complexity is in panel B of Figure 16, and the next one in panel A, with next levels corresponding to have more weakly marked panels. So if the compression is not optimal for the quality of the movie one has, it would be more secure to have more weakened frames to better protect against the attack that would consist in suppressing the marked zones and rebuilding them from nearby pictures (and possibly also using the marked image) by methods of image reconstruction, while less weak marks would be needed for that purpose if the compression is very close to optimal. It would be hard however to fully replace all marked zones if their locations are not divulged and the attacker has to inspect carefully the movie to locate them one by one, which provides a good reason to use cryptographically secure pseudo-random number generators to find the W-frames and/or the W-zones in the W- frames.

[0105] With reference now to Figure 17, we see an exemplary schematic view (with no intent of limitation) of a system to implement the watermarking method taught by this invention. The computer system 4001 has one or more CPUs (Central Processing Units) linked to one or more secure cryptographic co-processors at 4021 such as the IBM 4758 PCI Cryptographic Coprocessor or for instance a smart card for lower end implementations. Such a secure unit, doubled at least once at 4022 to protect against a faulty behavior that would mean the loss of the cryptographic keys, would help control the security of all aspects of the watermarking process according to methods that are well- known in the art of computer security. For instance, the secure unit would permit the access of the keys used to watermark all movies or a given movie only if some minimal number of agents collaborate to get to the secret keys using Shamir's secret sharing protocol in case for instance of a dispute on ownership. These secret keys are the ones IS-WTMRK-PCΓ that would be used in the CSPRNG's that generate the list of W-frames and of W-zones in the W-frames as well as the keys to the auxiliary marks used according to the discussion of Figure 16. The chosen CSPRNGs and other cryptographic functions that are chosen can:

- Be stored in Read Only Memory 1 , or ROM 1 , at 4201 ,

- Or, and stored in Random Access Memory 1 , or RAM 1 , at 4301 after being transmitted to the system through some Input/Output devices and communication system such as:

1. the reader 4610 reading from medium 46101 (for instance a DVD),

2. or through a LAN (Local Area Network) at 46200 containing some of the computers of the firm,

3. or from a WAN (Wide Area Network) such as a private network or the Internet at 46300.

[0106] Similarly, and using one of the same sources, the used library of watermarks and/or methods to generate marks (possibly after extracting a template from the movie as taught previously) could be stored in ROM 2 or RAM 2 (that need not be physically distinct from ROM 1 and RAM 1 respectively) with the routines that need to be performed being performed in the CPU at 4011 , possibly with the help and the security protection of the coprocessor at 4021.

[0107] The watermarking process as taught before will be using the process described in figure 7 or 9 according to what type of document gets to be watermarked, stored in ROM 3 or RAM 3 (that need not be physically distinct from ROM 2 and 1 and RAM 2 and 1 respectively) with the routines that need to be performed being performed in the CPU at 4011 , possibly with the help and the security protection of the coprocessor at 4021. IS-WTMRK-PCT

[0108] To apply the watermark on a movie, the movie would first be read for instance (with no intent of limitation):

From a an analog source such that from a movie reel at 46011 read at reader 4601,

From an analog or digital source on a tape 46021 read at 4602, From a digital source on a DVD 46101 read at 4610.

[0109] Then the movie would be digitized if not already digital and optionally compressed before marking using a digitizer or encoder that can be stored in one of the RAMs or ROMs, or further ones, and using the CPU at 4011 , or some other similar unit. The movie can also get the frame-by-frame- visible marks before being digitized, in which case auxiliary treatment such as adding other marks could be done in compressed or uncompressed digital form.

[0110] The unmarked digital copy would be recorded on at least one recorder at 4911, with preferably more than one copy at 49111 kept in a secure way, preferably after being encrypted using the secure coprocessor at 4021. Copies with the mark would be recorded (possibly in mass) on at least one recorder 4901 to 4903, with copies from 49011 to 49031 being distributed. Notice that as well known in the art, the watermarks could comprise a buyer's identification, or an identification of the copy in the form of a serial number (among other marks, some or most of which are the same on all copies) of the copy being made instead of all movies carrying only the same marks.

[0111] We have seen that the invention disclosed here can incorporate previously know methods of watermarking. As a consequence, the invention presented here can help one not only solve the deadlock attack, but can also be used in the same way as one uses said previously know methods of watermarking, including, with no intent on limitation, personalization of copies to protect against illegal sales or re-sales by identification of each IS-WTMRK-PCΓ copy, recognition and counting of usages of the watermarked copies, for instance on the World Wide Web, etc.

[0112] Although this invention has been disclosed in the context of certain preferred embodiments and examples, it will be understood by those skilled in the art that the present invention extends beyond the specifically disclosed embodiments to other alternative embodiments and/or uses of the invention and obvious modifications and equivalents thereof and that changes may be made therein which still fall within the spirit and scope of the invention. Thus, it is intended that the scope of the present invention herein disclosed should not be limited by the particular disclosed embodiments described above, but should be determined by the appended claims.

Claims

IS-WTMRK-PCΓ

Claim 1. A watermarking method for movie pictures that marks some selected zones of some selected frames with marks that are visible in careful frame-by-frame examination conducted by most human agents with normal viewing skills but that are not perceptible during normal viewing of the marked movies, nor easily recognizable as marks by machines such as computers.

Claim 2. A watermarking method for movie pictures according to claim 1 implemented on the video signal in analog or digital form, before it gets compressed. Claim 3. A watermarking method for movie pictures according to claim 1 implemented on the digital video signal after it gets compressed.

Claim 4. A watermarking method for movie pictures according to claim 3 where the compression method is in any of the MPEG standards.

Claim 5. A watermarking method for movie pictures according to claim 1 implemented on the video signal so that at least one of the frame carrying the watermarks and the zones that carry the watermarks in the frames carrying the watermarks is produced by a system that is kept secret.

Claim 6. A watermarking method for movie pictures according to claim 1 that also employs invisible robust watermarks to help protect the overall watermarking scheme against attacks.

Claim 7. A watermarking method for movie pictures according to claim 1 that also employs fainter robust watermarks that are hard to recognize by humans in a frame-by-frame examination, placed in at least one of the positions after or before the robust watermarks as in claim 1 that are hard but easier that the fainter ones to recognize by humans in a frame-by-frame examination, to help protect the overall watermarking scheme against attacks.

Claim 8. A watermarking method for movie pictures according to claim 1 , implemented on the video signal so that the zones that carry the watermarks in the frames carrying the IS-WTMRK-PCΓ watermarks is produced by a system that is possibly kept secret, whereby tentative positions for the successive zones are either computed or drawn, and then checked to be out of predefined neighborhoods of zones that have recently been used as watermark zones.

Claim 9. A watermarking method for movie pictures according to claim 8 implemented on the uncompressed video signal.

Claim 10. A watermarking method for movie pictures according to claim 8 implemented on the compressed video signal.

Claim 11. A watermarking method for movie pictures according to claim 1 , used to establish who is the true owner of a document and defeating the deadlock attack without the need of imposing a type of watermarks such as reputed non-invertible watermarks to the adversary parties.

Claim 12. A watermarking method for movie pictures according to claim 1, used to establish who is the true owner of a document and defeating the deadlock attack without the need of having recourse to a trusted third party.

Claim 13. A watermarking method for movie pictures according to claim 1, used to establish who is the true owner of a document and marking in a convincing way, without the need for the jury and judges in a dispute case to have to trust at least one of a complicated examination by machines whose detailed functioning would need technological expertise, or examination using some algorithm in a way that would need to be trusted and whose mastery would require mathematical kills.

Claim 14. A watermarking method for movie pictures according to claim 1, implemented so that the system can benefit from progresses in generating images that are hard to recognize or even to detect by a computer but can be recognized by careful frame by frame examination.

Claim 15. A watermarking method for movie pictures according to claim 1 , where some or all of the hard to see watermarks that are invisible during normal viewing of the movies are obtained by repeating faint versions of possibly distorted versions of elements that are part IS-WTMRK-PCΓ of what is shown in the movie, such a numbers, letters, actor faces, or anything else that may be shown in the movie.

Claim 16. A watermarking method for movie pictures according to claim 1 , where in non- compressed format, several or all images will carry of few strange or recognizable shaped zones with luminance and/or chrominance excess or default, taking care that the zones where such spots appear do not repeat too frequently.

Claim 17. A watermarking method for movie pictures according to claim 1, where, in compressed formats such as MPEG formats (without limitation on the high quality compression method), these modifications on the non-compressed format would transport by themselves through compression from imposing the watermarks on the analog or the digital uncompressed format, or a combination of these two forms of marking. Claim 18. A watermarking method for movie pictures according to claim 1 , where one adds the watermark at compression 8 time. In MPEG-2 or MPEG-4 for instance, one distinguish 3 types of frames, called l-pictures, P-pictures, and B-pictures, the first one with no reference to other frames, the second ones computed from previous frames, and the last ones computed from previous and following frames. Among all frames, only the l-pictures would not need to contain the erasure of what is done on the previous frame. One can thus:

Either watermark many or all I, P, and B frames, taking care of all erasures (or other needed form of visual correction) on the following frame,

Or only watermark the frames that come just before I frames, Or only watermark some or all l-pictures.

Claim 19. The method of watermarking of claim 1 furthermore allowing current uses of invisible robust watermarks including personalization of copies to protect against illegal sales or re-sales by identification of each copy and recognition and counting of usages of the watermarked copies, for instance on the World Wide Web, etc. Claim 20. A computer system to create watermarks on movies comprising: IS-WTMRK-PCT

Means to read an analog or digital unmarked movie,

Means to watermark at feast one of the uncompressed analog movie or the uncompressed digital movie or the compressed movie including:

At least one of a library and system generating watermarks that are visible in careful frame-by-frame examination but not perceptible in normal movie viewing conditions,

At least one of an arithmetic algorithm and a pseudo-random system to choose the W-frames and W-zones on the w-frames,

- Means to implement such choices in a cryptographically secure way,

- Means to identify buyers,

- Means to identify copies, and

- Means to perform all identifications in a cryptographically secure way.

Claim 22. A system to watermark movie pictures according to claim 21 such that the marking is implemented on the video signal in analog or digital form before it gets compressed.

Claim 23. A system to watermark movie pictures according to claim 21 such that the marking is implemented on the video signal in analog or digital form after it gets compressed.

Claim 24. A system to watermark movie pictures according to claim 23 where the compression method is in any of the MPEG standards.

Claim 25 A system to watermark movie pictures according to claim 21 implemented on the video signal so that at least one of the frame carrying the watermarks and the zones that carry the watermarks in the frames carrying the watermarks is produced by a system that is kept secret.

Claim 26. A system to watermark movie pictures according to claim 21 that also employs invisible robust watermarks to help protect the overall watermarking scheme against attacks. IS-WTMRK-PCT

Claim 27. A system to watermark movie pictures according to claim 21 that also employs fainter robust watermarks that are hard to recognize by humans in a frame-by-frame examination, placed in at least one of the positions after or before the robust watermarks as in claim 1 that are hard but easier that the fainter ones to recognize by humans in a frame-by-frame examination, to help protect the overall watermarking scheme against attacks.

Claim 28. A system to watermark movie pictures according to claim 21, implemented on the video signal so that the zones that carry the watermarks in the frames carrying the watermarks is produced by a system that is possibly kept secret, whereby tentative positions for the successive zones are either computed or drawn, and then checked to be out of predefined neighborhoods of zones that have recently been used as watermark zones.

Claim 29. A system to watermark movie pictures according to claim 28 implemented on the uncompressed video signal.

Claim 30. A system to watermark movie pictures according to claim 28 implemented on the compressed video signal.

Claim 31. A system to watermark movie pictures according to claim 21 , used to establish who is the true owner of a document and defeating the deadlock attack without the need of imposing a type of watermarks such as reputed non-invertible watermarks to the adversary parties.

Claim 32. A system to watermark movie pictures according to claim 21 , used to establish who is the true owner of a document and defeating the deadlock attack without the need of having recourse to a trusted third party.

Claim 33 A system to watermark movie pictures according to claim 21, used to establish who is the true owner of a document and marking in a convincing way, without the need for the jury and judges in a dispute case to have to trust at least one of a complicated examination by machines whose detailed functioning would need technological expertise, IS-WTMRK-PCT or examination using some algorithm in a way that would need to be trusted and whose mastery would require mathematical kills.

Claim 34. A system to watermark movie pictures according to claim 21 , implemented so that the system can benefit from progresses in generating images that are hard to recognize or even to detect by a computer but can be recognized by careful frame by frame examination.

Claim 35 A system to watermark movie pictures according to claim 21, where some or all of the hard to see watermarks that are invisible during normal viewing of the movies are obtained by repeating faint versions of possibly distorted versions of elements that are part of what is shown in the movie, such a numbers, letters, actor faces, or anything else that may be shown in the movie.

Claim 36. A system to watermark movie pictures according to claim 21, where in non- compressed format, several or all images will carry of few strange or recognizable shaped zones with luminance and/or chrominance excess or default, taking care that the zones where such spots appear do not repeat too frequently.

Claim 37. A system to watermark movie pictures according to claim 21, where, in compressed formats such as MPEG formats (without limitation on the high quality compression method), these modifications on the non-compressed format would transport by themselves through compression from imposing the watermarks on the analog or the digital uncompressed format, or a combination of these two forms of marking. Claim 38. A system to watermark movie pictures according to claim 21 , where one adds the watermark at compression 8 time. In MPEG-2 or MPEG-4 for instance, one distinguish 3 types of frames, called l-pictures, P-pictures, and B-pictures, the first one with no reference to other frames, the second ones computed from previous frames, and the last ones computed from previous and following frames. Among all frames, only the l-pictures would not need to contain the erasure of what is done on the previous frame. One can thus: IS-WTMRK-PCT

Either watermark many or all I, P, and B frames, taking care of all erasures (or other needed form of visual correction) on the following frame,

Or only watermark the frames that come just before I frames, Or only watermark some or all l-pictures.

Claim 39. A computer readable medium encoding instruction for performing a watermarking method for movie pictures comprising: marking some selected zones of some selected frames with marks that are visible in careful frame-by-frame examination conducted by most human agents with normal viewing skills but that are not perceptible during normal viewing of the marked movies, nor easily recognizable as marks by machines such as computers. Claim 40. . The computer readable medium of claim 39 where said watermarking method is implemented on the video signal in analog or digital form, before it gets compressed. Claim 41. The computer readable medium of claim 39 where said watermarking method is implemented on the digital video signal after it gets compressed.

Claim 42. The computer readable medium of claim 41 where the compression method is in any of the MPEG standards.

Claim 43. The computer readable medium of claim 39 where said watermarking method is implemented on the video signal so that at least one of the frame carrying the watermarks and the zones that carry the watermarks in the frames carrying the watermarks is produced by a system that is kept secret.

Claim 44. The computer readable medium of claim 39 where said watermarking method also employs invisible robust watermarks to help protect the overall watermarking scheme against attacks.

Claim 45. The computer readable medium of claim 39 where said watermarking method also employs fainter robust watermarks that are hard to recognize by humans in a frame- by-frame examination, placed in at least one of the positions after or before the robust watermarks as in claim 1 that are hard but easier that the fainter ones to recognize by IS-WTMRK-PCT humans in a frame-by-frame examination, to help protect the overall watermarking scheme against attacks.

Claim 46. The computer readable medium of claim 39 where said watermarking method is implemented on the video signal so that the zones that carry the watermarks in the frames carrying the watermarks is produced by a system that is possibly kept secret, whereby tentative positions for the successive zones are either computed or drawn, and then checked to be out of predefined neighborhoods of zones that have recently been used as watermark zones.

Claim 47. The computer readable medium of claim 39 where said watermarking method is implemented on the uncompressed video signal.

Claim 48. The computer readable medium of claim 39 where said watermarking method is implemented on the compressed video signal.

Claim 49. The computer readable medium of claim 39 where said watermarking method is used to establish who is the true owner of a document and defeating the deadlock attack without the need of imposing a type of watermarks such as reputed non-invertible watermarks to the adversary parties.

Claim 50. The computer readable medium of claim 39 where said watermarking method is used to establish who is the true owner of a document and defeating the deadlock attack without the need of having recourse to a trusted third party.

Claim 51. The computer readable medium of claim 39 where said watermarking method is used to establish who is the true owner of a document and marking in a convincing way, without the need for the jury and judges in a dispute case to have to trust at least one of a complicated examination by machines whose detailed functioning would need technological expertise, or examination using some algorithm in a way that would need to be trusted and whose mastery would require mathematical kills.

Claim 52. The computer readable medium of claim 39 where one adds the watermark at compression 8 times. In MPEG-2 or MPEG-4 for instance, one distinguish 3 types of frames, called l-pictures, P-pictures, and B-pictures, the first one with no reference to other IS-WTMRK-PCT frames, the second ones computed from previous frames, and the last ones computed from previous and following frames. Among all frames, only the l-pictures would not need to contain the erasure of what is done on the previous frame. One can thus:

Either watermark many or all I, P, and B frames, taking care of all erasures (or other needed form of visual correction) on the following frame,

Or only watermark the frames that come just before I frames,

Or only watermark some or all l-pictures.

Claim 53. The computer readable medium of claim 39 furthermore allowing current uses of invisible robust watermarks including personalization of copies to protect against illegal sales or re-sales by identification of each copy and recognition and counting of usages of the watermarked copies, for instance on the World Wide Web, etc.