CA2108258A1 - Digital image compositing system and method - Google Patents

Abstract

A digital image compositing system (20) has background, foreground, key and mask busses (22, 24, 26 and 28). An analog key input (30) is connected to the key and mask busses (26 and 28) through an analog/digital (A/D) converter (32) by crosspoint switches (34). Four digital D1 video signal inputs (36) are connected to the busses (22-24) through additional crosspoint switches (34). The key and mask busses (26) and (28) are respectively connected to frame stores (38) and (40). A key processing toolkit subsystem (42) is connected to the output of the frame stores (38) and (40). A
compositor subsystem (44) is connected to the background and foreground busses (22) and (24) and to the output of the key processing toolkit subsystem (42). A D1 program output (46) is connected to the output of the compositor subsystem (44) on program bus (48). A D1 key output (50) is connected to the output of key processing toolkit subsystem (42) on processed key bus (52). A D1 auxiliary output (54) is connected to program bus (48), background bus (22), foreground bus (24), processed key bus (52) and mask bus (28) by crosspoint switches (56).

Description

~lu~ 9~l0284 9 210~2~38 DIGITAL IMAGE COMPOSITING SYSTEM ~ND METHOD

BAC~GROUND OF T~ INVENTION
1~ Field o~ the Invention:
The presen~ invention relate~ generally to digital compositing, i.e., the lay~ring or putting together of images in a digital video domain. More particularly, it relates to a system and method for digital image compositing which gives th~ user an incr~ased ~lexibility for such compositing. The invention further relates ~o a us~r interface that enables the us~r to take advantage of the increased flexibility more conveniently.

A variety of systems are known in th~ art ~or combining separate video im~ges using a key to provide a foreground image superimposed o~ a background~ For example, such gystem~ ar~ disclosed in ~reenbu~g, U.5.
Patent 4 , 357 ~ 624 and K~llar et al., U~S. Patent 4,602,286~ It is further known to carry out various forms of so~t edge keying to provide a reali~tic transition between the foreground image and the background image. One ~,orm of soft edge ~eying is disclosed in the Kellar et al. patent, and in Bennett et al., U.S. Patent 4,463,373. Other form of ~oft edge kPying are di closed in the following issued U.K patents and published U.K. patent applications: ~,178,330;
1,226,559; 1,570,773; 2,032,217; 2,092,346 and 2,109,193.
.: _ WV92/18937 PCT/US92/02~4~
21 0 ~Z~8 -2-In particular, one technique for soft edge keying that has particular utility is the use of sub-pixel interpolation, as disclosed in published European application 0 360 559.
Other techniques disclosed in the prior art that are useful in digital video image compositing include clock rate doubling and adaptive decimation, as disclosed in published European application 0 360 557, the use of modified keying signals to adjust boundaries between overlaid images, as disclosed in published European application n 360 560, and the use of dynamic rounding, as disclosed in Owen, D.P., "Dynamic Rounding in Digital Video Processing," SMPTE Journal, June 1989, pp. 447-450.
Various forms of control interfaces for video signal processing equipment are also known in the art. For example, such interfaces are disclosed in U.S. Patents 4,538,188; ~,763,186; 4,857,994 and 4,954,883, While the art pertaining to digital video image compositing is thus a well de~eloped one, a need still remains ~or further improvement in such systems in order to provide improved composite images and to make such equipment easier to operate.
~N~RY OF ~E INVBNTION
: Accordingly, it is an ob~ect of this invention to proYide a digital image compositing system and method incorporating flexible input and output crosspoint : switching.
It is another object of the in~ention to provide such a digital imagQ compositing system and method in which digital inputs and outputs are provided for all video signals.
It is a further object of the invention to provide such a digital imaye compositing system and method which will accept analog key or mask inpu~ signals.

r'~ ~

2 ~ PCT/US92/02~g It is still another object of the invention to provide such a digital image compositing system and method which will additionally provide any output video signal in analog form.
It is a still further object of the invention to provide such a digital image compositing system and method which can be cascaded to provide real time multiple layer compositing.
It is yet another object of the invention to provide such a digital image compositing system and method which incorporates a user interface that facilitates use of the flexibility provided by the system and method.
The attainment of these and related objects may be achieved through use of the novel digital image compositing system and method herein disclosed. A
digital image compositing system in accordance with this invention hasa plurality of digital video signal inputs, a key processing subsystem an~ a video image compositor.
A first plurality of crosspoint switches connect the plurality of digital video signal inputs to the key processing subsystem and to the video image compositor~
A key input is connected through the pl~rality of crosspoint switches to the key processing subsystem.
A second plurality of crosspoint switches connect the key processing subsystem and the video image compositor to~a plurality of digital video signal outputs.
The~attainment of the foregoing and related objects, advantages and ~eatures of thP invention should be more readily apparent to those skille~ in the art, after review of the following more detailed description of the invention, taken together with th~ drawings, in which:

, BRIEF DESCRIPTION OF T~E DRAWIN~S
Fi~ure l is a block diagram of a digital image compositing system in accordance with the invention.

WO92/18~37 PCT/US92/02849 ~ s a block diagram of an editing ~ystem incorporating the digital image composition system of Figure 1.
Figure 3 is a block diagram of a portion of the digital image compositing system of Figure 1.
Figures 4A-4D are graphs showing input output characteristics from the system portion of Figure 3.
Figure ~ is a diagram showing relations between video images and selective keying.
Figures 6A-6E are probability distribution diagrams showing different forms of signal processing in the digital image compositing system of Figure 1.
Figure 7 is a block diagram of a video editing system incorporating a plurality of the digital image compositing systems of Figure 1.
Figure 8 is a signal flow diagram useful for understanding operation of the digital image compositing system of Figure 1.
Figure 9 is a more detailed signal flow diagram of a portion of the signal flow diagram of Figure 8.
Figure 10 is a signal flow diagram representing a simplification of the signal flow portion shown in Figure 9.
Figures 11A-llL are charts showing inputs to and outputs from.the signal flow diagram of Figure lO.
Figures 12A-12G are diagrams of input output characteristics from a portion of the signal flow diagram of Figure 9.
Figure 13 is a block diagram of another portion of the digital image compositing system of Figure 1.
Figures 14A-14B are frequency spectra diagrams useful for understanding operation of the syst~m portion of Figure 13. ~

:

W092/18937 PCT/~92/02~9 -5- 2~ g 2S ~
Figure 15 is a signal ~low diagram of another embodiment of the digital image compositing system of Figure 1.
Figure 16 is a plan view of a Xeyboard and display for use with the digital image compositing system of Figures 1 and 14.

DETAI~ED DESCRIPTION OF T~E INVENTION
Turning now to the drawings, more particularly to Figure 1, there is ~hown a digital image compositing system 20 having background, foreground, key and mask busses 22, 2~, 26, and 28. An analog key input 30 is connected ~o the key and mask busses 26 and 28 through an analog/digital (A/D) converter 32 by crosspoint switches 34. Four digital Dl video signal inputs 36 are connected to the busses 22-2~ through additional crosspoint switches 34. The key a~d mask busses 26 and 28 are respectiveIy connected to frame stores 38 and 40.
key processing toolkit subsystem 42 is connected to the output of the frame stor~s 38 and 40. A compositor subsystem 44 is co~nected to the background and foreground busses 22 and 24 and to ~he output o~ th~ key : pro~essing toolkit subsystem 42. A Dl~program output 46 is connected tQ the ou~put of the compositor subsystem 44 on program bus 48. A D1 key output 50 i5 connected to the;output of key processing toolkit subsystem 42 on processed key:~bus 52. A D1 auxiliary oukput 54 is connected to program bus 48, background bus 22, - foreground bus 24, processed key bus ~2 and masX bus 28 by crosspoint switches 56. An analog output 58 is also : 30 connected to the busses 48, 22, 24, 52 and 28 by additional crosspoint switches 56 through D/A converter : 60. A microprocessor 62 is connected to provide control signals for all of the functional elements of the system 20. To avoid~ confusion in the drawing, the control W092/18937 PCT/~92/02~9 2~L0',~'~,58 6-connections are not shown. A control keyboard and display 64 is connected to the microprocessor at 660 The microprocessor 62 is also connected to a video editor at 68. The microprocessor 62 is desirably implemented with an NEC V50 microp~ocessor integrated circuit.
Under control of the microprocessvr 62, the sys~em 20 is able to accept Dl digital video signals at ~he inputs 36 and provide Dl digital video signals at the outputs 46, 54 and 58 in a plurality of video standards, for example, in either 525-line NTSC standard ~orm or 625-line PAL standard form. The Dl digital video ~ignal standard is contained in Dl Standard CCIR Recommendations Recommendation 601~ The 525-line NTSC Dl digital video signal implementation is described in SMPTE Recommended Practice RPl25-l984. The corresponding 625-line PAL Dl digital video signal implementation is described in EBU
Parallel Interface for 625 Line Digital Video Signals, Tech. 3246~E. The system 20 processes the digital video signals at a lO~bit resolution, in order to give better resolution ~han the conventional ~ or 9-bit processi~g.
- Figure2 shows a typical applicationconfiguration, in which the digital image compositing ~y~tem 20 is connected to Dl digital video signal sources 70 and 72 tQ provide he foreground and background inputs 36 and to an analog.video signal source 74 to provide th~ key : input 30. The program output 46 is connected to a digital Dl video tape recorder (VTR) 76, and the analog output 58 is connected to a video monitor 78.
Figures 3-5 show how selective bypas~/keying by timed selection of look ~p tables (LUTs) is implemented in the key processing toolkit subsystem 42 and in the compositor 44. The background input 24 and the foreground input 22 are supplied to a difference circuit 140. The output of the differen~e circuit 140 is supplied to a multiplier 142. The other input to the WO92/18g37 PCT/US92/02849 ~7~ ~ 2 ~ 8 multiplier 142 is supplied from look up table 144, based on the key input 26. The output of the multiplier 142 is supplied to a summing circuit 146. The other input to the summing circuit is the foreground signal 22. The output of the summing circuit 146 is the program signal.
The program signal is obtained in accordance with the following formula:
P~M - FG + (1-K) * (BG - FG) = K * FG ~ K) * BG
10Figures4A-4D show look-up table input/output (I/O) characteristics LUT-l through LUT-4, respecti~ely representing outputs 1.0, K, l-K and 0.0, stored in the look-up table 144. In Figure 5, the foreground, background and key inputs 22, 24 and 26 represent the signal elements of a TV frame. Represented at 148 and 150 are two selections of look-up table I/0 characteristics from the look-up table 144 to produce two different TV frames, each divided up into a top portion 152 and a bottom portion 154, with different look-up table characteristics selected for the portions 152 and 154, timed to produce the portions 152 and 154 during scanning of the TV frame. Shown at 156 and 158 are the resulting program output signals in the portions 152 and 154 for the two TV frames, as calculated ~y the : 25 above f~ormula. In this examplef the timed selection of look-up tables is used to produce different effects in separate areas of a video frame. However, the technique is not limited to separate areas, and can be used to generate a wide variety of interesting video effects.

3:0Figures 6A through 6E show how variable randomiz~d rounding is used to accommodate limited word length in video signal processing by eliminating fractional values.
Rounding in accordance with the conventional rule that fractions equal to one-half or more are round~d up to the next integer and fractions of less than ~ne-half are W092~18937 PCT/US92/02~9 ~ lO~5j~ -8-rounded down to the last integer yields the least variance of error, but the most noticeable false contour in the picture. The probability density ~unction of error for rounding is shown in Figure 6A, and is equal for all error values between -l/2 and ~l/2. Dither, i.e/, randomly increasing any fr~ction to the next digit or decreasing it to the last digit, makes false contour the least noticeable, but yields the largest variance of error by spreading the error distribution from -l to ~l . Variable randomized rounding provides s~lectable tradeoffs between the size of the error and the level of false contour. Depending on the magnitude of the fractional val~es, dithering is selectively applied.
,For example, l. If the fraction is close to 0, then discard it.
2. If the fraction is c~ose to l, then add l to the integer partO
3. If the fraction is close to 0.5, then apply dither.
~0 ~he choice of the width of tha range around 0.5 in which dither is appli~d determines the level of randomne~s of the outcome. In Figure 6B, conventional rounding i~ used for fractions less than 3/8 or greater than 5/8, with randomizedrounding between those valu~s. As shown, this decreases th~ probability of the greatest errors. In - : Figure 6C, conventional rounding is used for fractions less than l/4 or greater than 3/~t and in Figure 6d, it is used for fractions less than l/8 or greater than 7/8.
~ The choice of which rounding mode to u~e is user selectable~ Variable randomized rounding is applied to several points of the signal processing paths, such a~
at the ends of the foreground and background video proc~ssors, key processor and keying module.
Figure 7 shows how two or more of the video compositing systems 20, 20' can ~e cascaded to provide WO92~18937 P~T/U~92/02849 on-line multiple layering of images with a ~ixed overall delay of, for example, one frame. The key and mask inputs 2~ and 28 are provided at a time corresponding to the bsginning of the first line (lH) of a video frame to the ~rame stores 38 and 40. The background and foreground inputs 22 and 24 are similarly pro~ided to first in, first out (FIF0) memories 160 and 162~ The outputs of the f rame stores 38 and 40 and the FIF0 memories 160 and 162 are provided after a delay T1 of one line (lH) to the keying process and compositor sub~ystems 42, 4~ for processing, which results in an additional delay T2 of four lines (4H). Program and key outputs of the keying process and compositor subsystems 42, 44 are supplied at time 5H to delay mem~ries 164 and 166, where they undergo an additional delay T3 of one line (lH). The total delay in the compositing system 20 is thus 6H.
The program output 48 from the composit.ing system 20 is supplied as a background input to a lH delay T5 FIF0 memory 160' in the second compositing system 20' for a second layer of processing~ Additional foreground, key and mask inputs 24', 26' and 28' are supplied to FIF0 memory 16~', frame store 38' and frame store 40~, each of which has a 7H delay T4. The output of the ~IF0 memories 160' and 162' and the frame stores 38' and 40' thus are pro~ided after a total delay of 7H to keying process and composi~or subsystems 42',44', which have a processing delay of ab~ut 4H, supplying their program and key outputs to delay memories 164' and 166' after a total dela~ of llH. The delay memories 1~4' and 16~' add a delay T6 of 514H, giving a total delay of 525H, equal to one television frame, for the program and key outputs ~8 ' and 52 ' .
If a third video compositing system 20" is used for single level compositing, receiving the background, WO 92/18937 P~r/US92/02B49 2~0~5~ -lO-foreground, key and mask vîdeo inputs 22-28, r~spectively supplied to FIF0 memories 160" and 162" and frame stores 38" and 40t1. In the FIF0 memories 160" and 162" and frame stores 38" and 40" these inputs are supplied to 5 the lceying process and compositor subsystems 42", 44"
after a delay T7 e~ual to NH, where N is arbitrary.
Processing by the keying process and compositor subsystems 42", 44" results in a delay OI about 4H, and the program and key outputs from the keying process and 10 compositor subsystems 42", a~4" are provided after a total delay of (N + 4)~. Delay m~mories 164" and 166" provide a delay T~ equal to (521-N)H, so that the program and key outputs 48" and 52" from the viàeo compositing system 20" are provided at the same time as the program and k~y 15 outputs 48' and 52~ from the video compositing system 20'.
In operation, the video compositing systems 20, 20' and 20" are synchronized by the REF SYNC signal supplied to gen-lock sync generators 168, 168' and 168". E:ach 20 of the video c:ompositing systems 20, 20' and 20l' takes about 6H of minimum signal propagation delay. The FIFo and framestorememories 160-162 and 38-40, 160'~162' and 38'-40' and 160"-162" and 38"-40" at the input to each video compositing system 20, 20' and ~0" and signal delay 25 memories 1647166, 164'-166' and 164"-166" at ~he output of each system 20, 20' and 20" allow a wide range of input and output timing adjustment. Tens of the video compositing ~ystems can be cascaded to multiple layering, while maintaining exactly one 1~ frame of overall signal 30 delay. Note that each internal video bus for the systems 20, 20' and 20" uses different signal processing in the keying process and compositor subsystems 42, 44, 42', 44' and 42", 44". What is indicated in the drawings is therefore conceptual in natureO

W092/~8937 PCT/U~92J02~9 ~ ~t~ ~
Figure 8 shows details of the key processing toolkit 42 and the comparator 44 of Figure 1. The key and mask inputs ~6 and 28 are supplied through the frame stores 38 and 40. The output of the ~rame store 38 is supplied through a key disable/enabl~ switch 80 through a key inversion/noninversion circuit 82. The output of the key inversion/noninversion circuit ~2 is supplied to a multiplier 96. The output of the frame store 40 is supplied to a mask inversion/noninversion circuit 86, the output of which is supplied through a mask enable swi~ch 88 to a multiplier 90. The other input 92 to ths multiplier 90 is a fade control signal. The output of the multiplier g0 is a processed mask signal, which is supplied through a mask multiplication enable/disable switch 94 to the multiplier 96. In most cases, the processed mask signal is supplied in this path, ~o that ît is used to modify the key signal prior to key signal processing. The output of the multiplier 96 is supplied to a key processing subsystem 97. The output o~ the key processing subsystem 97 is supplied to a multiplier 104.
The multlplier input to the multiplier 104 is the output of a second mask multiplication enable/disable switch 106. The secvnd switch 106 is used to supply the mask input in certain special cases, such as ~or the generation o~ drop shadows in an image. Otherwise; the multiplier 104 passes the output of the interpolator 102 unchanged. The output of the multiplier 104 is supplied : : to an inversion~noninYersion circuit 110, the output of which is a processed key signal, which is suppli~d at ~14 to a multiplier 122 for keying. The output of the multiplier 104 is also supplied to an inversion/noninversion circuit 1~6 for supplying a processed key output at 120 through a delay memory 124, which may or may not be the same as the processed key : . 35 output at 114.

WOg2/18937 PCT/~S92/02~9 21~8~8 -12-In the compositor 44, the background and foreground inputs 22 and 24 are supplied to FIF0 memories 126 and 128. The output of the FIF0 memory 126 is supplied to a background video processing subsystem 130. The output of the video processing subsystem 130 i~ supplied to a background video/fill select switch 132, which allows user selection of the background video or a fill input 1340 The output of the select switch 132 is supplied to a subtracter 136 and as a processed background video output. The output of the FIF0 memory 128 is supplied to a foreground video processing subsystem 138, the output of which is supplied to a foreground video/fill select switch 170, which allows user selection of the foreground video or a fill input 172. The output of the select switch 170 is supplied to a non-additive mix (nam) or linear mix/matte select switch 174 and also as a foreground video output. The output of the seleck switch 174 is supplied to the ubtracter 136. The output of the subtracter 136 is supplied to the multiplier 122.
The output of the multiplier 122 is supplied to an adder 176, the other input of which is the for~ground video output. The output of the adder 176 is supplied through a delay memory 178 as the keyed output program signal.
Further details of the key processing toolkit subsystem 42 are shown in Figure g~ The key mask signal from fr.ame store 40 is supplied through ~ gain and offset ad~u~ment circuit 84 to the mask inversion/noninversion circuit 86. The output ofthe key inversion/noninversion circuit 82 is the multiplicand input to the multiplier 96. The output o~ the multiplier g6 is supplied to a noise reduction filter 98 and then to a clipping circuit 100. The output of the clipping oircuit lO0 is supplied to a key horizontal/vertical edge softening filter~intPrpolator 102;- The same filter circuit is used with different coefficients for both filters. The -13~ ?~ 2 .~ ~
filter/interpolator 102 is implemen~ed with a two-dimensional finite impulse response filter. Its structure is modified for two dimensions. For edge softening and interpolation, the same filter structure, driven by different coefficients supplied by the microprocessor, is used. The filter/interpolator 102 may be implemented as a variable bandwidth, low pass filter and interpolator, using multiple TRW TMC 2246 integrated circuits. The output of the interpolator 102 is supplied as a multiplicand to a multiplier 104. The filter 98, clipping circuit 100 and filter/interpolator 102 comprise the key processing subsys~em g7. After gain and offset adjustment at 108 and inversion or non-inversion at llO, the key signal is supplied to a look up table sPlector 112 for supplying the processed key signal at 114 for keying. The key signal may be separately processed for inversion/non-inversion at 116 and supplied to another look up table s~lector 118 for supply as the separa~e processed key output at ~20~
Figure 10 rsprese~ts a simplification of the Figure 9 subsystem 42 to show the effects of the inversion/non-inversion 82, 86 and 110 or 116 and the multiplier~ 96 and/or 104. Figures llA-llL show variatisns of area selections ~chievable through key and/or mask inversion as a result ~f the effects produced with the elements shown in Figure 10. In Figures 10 and llA-llL, the representation x~ means inverted x.
Figures 12~-12G show the results of microprocessor 62 (Figure 1) control of the clipping filter 100 in Figure 9O In these figures, the designations shown have the following meanings:
TT: Input top threshold level TB: Input bottom threshold le~el MT: Output top mapped level MB: Output bottom mapped level W~g2/18937 P~T/US9J-/~2~9 - 21 ~g2~ -14-I : Any input 0 : Output corresponding to I
Figure 12A represents the Cartesian subspace in which input and outp~t characteristics are de~ined.
Figures 12B-12G represent the input and output characteristics produced in accordance with the following rules:
1. If TT <= I then 0 - MT
2. If I < TT and TB < I then 0 = I
3 . I f I < TT and I <= TB then 0 = MB
Such flexible clipping of the key signal allows very many different input and output charact~ristics to be defined.
Figure 12B is a typical example o~ clipping. Figures 12D or 12G are realizable to give a large amount of freedom to the user.
Figure 13 provide further details on the multiplier 122 (Figure 8) of the compositor 44 and 14A 14B show certain relationships between component video keying and signal frequency spectrum, useful for understanding operation of the multiplier 122. Background and ~oreground component video luminance signals (Y~ and chrominance signals (U, V) are supplied to subtracters 180, 182 and 1~4. The outputs of the subtracters 180, ~82 and 184 are respectively supplied to interpolators and frequency doublers 186, 188 and 190. The outputs of the interpolators and frequency doublers 186, 188 and : l90 are supplied to multipliers lg2, 1~4 and 196. The key signal is supplied directly to the multipliers 194 and 196 and, through interpolator and frequency doubler 198, to the multiplier 192. The out~uts of the multipliers 192 t 194 and 196 are supplied through decimators 200, 202 and 204 to summing circuits 206.
The three summing circuits 206 correspond to the single summing circuit 176 in Figure 8. The interpolators and frequency doublers 186, 188, 190 and 198 and the WO92/18937 2 ~ ~ 8 .~ PCTJUS92/02~9 decimators 200, 202 and 204 are implemented with TRW TMC
2242 integrated circuits.
In operation, the multiplier 122 proces~es the component video signals as follows:
Y component Y:4 -> Y:8 -~ K:8 * Y:3 -> LPF -~ K*Y:4 C (U and V) component C:2 -> C:4 -~ K:4 * C:4 -> LPF -> K*C:2 Figures 14A and 14B show the signals and their processing. The input K:4 and Y:4 signals show the baseband freguency 210 corresponding to the Nyquist limit of l/2 the sampling frequen~y 212, along with the second harmonic 214 of the sampling frequency 212. The two K
and Y input signals have their sampling frequency dou~led to give a gap between the baseband frequency 210 and the doubled sampling frequency 216. As a result, spectra overlap betwePn the keyed ba~e~and frequency 218 and the keyed sampling frequency 220 is avoided. The keyed signal~ are then low pass filtered to give a gap between the low pass filtered baseband frequency 222 and the low pass filtered~sampling frequency ~24O The low pass ~ilters are part of t~e decimators 200, 202 and 204.
When the frequeney of the keyed signals is halved, a clean output of the K*Y:4 baseband frequency 226, sampling frequ n~y 228 and second harmonic 230 of the sampling frequency is obtained. S milarly, ~ith the U
and V componen~s, the input C:2 signals 232 are fxeguency dou~led to give gaps between the doubled ba~e~and frequen~y 234, the doubled sampling frequency 236 and the doubled second harmonic 238 of the sampling frequency 236~ However~ because the input K:4 signals 240 are not frequency doubled, spectral overlap occurs between the keyed K*C:4 baseband frequency 242, sampling frequency 244 and second harmonic~246 of the sampling frequency~
This overlap is not a problem, because the regions of WO92/18937 PCT/US92/02~9 the overlap are eliminated ln the low pass filtering to give the low pass filt~red K*C:4 baseband frequency 248, sampling frequency 250 and second harmonic 252 of the sampling frequency 250. When the frequency of these signals is halved to give the K*C:2 baseband frequency 254, sampling frequency 256 and second harmonic 258 of the sampling frequency 256, a clean output is also obtained.
Figure 15 shows another form of signal ~low for a 10 version of the video compositing system 20 of Figure 1 in which a switch 260 is provided between the multiplier 122 and the summer 176, to bypass the background -foreground signal from the subtracter 136. The keying multiplier 122, as shown in Figure 13, uses interpolation and decimation ~ilters to avoid the introduction of aliasing signals. However, such ~ilters ine~itably alter signal quality to some extent, and their outputs cannot be identical of the inputs. This impurity of output signal from the keying multiplier 122 causes two problems when 100% background signalis the intended keyed output, i.e.
Output:= K * F~ + (l-K) * BG
= FG + ~l-X) * (BG - FG) when K = 0, Output = FG + 1 * (BG - FG) Because of the filtering involved, 1 * (BG - FG) as ~he output from ~he ~eying multiplier section is not equal to (~G - ~G). As a result, the output consists of a slightly impure BG and some residue of FG. By adding a bypass for the (BG FÇ~ signal for the case wheniR
= 0, the signal impurity problem is eliminated. Switch 260 is controlled by K such that select bypassed ~G - FG) if K = 0 (nearly equals ), ..
35 select keyed (BG - FG) otherwise.

.. , , . . . . , .. . . , . , .. . .~ . , .. , ... ... , . ,~ . , , , , . .,,, ... .. . ~ . ... .. . .. . ....
.. ... .

W092/1X937 PCT/USg2/02~9 -17- Z ~ ~
Other than as shown and described, the construction and operation of the ~igure 15 embodiment is the same as that of the Figure 8 embodiment.
Figure 16 shows a control panel 220 for the video compositing system 20. The control panel 220 is housed in a single construction and consists of a 8x42 character display 222, with both character and graphics capabilities, four shaft encoders 224, a trackball 226, a keyboard 228 and a buzzer for audio feedback to the user. The system 20 responds in real time to the shaft encoders 224 or key depressions on the keyboard 228.
The keyboard 228 consists of three functionally separate sets of keys, namely, the softkeys 230, the numeric ~eys 232 and the function keys 234. The row of five softkeys 230 are located immediately below the display panel 222. Their functionality is context dependent, and any softkey 230 may be enabled or disabled for a given display. Consider Table 1.

WO92/18937 PCT/US92/~284g ~, 1 0 ,'3 ~ 3 8 ~ . --~ DIGITAL OUTPUT-2 Selection , 1 Output: Composite I CMPST FG BG KEY ~ASK
----------------_____________;--_________________ __~___ _______ _______ __~___ ,._____ 1 1 2 ~ 1 3 1 1 ~ I 1 5 ______ _______ _______ ______ ______ Table 1 Table 1 represents the state of the control panel during user selection of the ~igital Output signal. Here we ~ee that all of the five softkeys 230 have been enabled, and are prompting the user to choose between the ignal sources, namely~
CMPST -- Composite Signal F~ -- Foreground Video signal BG -- Background Video signal KEY -- Key signal MASK -- Mask signal.
The userfs las~ choice is displayed in the status line headed "Outp~t: ". This format îs typical of the interplay between display 222 and softkeys 230. The last line of the display panel is reserved entirely for this purpose. Not e~ery softkey 230 will be enabled as Table 3V 2 serves to demonstrate. In fact the general philosophy of the usex intérface is to keep every display as simpl~
as possible, 5~ as not to overpower the user.

wo g2"~93, ~ ~ ~ 5 2 5 ~ PCT~U592/02~9 ______,________________________.__._______ FOREGROUND - STATUS
Gain = 25 %
I Offset = 63 IRE I
1 ~.
' Port 2 - serialHue = 58 degs j Sa~ = 13 %
I INPUT AD~UST
1 0 --~
Table 2 The appropriate menu for any video compositing system 20 operations is activated by depressi~g a function key 234. These function keys 234 provide instant access to their respective displays and automatically enable the correct softkeys 230. The function keys ~34 are located in a 10 x 4 matrix just beneath the row of soft keys, and are clearly engraved.
Often, numeric input is required, and this is provided by a 4 x 4 ma~rix of numeric keys 232 located to the right of the function keys 234. The numeric k ypad ~onsists of ~e familiar digits 0 - 9, a sign key, a decim l point, and a clear key.
Numbers entered via the numeric key~ 232 are displayed, as soon as they are enterad~ in the top right corner of the display 222 (a~suming that the keypad is enabled) and are enclosed in square bra~kets. Table 3 shows a display after 0.75 has just been enteredO The softkeys 230 are then used to carry this number across to the appropriate entry, for example, [SUB-V] in Table 3.

.. . , . . , .. . ~ . .. . . . ..... . . .

W~9~/18937 PCT/US92/~284g -2~-~8~8 KEY - POSITIONING r0.75]
, Sub-Vertical = 0.5 Sub-Horizon~al = 0.75' 1 ~ .
~ Vertical = 12 lns Horizontal = 0 pix ¦ [SUB-V] ~VERT] ~SUB-H] [HO~IZ]
1 0 ~
Table 3 Often, the user needs to choose the correct numbers for his or her application visually. In other words, the numbers are chosen ~y observing the changes generated in the output signals of the video compositing system 20~ To aid the user in this, four shaft encoders 224 have been provided, and are located in a 2 x 2 array to the right hand side of the display 222. They are arranged in this manner to coincide with the four varia~les as shown in Table 3. Namely, the top left enco~er 224 corresponds to l'Sub-Verticale' entry, the bottom right to "Horizontal" and so on. Values are increased by rotating the shaft encoders 224 in a clvckwise direction, and decreased by a c~unter-Glockwise rotation. Their sensitivity is context dependent. As in the case of so~tkeys 230, not all encoders are enabled fo~ each and every menu (see Table 4~. .
., _~_____________ _________~_______________ ~ SK ~ POSITION []
i.

' Vertical = 26 lns Horizontal = 14 pix ¦ [VERT] [HORIZ~

Table 4 W092/18937 ~ ~ 8 2 ~ ~ PCT/US92/028~g The trackball 226 is located to the far right of the numeric key pad and may be used instead of two shaft encoders to control positional input in two planes simultaneously. ~s before it is only enabled in certain menus. Finally, a buzzer is prsvided, to warn the user o~ any attempt to enter incorrect numeric values.
In use of the ~ideo compositing system 2~, on power up the control panel display 2~2 ~ilOWS a simple banner message as shown in Table 5 and awaits user input via the keyboard 228, the four sha~t encoders 2~4 or the trackball 226.

________ __________ ______________________ , 15 , 7 ' - Accom ICM-4224 -_____________ ____________________________ Table 5 ~oreground and Background Input ~ele~tion Selection of foreground and background video is identical. For example, foreground video selection is accomplished by ~epressing the Foreground Select function key 234. In response to this request the display panel 222 shows the currently active selection, and any auxiliary information pertaini~g to that selection.
As an example of the foreground signal coming from the first Dl parallel input port se~ Table 6. Here the luminance gain and offset values are included for the user's convenience, together with the signal's hue and saturation levels. On the bottom display line two softkey prompts, INPUT and ADJUST, ~re displayed above W~9~/l8937 PCT~US92/02~9 21 ~82~ 22-the first and last so~tkey buttons 230, which are thereby enabled.

___ __________ _____ ____,________________ I FOREGROUND ~ STATUS
~ Gain = 25 %
Offset = 63 IRE
Port 1 - parallelHue ~ 92 degs ~ Sat = 13 %

~ INPUT ADJUST
__________________________________._______ Table 6 Input source selection may be altered by depressing the softkey just below the word INPUT ~see Table 7).
All fiv~ softkeys 230 now become available for the five selections shown at the bottom of Table 7.

__________________________ __ _________ .__ , FOREGROUND - INPUT
j ~nput: Port 1- parallel , ~ PORT CONFIG COLOR BLACK TEST
______________ Table 7 ,Should one wish to change the curr~nt input source for any one of the four other Dl input ports, one sr more ~ depressions of the PORT softkey 230~ar~ required. The : central status line will rotate through them on repeat2d depressions (see Table 8~.

21û~2~

_ _ _ _ ~, _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ ~ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ .
~ FOREGROUND - INPUT

51 Input: Port 2- parallel I PORT CONFIG COLOR BLACK TEST
1 0 ~
Table 8 Additionally, either the serial ~r parallel Yersion of this port may be chosen. The CONFIG button 230 toggles between these two possibilities (see Table 9).

--------O-~----------_---_____________ FOREGROUN~ - INPUT

' Input: Port 2- serial ' -I PORT CONFIG COLOR BL~CK TEST
___.___ ______________ _____O_______.___~_ T~ble 9 Returning the foreground select menu using the Last Menu or Foreground Select button 234 (see Table 6), and selecting the ADJUST softkey, one can modify the luminance and~color values t~ee Table l0).

FOREGROUND - ADJUST~]
Lum Gain = 25 % Lum Offset = 63 IRE
. .
, Hue = 92 degsSaturation = 13 %
I
' ~5AIN] [HUE3 ~UNITY~ [OFFSET~ [~AT]
__.______._._____________________ ________ - 40 Table l0 WOg2/1~937 PCT/US92/02849 ?,~3~5~ -24-Typically, numeric values may be changed in either of two ways. By rotating the sha~t encoder 224 which bears the same positional relationship to the corresponding display entry, values may be altered incre~entally. Alternatively, numbers may be entered with the numeric keys 232. Table 11 shows the display 222 after the number 58 has been entered. Note that it is displayed in the top right corner and is enclosed in square brackets. The square brackets serve to draw a correspondence between the number entered and the (in this case) four possible destinations. These destinations are selected by depressing the appropria~e softkeys, which are also enclosed in square brackets.

_____________ ~_______________.___________ I FOREGROUND - ADJUST [ 58 ]
~ Lum Ga in = 2 5 % Lum Offset = 63 IRE
¦ Hue - 92 degs Saturation = 13 %

, [GAIN] [HUE~ [UNITY] [OFFSET] [SATJ
________________.._________________________ Table 11 Table 12 shows the result of choosing the so~tkey [HUE]. The number (58) is transferred to the Hue - XX
status entry and the numeric store is automatically cleared.
.
_______________ _______________..__________ 3 O ~ FOREGROUND - ADJUST ~ ]
Lum Gain = 25 % Lum Offset = 63 IRE
H~e = 8 degs Saturation = 13 %

[ GAIN ] [ HUE ] [ UNI TY ] [ OFFSET ] ~ SAT ] ¦ :
____________ :
Table 12 WO92/1~937 PCT/US92/02~9 -25- ~
Typically the two methods are used together, the keypad to enter coarsP values, and the shaft encoders 224 to do fine tuning. On returning to the preceding menu, changes are reflected in the status informakion (Table 13).

__________________________________ _______ FOREGROUND - STATUS
~ Gain = 25 %
¦ Offset = 63 IRE ¦
Port 2 - serialHue ~ 58 degs ' Sat - 13 %
¦ INPUT ADJUST
~5 -----__________________________~_ Table 13 The general philosophy maintained throughout the user interface is to include only those values and prompts which are pertinent to current selection system setup. For example should color fill be selected by a depression o~ the COLOR so~tkey 230 ~see Table 14), then the display takes on the form of Table 15.
.

_ ___._ __ ___ _ ____ _ ___ __ _.__ _______ _ ________ ¦ FOREGROUND - INPUT
2 5 1 :
Input: Port 2- serial 'i i , PORT CONFIG COLOR BL~CK TEST

; Table 14 W~:) 92/18g37 P~r/us92/U2~49 2 ~ ~3 ~ 2 6-____._ _________________________________.__ I FOREGROUND - INPUT

~ Input: Color PORT CONFI~; COLOR BLACK TEST

Table 15 Backtracking one level of menu results in a somewhat different display (see Table 16) which should be contrasted with Table 6. Here only those adjustments relevant to Color Fill selection are shown, namely, the red, green and blue co,mponents of the fill color, which may be altered în an analogous manner, by selection of the ~DJUST softkey 230, and numeric value modification as described above (see Table 17).

~OREGROUND - STATUS

8 %
~ Color G = 100 %
~ B = 32 %
-, INPUT ADJ~ST

Table 16 :

W092/l8937 - PCT/US92/02~9 ~2 ~ Q 3 ~ r g __________________________~_________.___~__ t FOREGROUND - COLOR FILL [~ I

I Units: RGB R - 98 %
Ç = 100 % B - 32 UNITS [G] ~R} [B]
___________________ __~___________ Table 17 As a further alternative, a change of units is possible through the UNITS softkey 230. The values and units are changed accordingly (see Table 18) and remain the preferred units in status displays until a further units change is requested (see Table 19).

_____________________.____________ ._____ __ 7 FOREGROUND - COLOR FILL []
~0 ~ ~its: Y W Y = 78 %
U ~ - 12 ~ V = 14 %
ll ll 1 UNITS ~U] t~] [V3 _______________ _________________________._ Table 18 I FOREGROUND - STATUS

= 78 ~ ' Color U = ~ ~2 %
, V = 14 %
~ INPUT ADJUST
________________________________ ______ Table 19 ~ lack Fill is a further input alternative (see Table 20~, which requires no ad~ustment ~see Table 21).

WO92/18g37 PCT/US92/02~9 2 ~ ~
. --¦ FOREGROUND - INPUT

5 1 Input: Black I PORT CONFIG COLOR BLACX TEST
------------------------____________________________ Table 20 _________________._______________________, ~ FOREGROUND - STATUS

~ Black , INPUT
___________. _____________________ ________ Tabl~ 21 Finally, a ~est Pattern source i~ provided~ for diagnostic purposes (see Tables 22-23).

---~
I FOREGROUND - INPUT
,, ~ Input: Test ~ t ~ PORT CONFIG COLOR BhACK TEST
_ __________~______________________~.__ - 35 Table 22 WO g2/18937 PCr/US92~02~49 3 3 ~ 3 ~3;
____.._________ ________.__________________ FOREGROUND - STATUS
Gain = 25 ~6 ¦
Of ~set = 63 IRE
Test Hue 3 58 degs Sat = 13 %
INPUT ADJUST
10----------------________________________.___ Table 23 Mask ~nput ~el~ction The source of the mask input is selected along very similar lines to foreground and background input. From any menu the Mask Select,button is depressed resulting in a display as shown in Table 24.

___ ______________________________________ I ~SK - STATUS

~ Port 4 - parallel U/V Normal 25~ INPUT DISABLE CMPT POSITION INVERT

Table 24 ::
Note that the status display includes additional information as to whether the incoming signal is to be taken from the luminance or chromi~ance component, and whether this component is to be inverted or not. The source of the mask may be selected fro~ any one of the four D1 inputs~36 (~igure 1), an analog input, a shape generator .or the Test Pattern generator. This is achieved by depressing the INPUT softkey 230 ~see Table 25~.

~, W092/1~937 PCT/US92/0284g 21~82~ 8 -30-_____~_____________________ ____~_______~_ I MASK - INPUT
ll l , Input: Port 4- parallel ~ PORT CONFIG ANALOG SHAPE TEST
1 0 --~
Table 25 Selection proceeds as outlined for for~ground and background input. Returning to the Mask Selection menu, choice may be made between the Luminance (Y) or Chrominance ~U/V) component by toggling the CMPT softkey (see Table 26, Table 27).

__________________________________~____,___ , MASK - STATUS

' Port 4 - parallel Y Normal ~ INPUT DISABLE CMPT POSITION INVERT
_________.._________________________ ______ Table 26 ________________________________________._ , MASK - STATUS

' Port 4 - parallel U/V Inverted ' INPUT DISABLE CMPT POSITION I~ERT
__________ .._______ _______~ ______~___~__ Table 27 The incoming mask signal can be disabled and replaced by a constant source by toggling the D~SABLE

W092/18937 PCT/USg2/02849 -31- ~ ~ 8 f~
/ ENABLE softkey 230. A display corresponding to Table 28 is obtained for the disable state.

_______ ___~____. ______________________ __ ~ MASK - STATUS

' ~f 1 ~ ~
I ENABLE
___________~_______ __ _______ __ ______ Table 28 Finally, the mask itself may be repositioned by ~irst selecting the POSITION so~tkey 230, and then adjustiny the horizontal and vertical controls via the shaft encoders 22~ or numeric keys 232 in the now familiar way (see Table 29).

____ _____________.______ _______________ ~ MASK - POSITION ~]

~ Vertical = 26 lns Horizontal = l4 pix : 25 I [VERT] ~HORIZ~
______. ____.______~____.___________________ . Ta~le ~9 ~y I~put Sel~ctio~
The source o~ the key i.nput is selected in the same way as the mask, except for a few refinementsO Depress the Key Select button to display a menu v ry similar to that associated with the Mask Select button (see Table 30)-.

W092/18937 PCT/US~2/02 ~ ~ as258 ._ ______________,________ .________________ I KEY - STATUS

Port 3 - parallel U/V Normal ,l INPUT DISABLE CMPT POSITION INVERT
------------~--__________.____ _____ _____ Table 30 All softkeys 230 function as for the mask selection case, with the exception of the POSITION softkey 230.
As the key signal has th~ additional feature of permitting subpixel positioning accuracy, a fine tuning control is also supplied. Adjustm2nt is achieved by using the shaft encoders as coarse and fine tuning controls (see Table 31).

_______ ______________________________ 2 0 ~ KEY - POSITIONIN
j Sub-Vertical = 0.5 Sub-Horizontal = 0~7 ' Vertical = 12 lns ~orizonkal = O pix ~SUB-V~ [VERT] [SUB-H] ~HORIZ]

Table 31 Clipo Gain a~d Thr~hold ~dju~tme~t Since adjustmen~ of the clip, gain and threshold characteris~ics for the key signal are functionally related, the user interface groups these siX control elements into two user menus which can be interchanged by a single softkey 230 depression. Tables ~2-33 show these two displays. The central softkey 230 THRSHLD or ~AIN takes the user frQm the current display to its ~omplementary one. Note that the Upper and Lower Clip W092/1~937 PCT/US9~/02~9 f~ ?, ~, ~

control;:~ are provided on both of the menus for convenience~

_____=_____ ______________._______________ CLIP ~ GAIN ~]
Upper Clip = 100 %Gain = 106 %
' L~wer Clip = 14 %Offset = 15 IRE
1~ 1 ~U-C~P] [L-CLP] THRSHD [GAIN] [OFFSET]¦
______________ ___,.______________________ Table 32 _____________.____________________.________ ~ CLIP & THRESHOLD []
' Upper Clip = 100 % ~pper Thrshd - 97 % ' ¦ Lower Clip = 14 % Lower Thrshd = 13 % ¦

j[U-CLP] [L-CLP] GAIN [U-THR] ~L-THR]
_._ ______________________________________ Tabl~ 33 Compo~ition CG~trol A single menu has been dedicated to composition control. Numerous effects can be realized by the videv composition system 20, through the control of the ma~k and key signals. Manipulation of the foreground and background video, color fills or black to generate matte, : mix, fades etc. is easily achieved.
The Composition Control but~on leads one to the control menu as shown in Table 34. Each softkey 230 on this menu toggles between the alternative entries, except for the [FADE].
, WO92/18937P~T/US92/02~9 2 ~ ~
SYSTEM - CONTROL ~] 0 ~ Normal Normal On Of~ 78%

O KEY MASK BEFORE AFTER [FADE]
------------------______________________ _____ Table 34 As an example, consider the kP.y itself. Repeated depressions of the ~EY softkey 230 cycles one through the three alternatives, namely, Normal, Inverted and Off (see Tables 34-36). The MASK softkey 230 performs an identical function for the mask signal.

___________ ______... _._____________________ I SYSTEM - CONTROL []

,Inverted Normal On Off 78%
, 25, XEY MASK BEFORE AFTER [FADE] ¦

Table 35 ' ________________________________.______~__ . 1 SYSTEM - CONTROL
`~ .
' ~ Off Normal On O~f 78%

' KEY MASK BEFORE AFT R [!FADE3 _________________________ __~___________ : Table 36 One of the design elements which enables the video compositing system ~0 to achieve such a wide range of .. .. . . . . . .. . . . . . . .. ... . .

WO92/18937 PCT/US92/~2849 _35_ ~d~2~
compositional ef~ects is it~s ability to channel the action of the net key signal into both pre and post "key-process" phases. The enabling / disabling of these two ~ontrols are provided through the BEFORE and AFTER
softkeys 230, which alternate between the On and Off status.
The percentage of mix between the foregr~und and background is controlled through the ~F~DE] softkey 230 via numeric key 232 entry, or the relevant ~haft encoder 224. In addition to this "analog" mode of entry, the Cut button 234 alternates the Fade le~el between 0% and 100% ~see Tables 37-38).

_______________________________~__________ SYSTEM - CONTROL [ ]

I
' Normal Normal On OfP100%

, KEY MASK BEFORE AFTER [FADE] ¦
___.____.______ __________________ _______ Table 37 _____________________.____________________ I SYSTEM - CONTROL ~ ]
.
' Normal ~ormal On Off 0%

! KEY MASK BEFORE AFTER [FADE] !
___ __ __ ____ ~_ _ _ _ ___ ______ _ _____ _ __ _ __ _ ___ Q

Table 38 Cori~g and Softening The key signal may be further modified by coring and softening. These controls are accessed by the Coring :

W~92/18937 PCT/US92/02~9 2 1 ~ ~ 2 rj ~ -36-and Softening button 234. In essence the coring level selects a low pass filter profile. In addition the horizontal and vertical edges of the key shape may be ~oftened through the two controls provided (see Table 39) ___._______________________________________ CORING & SOFTENING rJ
~ Co~ing Level = O
! vertical= ~ Horizontal= o ! ~VERT] ~CORE~ [HORIZ] !
----------_- ______________________ Table 39 .Tra~sitio~ Control Transitions are controlled by specifying a transition profile and a duration. These are accessed by depressing the ~uto Transition button 234 to give the display shown in Table ~0. As before, both numeric keys : 232 and shaft encoders 224 may be employed ~or data input (see Table 40).

____ __ _ ________________________________ 25 I AUTO - TRAMSITION C]

, ~ ~ Profile - 4 Duration = 120 frames ~
30 ! ~: :
! tPRGF] ~DUR]
___________________ _____ ______________ ._ Table 40 3 5 Gutput Timirlg The fully composed "composite" video, and the --processed key signal both have an output timing control ~0~2/1~937 PCT/~S92/02~49 -37~ r $
which may be reached by a depression of the Output Timing button 234. Selection is a two stage process. The first menu to be displayed is shown in Table 41. From here one chooses either ~he KEY or CMPST svftkey 230, to give displays indicated in Tables ~2 and 43 respectively.

__ _______ _______________________________ ~ OUTPUT TIMING

, , KEY CMPST
---_______________________________ Table 41 _____________ _________________;__~____ __ , OUTPUT TIMING - KEY []

~ Vertical = 2 lns Horizontal = 3 pix , ' KEY CMPST ~VERT] r HORIZ~ ' ___.___________..______________ _____,_____ Table 42 _.____________________________.___ _______ ~ OUTPUT TIMING - COMPOSITE

~ Vertical = 6 lns Horizontal = 5 pix ' KEY CMPST [VERT] ~HORIZ]
____________________________ _____________ Table 43 WO92/1~937 PCr/US92/02 2 ~08~ ~ 8 -38-Digital Outputo2 Selection The video compositing system 20 provides five digital output choices ~rom "composite", foreground, background, key and mask signals. Selection is made by first depressing the Digital-2 Output selection button 234 and ~hen the softkey 230 corresponding to signal required (see Table 44) __________________________________________ , DIGITAL OUTPUT-2 ~election ' Output: Composite I CMPST FG BG KEY MASK
___________ __________________.~______ ____ Table 44 ~alog Output 8~1ection An analog output is provided to monikor the outgoing sign~ls. Choices are made via the Analog Output button 234 in the ~ame way as for Digital-2 Output selection.

_________ ____ ________~;O__________O______ I ANALOG OUTPUT Selection I Outpu't: Composite CMPST FG BG KEY MASK

Table 45 Attached as an appendix hereto and forming a part of this application is a source code listing in the C
programming language of control programs for the microprocessor contr~ller 62 ~Figure l). These programs W092/18~37 P~T/US9~/0284g ~39~ ~J I ~ ~ 2 ~ ~
control all o~ the hardware elements shown in the detailed block diagrams of Figures 8, 9, 13 and 15, communications with the control panel of Figure 16 and communications with other systems, such as video editors, connected to the system 20. The software al50 provides the menus shown above in Tables 1-45.
In practice, the digital image compositing system of this invention is a component digital single-layer high accuracy compositing module that can stand alone or be an integral part of a large system. New techniques for combining layers provide a level of compositing performance and quality that has never been achieved before. The system can be used for all Dl compositing applications as well as be the center of a very cost effective Dl editing suite. The system provides advanced capabilities in layering performance. New digital processing techniques preserve foreground and background edge detail, thereby achieving unsurpassed layering realismO
The system is easily integrated into digital and analog compone~t systems where digital component editing is required. It can be the center of new Dl editing and compositing suites b~ replacing more costly Dl switchers.
For graphics applications, it provides real-time layering at a lower price than alternative means. In t~lecine, the system replaces a switcher for preview or production compositing while pulling mattes.
Both linear and matte keying is provided. Matte keying simulates the optical matte process which leaves the edges of the foreground video unprocessed, thereby pre~erving the foreground's edge detail.
The system can be controlled through RS 422 and GPI
ports. Serial RS-422 ports allow an external device to emulate the system's control panel and allow the system to be controlled as if it were a ~rass Valley Group lO0 21082~ ~40-series switcher. The functions of the GPIs can be assigned by the sperator.
The system can be controlled using an internal time line. Events which are assigned to the system's time line are automatically initiated and can be stepped through on a field-by-field basis. The time line can be programmed from the control panel or by an external controller.
Even though the system is a single-layer module, it can be combined with additional sys~ems to do parallel multi-layering. The delay through a single system is exactly one frame. When cascaded, the delay through each system is adjusted to maintain one frame delay through the completed cascaded array.
15A complete kit of tools is provided to modify the key signal. H and V position is adjustable in sub-pixel increments to match key to foreground. To improve the transitions between foreground and background, ad~ustable H and V edge softening is provided. Upper and lower clip points can be set over the complete range of the key signal to adjust the keying effect. Used with the clip points, wide-range gain and offset adjustm~nts also provide a power~ul way to adjust the look of the keying process. ~ To smsoth edges from analog keys, noise reduction is availa~le. Inversion of the key and mask is alsoA.provided.
Full adjustment of foreground and background parameters is provided, including luminance gain and offset and chrominance saturation and hlle rotation.
30Timed events are provided for transitions ~etween ~sources. These include cuts and dissolves between foreground, ~ackgroun~ and fill. Also, keys can be faded up or down. The foreground and background fill can be adjusted and programmed in RGB and Y W values.

WO92/18937 21 ~ 8 CT/US92/U2849 The system operates in both 525 and 625 4:2:2 standards. Selection is automatic or c~n be manually set by the operator.
Keyboard macros can be programmed and assigned to the time line or GPIs.
An analog luminance input is provided to accept analog key and mask sources. This input is converted to 10 bit 4:0:0 and îs connectable to th~ internal key and mask busses through key and mask ~rame stores. A
full quality 10 bit RGB, Y W or Beta¢am output is provided for monitoring and recording on analog VTRs.
It should now be readily apparent to those skilled in the art that a novel digital image compositing system and method capable of achieving the stated objects of the invention has been provided. The digital image compositing system and method incorporates flexible input and output crosspoint switching. Digital inputs and outputs are provided for all video signals. The digital image comp~siting system and method will accept analog key or mask input signals. The digital image compositing system and method will additionally provide any output video signal in analog form. The digital image compositing system and method can be cascaded to provide real time multiple layer compositing. ~he digital image compositing system and method incorporates a user interface that facilitates use of ~he flexibility p~ovided by the~system and ~ethod.
It should fur~her be apparent to those skilled in the art that various changes in form and details o~ the invenkion as shown and described may be made. It is intended that such changes be included within the spirit and scope-of the claims appended hereto.
, .

,. .. .
., _ . . ......... . . . . . . . ... . ..
.. . .... .. .. . . . . .. _ . ...... . .. ... . . . .

Claims (14)

WHAT IS CLAIMED IS:

1. A digital image compositing system, which comprises a plurality of digital video signal inputs, a key processing subsystem, a video image compositor, a first plurality of crosspoint switches for connecting said plurality of digital video signal inputs to said key processing subsystem and to said video image compos-itor, a key input connected through said plurality of crosspoint switches to said key processing subsystem, a plurality of digital video signal outputs, and a second plurality of crosspoint switches for connecting said key processing subsystem and said video image compositor to said plurality of digital video signal outputs, wherein said key processing subsystem has at least one charac-teristic selected from the group consisting of:
(i) said key processing subsystem includes a look up table containing a plurality of user selectable look up table input/output characteristics, said key process-ing subsystem being configured to allow selection of different look up cable input/output characteristics for a television frame to provide different outputs for different portions of the television frame;
(ii) said key processing system is configured to provide user selectable randomized rounding of fraction-al values of the digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit;
(iii) said key processing subsystem includes a microprocessor controllable clipping circuit that can select from a plurality of different clipping character-istics;
(iv) said key processing subsystem includes a microprocessor controllable clipping circuit, responsive to a user selection, for selecting from a plurality of different clipping characteristics;

(vi) said key processing subsystem includes a mi-croprocessor-controlled edge softening filter/interpolating circuit for edge blending of a key signal supplied to said key input;
(vii) said key processing subsystem includes a microprocessor-controlled two-dimensional finite impulse response filter for edge blending of a key signal sup-plied to said key input;
(vii) said key processing system and said video image compositor are configured to provide user selectable randomized rounding of fractional values of digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit;
(viii) said key processing subsystem includes a look up table containing a plurality of look up table input/output characteristics, and means controlling said look up table to allow selection of different look up table input/output characteristics for a television frame to provide different outputs for different por-tions of the television frame; and (ix) said key processing subsystem includes a look up table containing a plurality of look up table input/output characteristics, and user-selected means controlling said look up table to allow user selection of different look up table input/output characteristics.

2. The digital video image compositing system of Claim 1, additionally comprising a control microproces-sor connected to said key processing subsystem, said video image compositor, said first plurality of crosspoint switches, and said second plurality of crosspoint switches.

3. The digital video image compositing system of Claim 2, additionally comprising a keyboard and display connected to said key processing subsystem and to said video image compositor through said microprocessor to provide real time response to inputs through said key-board, wherein said keyboard has at least one character-istic selected from the group consisting of:
(i) said keyboard includes a plurality of function keys, a plurality of numeric keys and a plurality of shaft encoders;
(ii) said keyboard includes a plurality of soft keys having software definable functions, said plurality of soft keys being positioned on said keyboard below said display, so that functions of said soft keys de-fined by software are displayed adjacent each of said plurality of soft keys; and (iii) said keyboard additionally includes a trackball.

4. The digital video image compositing system of Claim 1, additionally comprising a mask input connected by said first plurality of crosspoint switches to said key processing subsystem, a fade control input in said key processing system connected to modify a mask input signal from said mask input to produce a modified mask input signal, the modified mask input signal being con-nected to modify a key input signal from said key input.

5. The digital video image compositing system of Claim 4, wherein the modified mask input signal is con-nected for selectively modifying the key input signal prior to and after processing of the key input signal by said key processing subsystem.

6. The digital video image compositing system of Claim 5, wherein said digital video image compositing system includes a selectively operable inversion circuit having at least one characteristic selected from the group consisting of:

(i) said circuit is connected between said key input and said key processing subsystem;
(ii) said circuit is connected between said key processing subsystem and said video image compositor;
(iii) said circuit is connected between said key processing subsystem and a key output provided in said digital video image compositing system; and (iv) said circuit is connected between said mask input and said key processing subsystem, said circuit capable of selective operation upon user control.

7. The digital video image compositing system of Claim 1, including frame store means having at least one characteristic selected from the group consisting of:
(i) said means includes at least one frame store connected between said first plurality of crosspoint switches and said key processing subsystem;
(ii) said means includes at least one frame store connected between said key input and said first plurali-ty of crosspoint switches; and (iii) said means includes a frame store comprising a mask input connected by said first plurality of crosspoint switches to said key processing subsystem.

8. A digital image compositing system, which com-prises a plurality of digital video signal inputs, a key processing subsystem, a video image compositor, a first plurality of crosspoint switches for connecting said plurality of digital video signal inputs to said key processing subsystem and to said video image compositor, a key input connected through said plurality of crosspoint switches to said key processing subsystem, a plurality of digital video signal outputs, and a second plurality of crosspoint switches for connecting said key processing subsystem and said video image compositor to said plurality of digital video signal outputs, wherein said video image compositor has at least one character-istic selected from the group consisting of:
(i) said video image compositor is configured to provide user selectable randomized rounding of fraction al values of digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit; and (ii) said video image compositor and said key processing subsystem are configured to provide user selectable randomized rounding of fractional values of digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit.

9. The digital video image compositing system of Claim 8, additionally comprising a mask input connected by said first plurality of crosspoint switches to said key processing subsystem, a fade control input in said key processing system connected to modify a mask input signal from said mask input to produce a modified mask input signal, the modified mask input signal being con-nected to modify a key input signal from said key input.

10. The digital video image compositing system of Claim 9, wherein the modified mask input signal is con-nected for selectively modifying the key input signal prior to and after processing of the key input signal by said key processing subsystem.

11. The digital video image compositing system of Claim 10, wherein said digital video image compositing system includes a selectively operable inversion circuit having at least one characteristic selected from the group consisting of:

(i) said circuit is connected between said key input and said key processing subsystem;
(ii) said circuit is connected between said key processing subsystem and said video image compositor;
(iii) said circuit is connected between said key processing subsystem and a key output provided by said digital video imaging compositing system; and (iv) said circuit is connected between said mask input and said key processing subsystem.

12. The digital video image compositing system of claim 11, wherein said key processing subsystem includes at least one characteristic selected from the group consisting of:
(i) said key processing subsystem includes a mi-croprocessor controllable clipping circuit that can select from a plurality of different clipping character-istics in response to a user selection;
(ii) said key processing subsystem includes a microprocessor-controlled edge softening filter/interpolating circuit for edge blending of a key signal supplied to said key input;
(iii) said key processing subsystem includes a microprocessor-controlled two-dimensional finite impulse response filter for edge blending of a key signal sup-plied to said key input; and (iv) said key processing subsystem includes a microprocessor controlled edge softening filter/interpolating circuit for edge blending of a key signal supplied to said key input, wherein edge blending occurs under user control.

13. The digital video image compositing system of Claim 12, including frame store means having at least one characteristic selected from the group consisting of:

(i) said means includes at least one frame store connected between said first plurality of crosspoint switches and said key processing subsystem;
(ii) said means includes at least one frame store connected between said key input and said first plurali-ty of crosspoint switches; and (iii) said means includes a frame store comprising a mask input connected by said first plurality of crosspoint switches to said key processing subsystem.

14. A digital image compositing method comprising the steps of selectively providing a plurality of digi-tal video input signals including a key signal to a key processing means and a video image compositor, process-ing the selected plurality of digital video input sig-nals in the key processing means processing the selected plurality of the digital video input signals in the video image compositor, and selectively providing a plurality of processed digital video output signals from the key processing means and the video image compositor, including at least one set of steps selected from the group consisting of:
(i) sub-pixel positioning by interpolation for edge blending the key signal;
(ii) selecting different look-up table input/output characteristics for a television frame to provide dif-ferent outputs for different portions of the television frame;
(iii) also processing said selected plurality of digital video input signals in the video image composi-tor including selecting randomized rounding of fraction-al values of digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit;
(iv) providing a three key signal, a mask signal and a fade control signal to said key processing means, processing the key signal in the key processing means, modifying the mask signal using the fade control signal to provide a modified mask signal, and selectively using the modified mask signal to modify the key signal prior to and after processing of the key input signal; and (v) selectively inverting the key signal prior to supplying the key signal to the key processing means to give a selectively inverted key signal, forming a modi-fied key signal by processing said selectively inverted key signal in said key processing means.