WO1992018937A1 - Digital image compositing system and method - Google Patents

Digital image compositing system and method Download PDF

Info

Publication number
WO1992018937A1
WO1992018937A1 PCT/US1992/002849 US9202849W WO9218937A1 WO 1992018937 A1 WO1992018937 A1 WO 1992018937A1 US 9202849 W US9202849 W US 9202849W WO 9218937 A1 WO9218937 A1 WO 9218937A1
Authority
WO
WIPO (PCT)
Prior art keywords
key
digital video
signal
video image
input
Prior art date
Application number
PCT/US1992/002849
Other languages
French (fr)
Inventor
Sohei Takemoto
Kenneth A. Woodhouse
Original Assignee
Accom, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Accom, Inc. filed Critical Accom, Inc.
Publication of WO1992018937A1 publication Critical patent/WO1992018937A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/272Means for inserting a foreground image in a background image, i.e. inlay, outlay
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Definitions

  • the present invention relates generally to digital compositing, i.e., the layering or putting together of " images in a digital video domain. More particularly, it relates to a system and method for digital image compositing which gives the user an increased flexibility for such compositing. The invention further relates to a user interface that enables the user to take advantage of the increased flexibility more conveniently.
  • a digital image compositing system in accordance with this invention has a plurality of digital video signal inputs, a key processing subsystem and a video image compositor.
  • a first plurality of crosspoint switches connect the plurality of digital video signal inputs to the key processing subsystem and to the video image compositor.
  • a key input is connected through the plurality of crosspoint switches to the key processing subsystem.
  • a second plurality of crosspoint switches connect the key processing subsystem and the video image compositor to a plurality of digital video signal outputs.
  • Figure 1 is a block diagram of a digital image compositing system in accordance with the invention.
  • Figure 2 is a block diagram of an editing system incorporating the digital image composition system of Figure 1.
  • Figure 3 is a block diagram of a portion of the digital image compositing system of Figure 1.
  • Figures 4A-4D are graphs showing input-output characteristics from the system portion of Figure 3.
  • Figure 5 is a diagram showing relations between video images and selective keying.
  • Figures 6A-6E are probability distribution diagrams showing different forms of signal processing in the digital image compositing system of Figure 1.
  • Figure 7 is a block diagram of a video editing system incorporating a plurality of the digital image compositing systems of Figure 1.
  • Figure 8 is a signal flow diagram useful for understandingoperationof thedigital imagecompositing system of Figure 1.
  • Figure 9 is a more detailed signal flow diagram of a portion of the signal flow diagram of Figure 8.
  • Figure 10 is a signal flow diagram representing a simplification of the signal flowportion shown in Figure 9.
  • FIGS 11A-11L are charts showing inputs to and outputs from the signal flow diagram of Figure 10.
  • Figures 12A-12G are diagrams of input-output characteristics froma portion ofthe signal flowdiagram of Figure 9.
  • Figure 13 is a block diagram of another portion of the digital image compositing system of Figure 1.
  • Figures 14A-14B are frequency spectra diagrams useful for understanding operation of the systemportion of Figure 13.
  • Figure 15 is a signal flow diagram of another embodiment of the digital image compositing system of Figure l.
  • Figure 16 is a plan view of a keyboard and display for use with the digital image compositing system of Figures 1 and 14.
  • FIG. 1 there is shown a digital image compositing system 20 having background, foreground, key and mask busses 22, 24, 26, and 28.
  • An analog key input 30 is connected to the key and mask busses 26 and 28 through an analog/digital (A/D) converter 32 by crosspoint switches 34.
  • A/D analog/digital
  • Four digital DI video signal inputs 36 are connected to the busses 22-24 through additional crosspoint switches 34.
  • the key and mask busses 26 and 28 are respectively connected to frame stores 38 and 40.
  • a key processing toolkit subsystem 42 is connected to the output of the frame stores 38 and 40.
  • a compositor subsystem 44 is connected to the background and foreground busses 22 and 24 and to the output of the key processing toolkit subsystem 42.
  • a DI program output 46 is connected to the output of the compositor subsystem 44 on program bus 48.
  • a DI key output 50 is connected to the output of key processing toolkit subsystem 42 on processed key bus 52.
  • a DI auxiliary output 54 is connected to program bus 48, background bus 22, foreground bus 24, processed key bus 52 and mask bus 28 by crosspoint switches 56.
  • An analog output 58 is also connected to the busses 48, 22, 24, 52 and 28 by additional crosspoint switches 56 through D/A converter 60.
  • a microprocessor 62 is connected to provide control signals for all of the functional elements of the system 20. To avoid confusion in the drawing, the control connections are not shown.
  • a control keyboard and display 64 is connected to the microprocessor at 66.
  • Themicroprocessor 62 is also connectedto a video editor at 68.
  • the microprocessor 62 is desirably implemented with an NEC V50 microprocessor integrated circuit.
  • the system 20 is able to accept DI digital video signals at the inputs 36 and provide DI digital video signals at the outputs 46, 54 and 58 in a plurality of video standards, for example, in either 525-line NTSC standard form or 625-line PAL standard form.
  • the DI digital video signal standard is contained in DI Standard CCIRRecommendations Recommendation 601.
  • the 525-line NTSC DI digital video signal implementation is described in SMPTE Recommended Practice RP125-1984.
  • the corresponding 625-line PAL DI digital video signal implementation is described in EBU Parallel Interface for 625 Line Digital Video Signals, Tech.3246-E.
  • the system 20 processes the digital video signals at a 10-bit resolution, in order to give better resolution than the conventional 8 or 9-bit processing.
  • Figure 2 shows atypical applicationconfiguration, in which the digital image compositing system 20 is connected to DI digital video signal sources 70 and 72 to provide the foreground and background inputs 36 and to an analog video signal source 74 to provide the key input 30.
  • the program output 46 is connected to a digital DI video tape recorder (VTR) 76, and the analog output 58 is connected to a video monitor 78.
  • VTR digital DI video tape recorder
  • Figures 3-5 show how selective bypass/keying by timed selection of look up tables (LUTs) is implemented in the key processing toolkit subsystem 42 and in the compositor 44.
  • the background input 24 and the foreground input 22 are supplied to a difference circuit 140.
  • the output of the difference circuit 140 is supplied to a multiplier 142.
  • the other input to the multiplier 142 is supplied from look up table 144, based on the key input 26.
  • the output of the multiplier 142 is supplied to a summing circuit 146.
  • the other input to the summing circuit is the foreground signal 22.
  • the output of the summing circuit 146 is the program signal.
  • the program signal is obtained in accordance with the following formula:
  • FIGS. 4A-4D show look-up table input/output (I/O) characteristics LUT-1 through LUT-4 , respectively representing outputs 1.0, K, 1-K and 0.0, stored in the look-up table 144.
  • the foreground, background and key inputs 22, 24 and 26 represent the signal elements of a TV frame.
  • Represented at 148 and 150 are two selections of look-up table I/O characteristics from the look-up table 144 to produce two different TV frames, each divided up into a top portion 152 and a bottom portion 154, with different look-up table characteristics selected for the portions 152 and 154, timed to produce the portions 152 and 154 during scanning of the TV frame.
  • Shown at 156 and 158 are the resulting program output signals in the portions 152 and 154 for the two TV frames, as calculated by the above formula.
  • the timed selection of look-up tables is used to produce different effects in separate areas of a video frame.
  • the technique is not limited to separate areas, and can be used to generate a wide variety of interesting video effects.
  • Figures 6A through 6E show how variable randomized rounding is used to accommodate limited word length in video signal processing by eliminating fractional values. Rounding in accordance with the conventional rule that fractions equal to one-half or more are rounded up to the next integer and fractions of less than one-half are rounded down to the last integer yields the least variance of error, but the most noticeable false contour in the picture.
  • the probability density function of error for rounding is shown in Figure 6A, and is equal for all error values between -1/2 and +1/2. Dither, i.e. , randomly increasing any fraction to the next digit or decreasing it to the last digit, makes false contour the least noticeable, but yields the largest variance of error by spreading the error distribution from -1 to +1 .
  • Variable randomized rounding provides selectable tradeoffs between the size of the error and the level of false contour. Depending on the magnitude of the fractional values, dithering is selectively applied. For example, l. If the fraction is close to 0, then discard it.
  • Figure 7 shows how two or more of the video compositing systems 20, 20' can be cascaded to provide on-line multiple layering of images with a fixed overall delay of, for example, one frame.
  • the key and mask inputs 26 and 28 are provided at a time corresponding to the beginning of the first line (IH) of a video frame to the frame stores 38 and 40.
  • the background and foreground inputs 22 and 24 are similarly provided to first in, first out (FIFO) memories 160 and 162.
  • the outputs of the frame stores 38 and 40 and the FIFO memories 160 and 162 are provided after a delay Tl of one line (IH) to the keying process and compositor subsystems 42, 44 for processing, which results in an additional delay T2 of four lines (4H) .
  • Program and key outputs of the keying process and compositor subsystems 42, 44 are supplied at time 5H to delay memories 164 and 166, where they undergo an additional delay T3 of one line (IH) .
  • the total delay in the compositing system 20 is thus 6H.
  • the program output 48 from the compositing system 20 is supplied as a background input to a IH delay T5 FIFO memory 160' in the second compositing system 20' for a second layer of processing. Additional foreground, key and mask inputs 24' , 26' and 28' are supplied to FIFO memory 162', frame store 38* and frame store 40', each of which has a 7H delay T4.
  • the outputs of the FIFO memories 160' and 162' and the frame stores 38' and 40' thus are provided after a total delay of 7H to keying process and compositor subsystems 42',44', which have a processing delay of about 4H, supplying their program and key outputs to delay memories 164' and 166' after a total delay of 11H.
  • the delay memories 164' and 166' add a delay T6 of 514H, giving a total delay of 525H, equal to one television frame, for the program and key outputs 48' and 52'.
  • a third video compositing system 20" is used for single level compositing, receiving the background, foreground, key and mask video inputs 22-28, respectively supplied to FIFO memories 160" and 162" and frame stores
  • Delay memories 164" and 166" provide a delay T8 equal to (521-N)H, so that the program and key outputs 48" and 52" fromthevideo compositing system
  • the video compositing systems 20, 20' and 20" are synchronized by the REF SYNC signal supplied to gen-lock sync generators 168, 168' and 168".
  • Each of the video compositing systems 20, 20' and 20" takes about 6H of minimum signal propagation delay.
  • the FIFO and framestore memories 160-162 and 38-40, 160'-162' and 38'-40' and 160"-162" and 38"-40" at the input to each video compositing system 20, 20' and 20" and signal delay memories 164-166, 164'-166' and 164"-166" at the output of each system 20, 20' and 20" allow a wide range of input and output timing adjustment.
  • FIG. 8 shows details of the key processing toolkit 42 and the comparator 44 of Figure 1.
  • the key and mask inputs 26 and 28 are supplied through the frame stores 38 and 40.
  • the output of the frame store 38 is supplied through a key disable/enable switch 80 through a key inversion/noninversion circuit 82.
  • the output of the key inversion/noninversion circuit 82 is supplied to a multiplier 96.
  • the output of the frame store 40 is supplied to a mask inversion/noninversion circuit 86, the output of which is supplied through a mask enable switch 88 to a multiplier 90.
  • the other input 92 to the multiplier 90 is a fade control signal.
  • the output of the multiplier 90 is a processed mask signal, which is supplied through a mask multiplication enable/disable switch 94 to the multiplier 96. In most cases, the processed mask signal is supplied in this path, so that it is used to modify the key signal prior to key signal processing.
  • the output of the multiplier 96 is supplied to a key processing subsystem 97.
  • the output of the key processing subsystem 97 is supplied to a multiplier 104.
  • the multiplier input to the multiplier 104 is the output of a second mask multiplication enable/disable switch 106.
  • the second switch 106 is used to supply the mask input in certain special cases, such as for the generation of drop shadows in an image. Otherwise, the multiplier 104 passes the output of the interpolator 102 unchanged.
  • the output of the multiplier 104 is supplied to an inversion/noninversion circuit 110, the output of which is a processed key signal, which is supplied at 114 to a multiplier 122 for keying.
  • the output of the multiplier 104 is also supplied to an inversion/noninversion circuit 116 for supplying a processed key output at 120 through a delay memory 124, which may or may not be the same as the processed key output at 114.
  • the background and foreground inputs 22 and 24 are supplied to FIFO memories 126 and 128.
  • the output of the FIFO memory 126 is supplied to a background video processing subsystem 130.
  • the output of the video processing subsystem 130 is supplied to a background video/fill select switch 132, which allows user selection of the background video or a fill input 134.
  • the output of the select switch 132 is supplied to a subtracter 136 and as a processed background video output.
  • the output of the FIFO memory 128 is supplied to a foreground video processing subsystem 138, the output of which is supplied to a foreground video/fill select switch 170, which allows user selection of the foreground video or a fill input 172.
  • the output of the select switch 170 is supplied to a non-additivemix (nam) or linear mix/matte select switch 174 and also as a foregroundvideo output.
  • the output of the select switch 174 is supplied to the subtracter 136.
  • the output of the subtracter 136 is supplied to the multiplier 122.
  • the output of the multiplier 122 is supplied to an adder 176, the other input of which is the foreground video output.
  • the output of the adder 176 is supplied through a delay memory 178 as the keyed output program signal. Further details of the key processing toolkit subsystem 42 are shown in Figure 9.
  • the key mask signal from frame store 40 is supplied through a gain and offset adjustment circuit 84 to themask inversion/noninversion circuit 86.
  • the output ofthekey inversion/noninversion circuit 82 is the multiplicand input to the multiplier 96.
  • the output of the multiplier 96 is supplied to a noise reduction filter 98 and then to a clipping circuit 100.
  • the output of the clipping circuit 100 is supplied to a key horizontal/vertical edge softening filter/interpolator 102.
  • the same filter circuit isused with different coefficients for both filters.
  • the filter/interpolator 102 is implemented with a two- dimensional finite impulse response filter. Its structure is modified for two dimensions. For edge softening and interpolation, the same filter structure, driven by different coefficients supplied by the microprocessor, is used.
  • the filter/interpolator 102 may be implemented as a variable bandwidth, low pass filter and interpolator, using multiple TRW TMC 2246 integrated circuits.
  • the output of the interpolator 102 is supplied as a multiplicand to a multiplier 104.
  • the filter 98, clipping circuit 100 and filter/interpolator 102 comprise the key processing subsystem 97.
  • the key signal is supplied to a look up table selector 112 for supplying the processed key signal at 114 for keying.
  • the key signal may be separately processed for inversion/non-inversion at 116 and supplied to another look up table selector 118 for supply as the separate processed key output at 120.
  • Figure 10 represents a simplification of the Figure 9 subsystem 42 to show the effects of the inversion/non ⁇ inversion 82, 86 and 110 or 116 and the multipliers 96 and/or 104.
  • Figures 11A-11L show variations of area selections achievable through key and/or mask inversion as a result of the effects produced with the elements shown in Figure 10.
  • the representation x ⁇ means inverted x.
  • FIGS 12A-12G show the results of microprocessor 62 (Figure 1) control of the clipping filter 100 in Figure 9.
  • Figures 12A-12G show the results of microprocessor 62 ( Figure 1) control of the clipping filter 100 in Figure 9.
  • the designations shown have the following meanings:
  • Figure 12A represents the Cartesian subspace in which input and output characteristics are defined.
  • Figures 12B-12G represent the input and output characteristics produced in accordance with the following rules:
  • Figure 12B is a typical example of clipping.
  • Figures 12D or 12G are realizable to give a large amount of freedom to the user.
  • Figure 13 provide further details on themultiplier 122 ( Figure 8) of the compositor 44 and 14A-14B show certain relationships between componentvideo keyingand signal frequency spectrum, useful for understanding operation of the multiplier 122.
  • Background and foreground component video luminance signals (Y) and chrominance signals (U, V) are supplied to subtracters 180, 182 and 184.
  • the outputs of the subtracters 180, 182 and 184 are respectively supplied to interpolators and frequency doublers 186, 188 and 190.
  • the outputs of the interpolators and frequency doublers 186, 188 and 190 are supplied to multipliers 192, 194 and 196.
  • the key signal is supplied directly to the multipliers 194 and 196 and, through interpolator and frequency doubler 198, to the multiplier 192.
  • the outputs of the multipliers 192, 194 and 196 are supplied through decimators 200, 202 and 204 to summing circuits 206.
  • the three summing circuits 206 correspond to the single summing circuit 176 in Figure 8.
  • the interpolators and frequency doublers 186, 188, 190 and 198 and the decimators 200, 202 and 204 are implemented with TRW TMC 2242 integrated circuits.
  • the multiplier 122 processes the component video signals as follows: Y component
  • FIGS. 14A and 14B show the signals and their processing.
  • the input K:4 and Y:4 signals show the baseband frequency 210 corresponding to the Nyquist limit of 1/2 the sampling frequency 212, along with the second harmonic 214 of the sampling frequency 212.
  • the two K and Y input signals have their sampling frequency doubled to give a gap between the baseband frequency 210 and the doubled sampling frequency 216. As a result, spectra overlap between the keyed baseband frequency 218 and the keyed sampling frequency 220 is avoided.
  • the keyed signals are then low pass filtered to give a gap between the low pass filtered baseband frequency 222 and the low pass filtered sampling frequency 224.
  • the low pass filters are part of the decimators 200, 202 and 204.
  • the frequency of the keyed signals is halved, a clean output of the K*Y:4 baseband frequency 226, sampling frequency 228 and second harmonic 230 of the sampling frequency is obtained.
  • the input C:2 signals 232 are frequency doubled to give gaps between the doubled baseband frequency 234, the doubled sampling frequency 236 and the doubled second harmonic 238 of the sampling frequency 236.
  • Figure 15 shows another form of signal flow for a version of the video compositing system 20 of Figure 1 in which a switch 260 is provided between the multiplier
  • the keying multiplier 122 uses interpolation and decimation filters to avoid the introduction of aliasing signals. However, such filters inevitably alter signal quality to some extent, and their outputs cannot be identical of the inputs. This impurity of output signal fromthekeyingmultiplier 122 causes twoproblems when 100% background signal is the intendedkeyedoutput, i.e.
  • Figure 16 shows a control panel 220 for the video compositing system 20.
  • the control panel 220 is housed in a single construction and consists of a 8x42 character display 222, with both character and graphics capabilities, four shaft encoders 224, a trackball 226, a keyboard 228 and a buzzer for audio feedback to the user.
  • the system 20 responds in real time to the shaft encoders 224 or key depressions on the keyboard 228.
  • the keyboard 228 consists of three functionally separate sets of keys, namely, the softkeys 230, the numeric keys 232 and the function keys 234.
  • the row of five softkeys 230 are located immediately below the display panel 222. Their functionality is context dependent, and any softkey 230 may be enabled or disabled for a given display. Consider Table 1.
  • Table 1 represents the state of the control panel during user selection of the Digital Output signal.
  • all of the five softkeys 230 have been enabled, and are prompting the user to choose between the signal sources, namely, CMPST — Composite Signal
  • the appropriate menu for any video compositing system 20 operations is activated by depressing a function key 234.
  • These function keys 234 provide instant access to their respective displays and automatically enable the correct softkeys 230.
  • the function keys 234 are located in a 10 x 4 matrix just beneath the row of soft keys, and are clearly engraved.
  • numeric input is required, and this is provided by a 4 x 4 matrix of numeric keys 232 located to the right of the function keys 234.
  • the numeric keypad consists of the familiar digits 0 - 9, a sign key, a decimal point, and a clear key.
  • Numbers entered via the numeric keys 232 are displayed, as soon as they are entered, in the top right corner of the display 222 (assuming that the keypad is enabled) and are enclosed in square brackets. Table 3 shows a display after 0.75 has just been entered. The softkeys 230 are then used to carry this number across to the appropriate entry, for example, [SUB-V] in Table 3. KEY - POSITIONING [0.75]
  • Table 3 Often, the user needs to choose the correct numbers for his or her application visually. In other words, thenumbers are chosenby observingthe changes generated in the output signals of the video compositing system 20.
  • four shaft encoders 224 have been provided, and are located in a 2 x 2 array to the right hand side of the display 222. They are arranged in this manner to coincide with the four variables as shown in Table 3. Namely, the top left encoder 224 corresponds to "Sub-Vertical" entry, the bottom right to "Horizontal" and so on. Values are increased by rotating the shaft encoders 224 in a clockwise direction, and decreased by a counter-clockwise rotation. Their sensitivity is context dependent. As in the case of softkeys 230, not all encoders are enabled for each and every menu (see Table 4) .
  • the trackball 226 is located to the far right of the numeric key pad and may be used instead of two shaft encoders to control positional input in two planes simultaneously. As before it is only enabled in certain menus. Finally, a buzzer is provided, to warn the user of any attempt to enter incorrect numeric values.
  • control panel display 222 shows a simple banner message as shown in Table 5 and awaits user input via the keyboard 228, the four shaft encoders 224 or the trackball 226.
  • foreground video selection is accomplished by depressing the Foreground Select function key 234. In response to this request the display panel
  • Table 6 Input source selection may be altered by depress 15 the softkey just below the word INPUT (see Table All five softkeys 230 now become available for the f selections shown at the bottom of Table 7.
  • numeric values may be changed in either of two ways. By rotating the shaft encoder 224 which bears the same positional relationship to the corresponding display entry, values may be altered incrementally. Alternatively, numbers may be entered with the numeric keys 232. Table 11 shows the display 222 after the number 58 has been entered. Note that it is displayed in the top right corner and is enclosed in square brackets. The square brackets serve to draw a correspondence between the number entered and the (in this case) four possible destinations. These destinations are selected by depressing the appropriate softkeys, which are also enclosed in square brackets.
  • Lum Gain 25 %
  • Lum Gain 25 %
  • Table 12 Typically the two methods are used together, the keypad to enter coarse values, and the shaft encoders 224 to do fine tuning. On returning to the preceding menu, changes are reflected in the status information (Table 13) .
  • Table 13 The general philosophy maintained throughout the user interface is to include only those values and prompts which are pertinent to current selection system setup. For example should color fill be selected by a depression of the COLOR softkey 230 (see Table 14) , then the display takes on the form of Table 15.
  • Table 15 Backtracking one level of menu results in a somewha different display (see Table 16) which should b contrasted with Table 6.
  • Table 16 Backtracking one level of menu results in a somewha different display (see Table 16) which should b contrasted with Table 6.
  • Table 16 Backtracking one level of menu results in a somewha different display (see Table 16) which should b contrasted with Table 6.
  • those adjustment relevant to Color Fill selection are shown, namely, th red, green and blue components of the fill color, whic may be altered in an analogous manner, by selection of the ADJUST softkey 230, and numeric value modificatio as described above (see Table 17) .
  • Table 17 As a further alternative, a change of units is possible through the UNITS softkey 230. The values and units are changed accordingly (see Table 18) and remain the preferred units in status displays until a further units change is requested (see Table 19) .
  • Table 21 Finally, a Test Pattern source is provided, for diagnostic purposes (see Tables 22-23) .
  • the source of the mask input is selected along very similar lines to foreground and background input. From any menu the Mask Select button is depressed resulting in a display as shown in Table 24.
  • the status display includes additional information as to whether the incoming signal is to be taken from the luminance or chrominance component, and whether this component is to be inverted or not.
  • the source of the mask may be selected from any one of the four DI inputs 36 ( Figure 1) , an analog input, a shape generator or the Test Pattern generator. This is achieved by depressing the INPUT softkey 230 (see Table 25) .
  • CMPT Chrominance
  • Table 27 The incoming mask signal can be disabled and replaced by a constant source by toggling the DISABLE / ENABLE softkey 230. A display corresponding to Table 28 is obtained for the disable state.
  • the mask itself may be repositioned by first selecting the POSITION softkey 230, and then adjusting the horizontal and vertical controls via the shaft encoders 224 or numeric keys 232 in the now familiar way (see Table 29) .
  • the source of the key input is selected in the same way as the mask, except for a few refinements.
  • the user interface groups these six cont elements into two user menus which can be interchan
  • a single menu has been dedicated to composition control. Numerous effects can be realized by the video composition system 20, through the control of the mask and key signals. Manipulation of the foreground and backgroundvideo, color fills or blackto generatematte, mix, fades etc. is easily achieved.
  • the Composition Control button leads one to the control menu as shown in Table 34.
  • Each softkey 230 on this menu toggles between the alternative entries, except for the [FADE] .
  • Table 34 As an example, consider the key itself. Repeated depressions of the KEY softkey 230 cycles one through the three alternatives, namely, Normal, Inverted and Off (see Tables 34-36) .
  • the MASK softkey 230 performs an identical function for the mask signal.
  • the percentage of mix between the foreground and background is controlled through the [FADE] softkey 230 via numeric key 232 entry, or the relevant shaft encoder 224.
  • the Cut button 234 alternates the Fade level between 0% and 100% (see Tables 37-38) .
  • the key signal may be further modified by coring and softening. These controls are accessed by the Coring and Softening button 234. In essence the coring level selects a low pass filter profile. In addition the horizontal and vertical edges of the key shape may be softened through the two controls provided (see Table 39) .
  • Transitions are controlled by specifying a transition profile and a duration. These are accessed by depressing the Auto Transition button 234 to give the display shown in Table 40. As before, both numeric keys 232 and shaft encoders 224 may be employed for data input (see Table 40) .
  • the first menu to be displayed is shown in Table 41. From here one chooses either the KEY or CMPST softkey 230, to give displays indicated in Tables 42 and 43 respectively.
  • the video compositing system 20 provides five digital output choices from "composite", foreground, background, key and mask signals. Selection is made by first depressing the Digital-2 Output selection button 234 and then the softkey 230 corresponding to signal required (see Table 44)
  • Analog utput Selection An analog output is provided to monitor the outgoing signals. Choices are made via the Analog Output button 234 in the same way as for Digital-2 Output selection.
  • the digital image compositing system of this invention is a component digital single-layer high accuracy compositing module that can stand alone or be an integral part of a large system.
  • New techniques for combining layers provide a level of compositing performance and quality that has never been achieved before.
  • the system can be used for all DI compositing applications as well as be the center of a very cost effective DI editing suite.
  • the system provides advanced capabilities in layering performance.
  • New digital processing techniques preserve foreground and background edge detail, thereby achieving unsurpassed layering realism.
  • the system is easily integrated into digital and analog component systems where digital component editing is required. It can be the center of new DI editing and compositing suites by replacing more costly DI switchers. For graphics applications, it provides real-time layering at a lower price than alternative means.
  • the system replaces a switcher for preview or production compositing while pulling mattes.
  • Matte keying simulates the optical matte process which leaves the edges of the foreground video unprocessed, thereby preserving the foreground's edge detail.
  • the system can be controlled through RS-422 and GPI ports.
  • Serial RS-422 ports allow an external device to emulate the system's control panel and allow the system to be controlled as if it were a Grass Valley Group 100 series switcher.
  • the functions of the GPIs can be assigned by the operator.
  • the system can be controlled using an internal time line. Events which are assigned to the system's time line are automatically initiated and can be stepped through on a field-by-field basis.
  • the time line can be programmed from the control panel or by an external controller.
  • the delay through a single system is exactly one frame.
  • the delay through each system is adjusted to maintain one frame delay through the completed cascaded array.
  • a complete kit of tools is provided to modify the key signal. H and V position is adjustable in sub-pixel increments to match key to foreground. To improve the transitions between foreground andbackground, adjustable
  • H and V edge softening is provided.
  • Upper and lower clip points can be set over the complete range of the key signal to adjust the keying effect. Used with the clip points, wide-range gain and offset adjustments also provide a powerful way to adjust the look of the keying process. To smooth edges from analog keys, noise reduction is available. Inversion of the key and mask is also provided.
  • Full adjustment of foreground and background parameters is provided, including luminance gain and offset and chrominance saturation and hue rotation.
  • Timed events are provided for transitions between sources. These include cuts and dissolves between foreground, background and fill. Also, keys can be faded up or down.
  • the foreground and background fill can be adjusted and programmed in RGB and YUV values. The system operates in both 525 and 625 4:2:2 standards. Selection is automatic or can be manually set by the operator.
  • Keyboard macros can be programmed and assigned to the time line or GPIs.
  • An analog luminance input is provided to accept analog key and mask sources. This input is converted to 10 bit 4:0:0 and is connectable to the internal key and mask busses through key and mask frame stores. A full quality 10 bit RGB, YUV or Betacam output is provided for monitoring and recording on analog VTRs.
  • the digital image compositing system and method incorporates flexible input and output crosspoint switching. Digital inputs and outputs are provided for all video signals.
  • the digital image compositing system and method will accept analog key or mask input signals.
  • the digital image compositing system and method will additionally provide any output video signal in analog form.
  • the digital image compositing system and method can be cascaded to provide real time multiple layer compositing.
  • the digital image compositing system and method incorporates a user interface that facilitates use of the flexibility provided by the system and method.

Abstract

A digital image compositing system (20) has background, foreground, key and mask busses (22, 24, 26 and 28). An analog key input (30) is connected to the key and mask busses (26 and 28) through an analog/digital (A/D) converter (32) by crosspoint switches (34). Four digital D1 video signal inputs (36) are connected to the busses (22-24) through additional crosspoint switches (34). The key and mask busses (26) and (28) are respectively connected to frame stores (38) and (40). A key processing toolkit subsystem (42) is connected to the output of the frame stores (38) and (40). A compositor subsystem (44) is connected to the background and foreground busses (22) and (24) and to the output of the key processing toolkit subsystem (42). A D1 program output (46) is connected to the output of the compositor subsystem (44) on program bus (48). A D1 key output (50) is connected to the output of key processing toolkit subsystem (42) on processed key bus (52). A D1 auxiliary output (54) is connected to program bus (48), background bus (22), foreground bus (24), processed key bus (52) and mask bus (28) by crosspoint switches (56).

Description

DIGITAL IMAGE COMPOSITING SYSTEM AND METHOD
BACKGROUND OF THE INVENTION
1. Field of the Invention:
The present invention relates generally to digital compositing, i.e., the layering or putting together of" images in a digital video domain. More particularly, it relates to a system and method for digital image compositing which gives the user an increased flexibility for such compositing. The invention further relates to a user interface that enables the user to take advantage of the increased flexibility more conveniently.
2. Description of the Prior Art:
A variety of systems are known in the art for combining separate video images using a key to provide a foreground image superimposed on a background. For example, such systems are disclosed in Greenburg, U.S. Patent 4,357,624 and Kellar et al., U.S. Patent 4,602,286. It is further known to carry out various forms of soft edge keying to provide a realistic transition between the foreground image and the background image. One form of soft edge keying is disclosed in the Kellar et al. patent, and in Bennett et al. , U.S. Patent 4,463,373. Other forms of soft edge keying are disclosed in the following issued U.K patents and published U.K. patent applications: 1,178,330; 1,226,559; 1,570,773; 2,032,217; 2,092,346 and 2,109 ,193. In particular, one technigue for soft edge keying that has particular utility is the use of sub-pixel interpolation, as disclosed in published European application 0 360 559. Other techniques disclosed in the prior art that are useful in digital video image compositing include clock rate doubling and adaptive decimation, as disclosed in published European application 0360557, the use of modified keying signals to adjust boundaries between overlaid images, as disclosed in published European application 0360560, and the use of dynamic rounding, as disclosed in Owen, D.P. , "Dynamic Rounding in Digital Video Processing," SMPTE Journal, June 1989, pp.447-450. Various forms of control interf ces for video signal processing equipment are also known in the art. For example, such interfaces are disclosed in U.S. Patents 4,538,188; 4,763,186; 4,857,994 and 4,954,883.
While the art pertaining to digital video image compositing is thus a well-developed one, a need still remains for further improvement in such systems in order to provide improved composite images and to make such equipment easier to operate. SUMMARY OF THE INVENTION
Accordingly, it is an object of this invention to provide a digital image compositing system and method incorporating flexible input and output crosspoint switching.
It is another object of the invention to provide such a digital image compositing system and method in which digital inputs and outputs are provided for all video signals.
It is a further object of the invention to provide such a digital image compositing system and methodwhich will accept analog key or mask input signals. It is still another object of the invention to provide such a digital image compositing system and method which will additionally provide any output video signal in analog form. It is a still further object of the invention to provide such a digital image compositing system and method which can be cascaded to provide real time multiple layer compositing.
It is yet another object of the invention to provide such a digital image compositing system and method which incorporates a user interface that facilitates use of the flexibility provided by the system and method.
The attainment of these and related objects may be achieved through use of the novel digital image compositing system and method herein disclosed. A digital image compositing system in accordance with this invention has a plurality of digital video signal inputs, a key processing subsystem and a video image compositor. A first plurality of crosspoint switches connect the plurality of digital video signal inputs to the key processing subsystem and to the video image compositor. A key input is connected through the plurality of crosspoint switches to the key processing subsystem. A second plurality of crosspoint switches connect the key processing subsystem and the video image compositor to a plurality of digital video signal outputs.
The attainment of the foregoing and related objects, advantages and features of the invention should be more readily apparent to those skilled in the art, after review of the following more detailed description of the invention, taken together with the drawings, in which:
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 is a block diagram of a digital image compositing system in accordance with the invention. Figure 2 is a block diagram of an editing system incorporating the digital image composition system of Figure 1.
Figure 3 is a block diagram of a portion of the digital image compositing system of Figure 1.
Figures 4A-4D are graphs showing input-output characteristics from the system portion of Figure 3.
Figure 5 is a diagram showing relations between video images and selective keying. Figures 6A-6E are probability distribution diagrams showing different forms of signal processing in the digital image compositing system of Figure 1.
Figure 7 is a block diagram of a video editing system incorporating a plurality of the digital image compositing systems of Figure 1.
Figure 8 is a signal flow diagram useful for understandingoperationof thedigital imagecompositing system of Figure 1.
Figure 9 is a more detailed signal flow diagram of a portion of the signal flow diagram of Figure 8.
Figure 10 is a signal flow diagram representing a simplification of the signal flowportion shown inFigure 9.
Figures 11A-11L are charts showing inputs to and outputs from the signal flow diagram of Figure 10.
Figures 12A-12G are diagrams of input-output characteristics froma portion ofthe signal flowdiagram of Figure 9.
Figure 13 is a block diagram of another portion of the digital image compositing system of Figure 1.
Figures 14A-14B are frequency spectra diagrams useful for understanding operation of the systemportion of Figure 13. Figure 15 is a signal flow diagram of another embodiment of the digital image compositing system of Figure l.
Figure 16 is a plan view of a keyboard and display for use with the digital image compositing system of Figures 1 and 14.
DETAILED DESCRIPTION OF THE INVENTION
Turning now to the drawings, more particularly to Figure 1, there is shown a digital image compositing system 20 having background, foreground, key and mask busses 22, 24, 26, and 28. An analog key input 30 is connected to the key and mask busses 26 and 28 through an analog/digital (A/D) converter 32 by crosspoint switches 34. Four digital DI video signal inputs 36 are connected to the busses 22-24 through additional crosspoint switches 34. The key and mask busses 26 and 28 are respectively connected to frame stores 38 and 40. A key processing toolkit subsystem 42 is connected to the output of the frame stores 38 and 40. A compositor subsystem 44 is connected to the background and foreground busses 22 and 24 and to the output of the key processing toolkit subsystem 42. A DI program output 46 is connected to the output of the compositor subsystem 44 on program bus 48. A DI key output 50 is connected to the output of key processing toolkit subsystem 42 on processed key bus 52. A DI auxiliary output 54 is connected to program bus 48, background bus 22, foreground bus 24, processed key bus 52 and mask bus 28 by crosspoint switches 56. An analog output 58 is also connected to the busses 48, 22, 24, 52 and 28 by additional crosspoint switches 56 through D/A converter 60. A microprocessor 62 is connected to provide control signals for all of the functional elements of the system 20. To avoid confusion in the drawing, the control connections are not shown. A control keyboard and display 64 is connected to the microprocessor at 66. Themicroprocessor 62 is also connectedto a video editor at 68. The microprocessor 62 is desirably implemented with an NEC V50 microprocessor integrated circuit.
Under control of the microprocessor 62, the system 20 is able to accept DI digital video signals at the inputs 36 and provide DI digital video signals at the outputs 46, 54 and 58 in a plurality of video standards, for example, in either 525-line NTSC standard form or 625-line PAL standard form. The DI digital video signal standard is contained in DI Standard CCIRRecommendations Recommendation 601. The 525-line NTSC DI digital video signal implementation is described in SMPTE Recommended Practice RP125-1984. The corresponding 625-line PAL DI digital video signal implementation is described in EBU Parallel Interface for 625 Line Digital Video Signals, Tech.3246-E. The system 20 processes the digital video signals at a 10-bit resolution, in order to give better resolution than the conventional 8 or 9-bit processing. Figure 2 shows atypical applicationconfiguration, in which the digital image compositing system 20 is connected to DI digital video signal sources 70 and 72 to provide the foreground and background inputs 36 and to an analog video signal source 74 to provide the key input 30. The program output 46 is connected to a digital DI video tape recorder (VTR) 76, and the analog output 58 is connected to a video monitor 78.
Figures 3-5 show how selective bypass/keying by timed selection of look up tables (LUTs) is implemented in the key processing toolkit subsystem 42 and in the compositor 44. The background input 24 and the foreground input 22 are supplied to a difference circuit 140. The output of the difference circuit 140 is supplied to a multiplier 142. The other input to the multiplier 142 is supplied from look up table 144, based on the key input 26. The output of the multiplier 142 is supplied to a summing circuit 146. The other input to the summing circuit is the foreground signal 22. The output of the summing circuit 146 is the program signal. The program signal is obtained in accordance with the following formula:
PGM = FG + (1-K) * (BG - FG) = K * FG + (1-K) * BG Figures 4A-4D show look-up table input/output (I/O) characteristics LUT-1 through LUT-4 , respectively representing outputs 1.0, K, 1-K and 0.0, stored in the look-up table 144. In Figure 5, the foreground, background and key inputs 22, 24 and 26 represent the signal elements of a TV frame. Represented at 148 and 150 are two selections of look-up table I/O characteristics from the look-up table 144 to produce two different TV frames, each divided up into a top portion 152 and a bottom portion 154, with different look-up table characteristics selected for the portions 152 and 154, timed to produce the portions 152 and 154 during scanning of the TV frame. Shown at 156 and 158 are the resulting program output signals in the portions 152 and 154 for the two TV frames, as calculated by the above formula. In this example, the timed selection of look-up tables is used to produce different effects in separate areas of a video frame. However, the technique is not limited to separate areas, and can be used to generate a wide variety of interesting video effects. Figures 6A through 6E show how variable randomized rounding is used to accommodate limited word length in video signal processing by eliminating fractional values. Rounding in accordance with the conventional rule that fractions equal to one-half or more are rounded up to the next integer and fractions of less than one-half are rounded down to the last integer yields the least variance of error, but the most noticeable false contour in the picture. The probability density function of error for rounding is shown in Figure 6A, and is equal for all error values between -1/2 and +1/2. Dither, i.e. , randomly increasing any fraction to the next digit or decreasing it to the last digit, makes false contour the least noticeable, but yields the largest variance of error by spreading the error distribution from -1 to +1 . Variable randomized rounding provides selectable tradeoffs between the size of the error and the level of false contour. Depending on the magnitude of the fractional values, dithering is selectively applied. For example, l. If the fraction is close to 0, then discard it.
2. If the fraction is close to 1, then add 1 to the integer part.
3. If the fraction is close to 0.5, then apply dither. The choice of the width of the range around 0.5 in which dither is applied determines the level of randomness of the outcome. In Figure 6B, conventional rounding is used for fractions less than 3/8 or greater than 5/8, with randomizedroundingbetweenthosevalues. As shown, this decreases the probability of the greatest errors. In Figure 6C, conventional rounding is used for fractions less than 1/4 or greater than 3/4, and in Figure 6d, it is used for fractions less than 1/8 or greater than 7/8. The choice of which rounding mode to use is user selectable. Variable randomized rounding is applied to several points of the signal processing paths, such as at the ends of the foreground and background video processors, key processor and keying module.
Figure 7 shows how two or more of the video compositing systems 20, 20' can be cascaded to provide on-line multiple layering of images with a fixed overall delay of, for example, one frame. The key and mask inputs 26 and 28 are provided at a time corresponding to the beginning of the first line (IH) of a video frame to the frame stores 38 and 40. The background and foreground inputs 22 and 24 are similarly provided to first in, first out (FIFO) memories 160 and 162. The outputs of the frame stores 38 and 40 and the FIFO memories 160 and 162 are provided after a delay Tl of one line (IH) to the keying process and compositor subsystems 42, 44 for processing, which results in an additional delay T2 of four lines (4H) . Program and key outputs of the keying process and compositor subsystems 42, 44 are supplied at time 5H to delay memories 164 and 166, where they undergo an additional delay T3 of one line (IH) . The total delay in the compositing system 20 is thus 6H.
The program output 48 from the compositing system 20 is supplied as a background input to a IH delay T5 FIFO memory 160' in the second compositing system 20' for a second layer of processing. Additional foreground, key and mask inputs 24' , 26' and 28' are supplied to FIFO memory 162', frame store 38* and frame store 40', each of which has a 7H delay T4. The outputs of the FIFO memories 160' and 162' and the frame stores 38' and 40' thus are provided after a total delay of 7H to keying process and compositor subsystems 42',44', which have a processing delay of about 4H, supplying their program and key outputs to delay memories 164' and 166' after a total delay of 11H. The delay memories 164' and 166' add a delay T6 of 514H, giving a total delay of 525H, equal to one television frame, for the program and key outputs 48' and 52'.
If a third video compositing system 20" is used for single level compositing, receiving the background, foreground, key and mask video inputs 22-28, respectively supplied to FIFO memories 160" and 162" and frame stores
38" and 40". In the FIFO memories 160" and 162" and frame stores 38" and 40" these inputs are supplied to the keying process and compositor subsystems 42", 44" after a delay T7 equal to NH, where N is arbitrary.
Processing by the keying process and compositor subsystems 42", 44" results in a delay of about 4H, and the program and key outputs from the keying process and compositorsubsystems 42", 44" areprovidedafter atotal delay of (N + 4)H. Delay memories 164" and 166" provide a delay T8 equal to (521-N)H, so that the program and key outputs 48" and 52" fromthevideo compositing system
20" are provided at the same time as the program and key outputs 48' and 52' from the video compositing system
20'.
In operation, the video compositing systems 20, 20' and 20" are synchronized by the REF SYNC signal supplied to gen-lock sync generators 168, 168' and 168". Each of the video compositing systems 20, 20' and 20" takes about 6H of minimum signal propagation delay. The FIFO and framestore memories 160-162 and 38-40, 160'-162' and 38'-40' and 160"-162" and 38"-40" at the input to each video compositing system 20, 20' and 20" and signal delay memories 164-166, 164'-166' and 164"-166" at the output of each system 20, 20' and 20" allow a wide range of input and output timing adjustment. Tens of the video compositing systems can be cascaded to multiple layering, while maintaining exactly one TV frame of overall signal delay. Note that each internal video bus for the systems 20, 20' and 20" uses different signal processing in the keying process and compositor subsystems 42, 44, 42', 44' and 42", 44". What is indicated in the drawings is therefore conceptual in nature. Figure 8 shows details of the key processing toolkit 42 and the comparator 44 of Figure 1. The key and mask inputs 26 and 28 are supplied through the frame stores 38 and 40. The output of the frame store 38 is supplied through a key disable/enable switch 80 through a key inversion/noninversion circuit 82. The output of the key inversion/noninversion circuit 82 is supplied to a multiplier 96. The output of the frame store 40 is supplied to a mask inversion/noninversion circuit 86, the output of which is supplied through a mask enable switch 88 to a multiplier 90. The other input 92 to the multiplier 90 is a fade control signal. The output of the multiplier 90 is a processed mask signal, which is supplied through a mask multiplication enable/disable switch 94 to the multiplier 96. In most cases, the processed mask signal is supplied in this path, so that it is used to modify the key signal prior to key signal processing. The output of the multiplier 96 is supplied to a key processing subsystem 97. The output of the key processing subsystem 97 is supplied to a multiplier 104. The multiplier input to the multiplier 104 is the output of a second mask multiplication enable/disable switch 106. The second switch 106 is used to supply the mask input in certain special cases, such as for the generation of drop shadows in an image. Otherwise, the multiplier 104 passes the output of the interpolator 102 unchanged. The output of the multiplier 104 is supplied to an inversion/noninversion circuit 110, the output of which is a processed key signal, which is supplied at 114 to a multiplier 122 for keying. The output of the multiplier 104 is also supplied to an inversion/noninversion circuit 116 for supplying a processed key output at 120 through a delay memory 124, which may or may not be the same as the processed key output at 114. In the compositor 44, the background and foreground inputs 22 and 24 are supplied to FIFO memories 126 and 128. The output of the FIFO memory 126 is supplied to a background video processing subsystem 130. The output of the video processing subsystem 130 is supplied to a background video/fill select switch 132, which allows user selection of the background video or a fill input 134. The output of the select switch 132 is supplied to a subtracter 136 and as a processed background video output. The output of the FIFO memory 128 is supplied to a foreground video processing subsystem 138, the output of which is supplied to a foreground video/fill select switch 170, which allows user selection of the foreground video or a fill input 172. The output of the select switch 170 is supplied to a non-additivemix (nam) or linear mix/matte select switch 174 and also as a foregroundvideo output. The output of the select switch 174 is supplied to the subtracter 136. The output of the subtracter 136 is supplied to the multiplier 122. The output of the multiplier 122 is supplied to an adder 176, the other input of which is the foreground video output. The output of the adder 176 is supplied through a delay memory 178 as the keyed output program signal. Further details of the key processing toolkit subsystem 42 are shown in Figure 9. The key mask signal from frame store 40 is supplied through a gain and offset adjustment circuit 84 to themask inversion/noninversion circuit 86. The output ofthekey inversion/noninversion circuit 82 is the multiplicand input to the multiplier 96. The output of the multiplier 96 is supplied to a noise reduction filter 98 and then to a clipping circuit 100. The output of the clipping circuit 100 is supplied to a key horizontal/vertical edge softening filter/interpolator 102. The same filter circuit isused with different coefficients for both filters. The filter/interpolator 102 is implemented with a two- dimensional finite impulse response filter. Its structure is modified for two dimensions. For edge softening and interpolation, the same filter structure, driven by different coefficients supplied by the microprocessor, is used. The filter/interpolator 102 may be implemented as a variable bandwidth, low pass filter and interpolator, using multiple TRW TMC 2246 integrated circuits. The output of the interpolator 102 is supplied as a multiplicand to a multiplier 104. The filter 98, clipping circuit 100 and filter/interpolator 102 comprise the key processing subsystem 97. After gain and offset adjustment at 108 and inversion or non- inversion at 110, the key signal is supplied to a look up table selector 112 for supplying the processed key signal at 114 for keying. The key signal may be separately processed for inversion/non-inversion at 116 and supplied to another look up table selector 118 for supply as the separate processed key output at 120. Figure 10 represents a simplification of the Figure 9 subsystem 42 to show the effects of the inversion/non¬ inversion 82, 86 and 110 or 116 and the multipliers 96 and/or 104. Figures 11A-11L show variations of area selections achievable through key and/or mask inversion as a result of the effects produced with the elements shown in Figure 10. In Figures 10 and 11A-11L, the representation x~ means inverted x.
Figures 12A-12G show the results of microprocessor 62 (Figure 1) control of the clipping filter 100 in Figure 9. In these figures, the designations shown have the following meanings:
TT: Input top threshold level TB: Input bottom threshold level MT: Output top mapped level MB: Output bottom mapped level I : Any input
O : Output corresponding to I
Figure 12A represents the Cartesian subspace in which input and output characteristics are defined. Figures 12B-12G represent the input and output characteristics produced in accordance with the following rules:
1. If TT <= I then 0 = MT
2. If I < TT and TB < I then 0 = 1 3. If I < TT and I <= TB then 0 = MB
Such flexible clipping of the key signal allows verymany different input and output characteristics to be defined. Figure 12B is a typical example of clipping. Figures 12D or 12G are realizable to give a large amount of freedom to the user.
Figure 13 provide further details on themultiplier 122 (Figure 8) of the compositor 44 and 14A-14B show certain relationships between componentvideo keyingand signal frequency spectrum, useful for understanding operation of the multiplier 122. Background and foreground component video luminance signals (Y) and chrominance signals (U, V) are supplied to subtracters 180, 182 and 184. The outputs of the subtracters 180, 182 and 184 are respectively supplied to interpolators and frequency doublers 186, 188 and 190. The outputs of the interpolators and frequency doublers 186, 188 and 190 are supplied to multipliers 192, 194 and 196. The key signal is supplied directly to the multipliers 194 and 196 and, through interpolator and frequency doubler 198, to the multiplier 192. The outputs of the multipliers 192, 194 and 196 are supplied through decimators 200, 202 and 204 to summing circuits 206. The three summing circuits 206 correspond to the single summing circuit 176 in Figure 8. The interpolators and frequency doublers 186, 188, 190 and 198 and the decimators 200, 202 and 204 are implemented with TRW TMC 2242 integrated circuits.
In operation, the multiplier 122 processes the component video signals as follows: Y component
Y:4 -> Y:8 -> K:8 * Y:8 -> LPF -> K*Y:4 C (U and V) component
C:2 -> C:4 -> K:4 * C:4 -> LPF -> K*C:2 Figures 14A and 14B show the signals and their processing. The input K:4 and Y:4 signals show the baseband frequency 210 corresponding to the Nyquist limit of 1/2 the sampling frequency 212, along with the second harmonic 214 of the sampling frequency 212. The two K and Y input signals have their sampling frequency doubled to give a gap between the baseband frequency 210 and the doubled sampling frequency 216. As a result, spectra overlap between the keyed baseband frequency 218 and the keyed sampling frequency 220 is avoided. The keyed signals are then low pass filtered to give a gap between the low pass filtered baseband frequency 222 and the low pass filtered sampling frequency 224. The low pass filters are part of the decimators 200, 202 and 204. When the frequency of the keyed signals is halved, a clean output of the K*Y:4 baseband frequency 226, sampling frequency 228 and second harmonic 230 of the sampling frequency is obtained. Similarly, with the U and V components, the input C:2 signals 232 are frequency doubled to give gaps between the doubled baseband frequency 234, the doubled sampling frequency 236 and the doubled second harmonic 238 of the sampling frequency 236. However, because the input K:4 signals 240 are not frequency doubled, spectral overlap occurs between the keyed K*C:4 baseband frequency 242, sampling frequency 244 and second harmonic 246 of the sampling frequency. This overlap is not a problem, because the regions of the overlap are eliminated in the low pass filtering to give the lowpass filtered K*C:4 baseband frequency 248, sampling frequency 250 and second harmonic 252 of the sampling frequency 250. When the frequency of these signals is halved to give the K*C:2 baseband frequency 254, sampling frequency 256 and second harmonic 258 of the sampling frequency 256, a clean output is also obtained.
Figure 15 shows another form of signal flow for a version of the video compositing system 20 of Figure 1 in which a switch 260 is provided between the multiplier
122 and the summer 176, to bypass the background - foreground signal from the subtracter 136. The keying multiplier 122, as shown inFigure 13, uses interpolation and decimation filters to avoid the introduction of aliasing signals. However, such filters inevitably alter signal quality to some extent, and their outputs cannot be identical of the inputs. This impurity of output signal fromthekeyingmultiplier 122 causes twoproblems when 100% background signal is the intendedkeyedoutput, i.e.
Output = K * FG + (1-K) * BG
= FG + (1-K) * (BG - FG) when K — 0, Output = FG + 1 * (BG - FG)
Because of the filtering involved, 1 * (BG - FG) as the output from the keying multiplier section is not equal to (BG - FG) . As a result, the output consists of a slightly impure BG and some residue of FG. By adding a bypass for the (BG - FG) signal for the case when K = 0, the signal impurity problem is eliminated. Switch 260 is controlled by K such that select bypassed (BG - FG) if K = 0 (nearly equals 0), select keyed (BG - FG) otherwise. Other than as shown and described, the construction and operation of the Figure 15 embodiment is the same as that of the Figure 8 embodiment.
Figure 16 shows a control panel 220 for the video compositing system 20. The control panel 220 is housed in a single construction and consists of a 8x42 character display 222, with both character and graphics capabilities, four shaft encoders 224, a trackball 226, a keyboard 228 and a buzzer for audio feedback to the user. The system 20 responds in real time to the shaft encoders 224 or key depressions on the keyboard 228.
The keyboard 228 consists of three functionally separate sets of keys, namely, the softkeys 230, the numeric keys 232 and the function keys 234. The row of five softkeys 230 are located immediately below the display panel 222. Their functionality is context dependent, and any softkey 230 may be enabled or disabled for a given display. Consider Table 1.
DIGITAL OUTPUT-2 Selection
Output: Composite
Figure imgf000020_0001
CMPST FG BG KEY MASK
Figure imgf000020_0002
I I I I 1 I I I 1 I I I I I I I
Table 1
Table 1 represents the state of the control panel during user selection of the Digital Output signal. Here we see that all of the five softkeys 230 have been enabled, and are prompting the user to choose between the signal sources, namely, CMPST — Composite Signal
FG — Foreground Video signal BG — Background Video signal KEY — Key signal MASK — Mask signal. The user's last choice is displayed in the status line headed "Output: ". This format is typical of the interplay between display 222 and softkeys 230. The last line of the display panel is reserved entirely for this purpose. Not every softkey 230 will be enabled as Table 2 serves to demonstrate. In fact the general philosophy of the user interface is to keep every display as simple as possible, so as not to overpower the user. FOREGROUND - STATUS
Gain = 25 % Offset = 63 IRE
Port 2 - serial Hue = 58 degs] Sat = 13 %
INPUT ADJUST
Table 2
The appropriate menu for any video compositing system 20 operations is activated by depressing a function key 234. These function keys 234 provide instant access to their respective displays and automatically enable the correct softkeys 230. The function keys 234 are located in a 10 x 4 matrix just beneath the row of soft keys, and are clearly engraved.
Often, numeric input is required, and this is provided by a 4 x 4 matrix of numeric keys 232 located to the right of the function keys 234. The numeric keypad consists of the familiar digits 0 - 9, a sign key, a decimal point, and a clear key.
Numbers entered via the numeric keys 232 are displayed, as soon as they are entered, in the top right corner of the display 222 (assuming that the keypad is enabled) and are enclosed in square brackets. Table 3 shows a display after 0.75 has just been entered. The softkeys 230 are then used to carry this number across to the appropriate entry, for example, [SUB-V] in Table 3. KEY - POSITIONING [0.75]
Sub-Vertical = 0.5 Sub-Horizontal = 0.75 Vertical = 12 Ins Horizontal = 0 pix
[SUB-V] [VERT] [SUB-H] [HORIZ]
Table 3 Often, the user needs to choose the correct numbers for his or her application visually. In other words, thenumbers are chosenby observingthe changes generated in the output signals of the video compositing system 20. To aid the user in this, four shaft encoders 224 have been provided, and are located in a 2 x 2 array to the right hand side of the display 222. They are arranged in this manner to coincide with the four variables as shown in Table 3. Namely, the top left encoder 224 corresponds to "Sub-Vertical" entry, the bottom right to "Horizontal" and so on. Values are increased by rotating the shaft encoders 224 in a clockwise direction, and decreased by a counter-clockwise rotation. Their sensitivity is context dependent. As in the case of softkeys 230, not all encoders are enabled for each and every menu (see Table 4) .
MASK - POSITION []
Vertical = 26 Ins Horizontal = 14 pix
[VERT] [HORIZ]
Figure imgf000022_0001
Table 4 The trackball 226 is located to the far right of the numeric key pad and may be used instead of two shaft encoders to control positional input in two planes simultaneously. As before it is only enabled in certain menus. Finally, a buzzer is provided, to warn the user of any attempt to enter incorrect numeric values.
In use of the video compositing system 20, on power up the control panel display 222 shows a simple banner message as shown in Table 5 and awaits user input via the keyboard 228, the four shaft encoders 224 or the trackball 226.
Figure imgf000023_0001
Table 5
Foreground and Background Input Selection
Selection of foreground and background video is identical. For example, foreground video selection is accomplished by depressing the Foreground Select function key 234. In response to this request the display panel
222 shows the currently active selection, and any auxiliary information pertaining to that selection. As an example of the foreground signal coming from the first DI parallel input port see Table 6. Here the
' luminance gain and offset values are included for the user's convenience, together with the signal's hue and saturation levels. On the bottom display line two softkey prompts, INPUT and ADJUST, are displayed above the first and last softkey buttons 230, which are the enabled.
FOREGROUND - STATUS
Gain = 25 % Offset = 63 IRE
Port 1 - parallel Hue = 92 degs] Sat = 13 %
10
INPUT ADJUST
Table 6 Input source selectionmay be altered by depress 15 the softkey just below the word INPUT (see Table All five softkeys 230 now become available for the f selections shown at the bottom of Table 7.
FOREGROUND - INPUT
20
Input: Port 1- parallel
25
PORT CONFIG COLOR BLACK TEST
Table 7
Should one wish to change the current input sou
30 for any one of the four other DI input ports, one or m depressions of the PORT softkey 230 are required. central status line will rotate through them on repea depressions (see Table 8) . FOREGROUND - INPUT
Input: Port 2- parallel
PORT CONFIG COLOR BLACK TEST
Table 8 Additionally, either the serial or parallel version of this port may be chosen. The CONFIG button 230 toggles between these two possibilities (see Table 9) .
FOREGROUND INPUT
Input: Port 2- serial
PORT CONFIG COLOR BLACK TEST
Table 9
Returning the foreground select menu using the Last Menu or Foreground Select button 234 (see Table 6) , and selecting the ADJUST softkey, one can modify the luminance and color values (see Table 10) .
FOREGROUND - ADJUST [] Lum Gain = 25 % Lum Offset = 63 IRE Hue = 92 degs Saturation = 13 %
[GAIN] [HUE] [UNITY] [OFFSET] [SAT]
Table 10 Typically, numeric values may be changed in either of two ways. By rotating the shaft encoder 224 which bears the same positional relationship to the corresponding display entry, values may be altered incrementally. Alternatively, numbers may be entered with the numeric keys 232. Table 11 shows the display 222 after the number 58 has been entered. Note that it is displayed in the top right corner and is enclosed in square brackets. The square brackets serve to draw a correspondence between the number entered and the (in this case) four possible destinations. These destinations are selected by depressing the appropriate softkeys, which are also enclosed in square brackets.
FOREGROUND - ADJUST [58]
Lum Gain = 25 % Lum Offset = 63 IRE Hue = 92 degs Saturation = 13 %
[GAIN] [HUE] [UNITY] [OFFSET] [SAT]
Table 11 Table 12 shows the result of choosing the softkey [HUE]. The number (58) is transferred to the Hue = XX status entry and the numeric store is automatically cleared.
FOREGROUND - ADJUST []
Lum Gain = 25 % Lum Offset = 63 IRE Hue = 58 degs Saturation = 13 %
Figure imgf000026_0001
[GAIN] [HUE] [UNITY] [OFFSET] [SAT]
Figure imgf000026_0002
Table 12 Typically the two methods are used together, the keypad to enter coarse values, and the shaft encoders 224 to do fine tuning. On returning to the preceding menu, changes are reflected in the status information (Table 13) .
FOREGROUND - STATUS
Port 2 - serial
INPUT
Figure imgf000027_0002
Table 13 The general philosophy maintained throughout the user interface is to include only those values and prompts which are pertinent to current selection system setup. For example should color fill be selected by a depression of the COLOR softkey 230 (see Table 14) , then the display takes on the form of Table 15.
FOREGROUND - INPUT
Input: Port 2- serial
PORT CONFIG COLOR BLACK TEST
Figure imgf000027_0001
Table 14
FOREGROUND - INPUT
Input: Color
PORT CONFIG COLOR BLACK TEST
Table 15 Backtracking one level of menu results in a somewha different display (see Table 16) which should b contrasted with Table 6. Here only those adjustment relevant to Color Fill selection are shown, namely, th red, green and blue components of the fill color, whic may be altered in an analogous manner, by selection of the ADJUST softkey 230, and numeric value modificatio as described above (see Table 17) .
FOREGROUND - STATUS
R = 98 %
Color G = 100 % B = 32 %
INPUT ADJUST
Table 16
FOREGROUND - COLOR FILL [] Units: RGB R = 98 % G = 100 % B = 32 %
UNITS [G] [R] [B]
Table 17 As a further alternative, a change of units is possible through the UNITS softkey 230. The values and units are changed accordingly (see Table 18) and remain the preferred units in status displays until a further units change is requested (see Table 19) .
FOREGROUND - COLOR FILL [3 Units: YUV Y = 78
U = - 12 % V = 14
UNITS [U] [Y] [V]
Table 18
FOREGROUND - STATUS
Color
INPUT
Figure imgf000029_0001
Table 19 Black Fill is a further input alternative (see Table 20) , which requires no adjustment (see Table 21) . FOREGROUND - INPUT
Input: Black
Figure imgf000030_0001
PORT CONFIG COLOR BLACK TEST
Table 20
FOREGROUND - STATUS
Black
Figure imgf000030_0002
INPUT
Figure imgf000030_0003
Table 21 Finally, a Test Pattern source is provided, for diagnostic purposes (see Tables 22-23) .
FOREGROUND - INPUT
Input: Test
Figure imgf000030_0004
PORT CONFIG COLOR BLACK TEST
Figure imgf000030_0005
Table 22 FOREGROUND - STATUS
Gain = 25 % Offset = 63 IRE
Test Hue = 58 degsj Sat = 13 %
INPUT ADJUST
Table 23
Mask Input Selection
The source of the mask input is selected along very similar lines to foreground and background input. From any menu the Mask Select button is depressed resulting in a display as shown in Table 24.
MASK - STATUS
Port 4 - parallel U/V Normal
INPUT DISABLE CMPT POSITION INVERT
Table 24 Note that the status display includes additional information as to whether the incoming signal is to be taken from the luminance or chrominance component, and whether this component is to be inverted or not. The source of the mask may be selected from any one of the four DI inputs 36 (Figure 1) , an analog input, a shape generator or the Test Pattern generator. This is achieved by depressing the INPUT softkey 230 (see Table 25) . MASK - INPUT
Input: Port 4- parallel
PORT CONFIG ANALOG SHAPE TEST
Table 25
Selection proceeds as outlined for foreground and background input. Returning to the Mask Selection menu, choice may be made between the Luminance (Y) or Chrominance (U/V) component by toggling the CMPT softkey
(see Table 26, Table 27).
MASK - STATUS
Port 4 - parallel Normal
Figure imgf000032_0001
INPUT DISABLE CMPT POSITION INVERT
Figure imgf000032_0002
Table 26
MASK - STATUS
Port 4 - parallel U/V Inverted
INPUT DISABLE CMPT POSITION INVERT
Table 27 The incoming mask signal can be disabled and replaced by a constant source by toggling the DISABLE / ENABLE softkey 230. A display corresponding to Table 28 is obtained for the disable state.
MASK - STATUS
Off
ENABLE
Table 28
Finally, the mask itself may be repositioned by first selecting the POSITION softkey 230, and then adjusting the horizontal and vertical controls via the shaft encoders 224 or numeric keys 232 in the now familiar way (see Table 29) .
MASK - POSITION []
Vertical = 26 Ins Horizontal = 14 pix
Figure imgf000033_0001
[VERT] [HORIZ]
Table 29
Key Input Selection
The source of the key input is selected in the same way as the mask, except for a few refinements. Depress the Key Select button to display a menu very similar to that associated with the Mask Select button (see Table 30) . KEY - STATUS
Port 3 - parallel U/V Normal
INPUT DISABLE CMPT POSITION INVERT
10
Table 30 All softkeys 230 function as for the mask selec case, with the exception of the POSITION softkey 2 As the key signal has the additional feature 15 permitting subpixel positioning accuracy, a fine tun control is also supplied. Adjustment is achieved using the shaft encoders as coarse and fine tun controls (see Table 31) .
KEY - POSITIONING []
Sub-Vertical = 0.5 Sub-Horizontal = 0.75 Vertical = 12 Ins Horizontal = 0 pix
Figure imgf000034_0001
[SUB-V] [VERT] [SUB-H] [HORIZ]
Table 31
30 Clip, Gain and Threshold Adjustment
Since adjustment of the clip, gain and thresh characteristics for the key signal are functiona related, the user interface groups these six cont elements into two user menus which can be interchan
35 by a single softkey 230 depression. Tables 32-33 s these two displays. The central softkey 230 THRSHLD GAIN takes the user from the current display to complementary one. Note that the Upper and Lower C controls are provided on both of the menus for convenience.
RE!
Figure imgf000035_0001
T]
Table 32
CLIP & THRESHOLD []
Upper Clip = 100 % Upper Thrshd = 97 % Lower Clip = 14 % Lower Thrshd = 13 %
[U-CLP] [L-CLP] GAIN [U-THR] [L-THR]
Table 33
Composition Control
A single menu has been dedicated to composition control. Numerous effects can be realized by the video composition system 20, through the control of the mask and key signals. Manipulation of the foreground and backgroundvideo, color fills or blackto generatematte, mix, fades etc. is easily achieved.
The Composition Control button leads one to the control menu as shown in Table 34. Each softkey 230 on this menu toggles between the alternative entries, except for the [FADE] . SYSTEM - CONTROL []
Normal Normal On Off 78%
KEY MASK BEFORE AFTER [FADE]
Table 34 As an example, consider the key itself. Repeated depressions of the KEY softkey 230 cycles one through the three alternatives, namely, Normal, Inverted and Off (see Tables 34-36) . The MASK softkey 230 performs an identical function for the mask signal.
SYSTEM - CONTROL []
Inverted Normal On Off 78%
KEY MASK BEFORE AFTER [FADE]
Figure imgf000036_0001
Table 35
SYSTEM - CONTROL []
Off Normal On Off 78%
KEY MASK BEFORE AFTER [FADE]
Figure imgf000036_0002
Table 36 One of the design elements which enables the video compositing system 20 to achieve such a wide range of compositional effects is it's ability to channel the action of the net key signal into both pre and post "key-process" phases. The enabling / disabling of these two controls are provided through the BEFORE and AFTER softkeys 230, which alternate between the On and Off status.
The percentage of mix between the foreground and background is controlled through the [FADE] softkey 230 via numeric key 232 entry, or the relevant shaft encoder 224. In addition to this "analog" mode of entry, the Cut button 234 alternates the Fade level between 0% and 100% (see Tables 37-38) .
SYSTEM - CONTROL
Normal Normal On Off
KEY MASK BEFORE AFTER [
Figure imgf000037_0001
Table 37
SYSTEM - CONTROL []
Normal Normal On Off
KEY MASK BEFORE AFTER [FADE]
Table 38
Coring and Softening
The key signal may be further modified by coring and softening. These controls are accessed by the Coring and Softening button 234. In essence the coring level selects a low pass filter profile. In addition the horizontal and vertical edges of the key shape may be softened through the two controls provided (see Table 39) .
CORING & SOFTENING []
Coring Level = 0
Vertical = 0 Horizontal = 0
Figure imgf000038_0001
[VERT] [CORE] [HORIZ]
Table 39
Transition Control
Transitions are controlled by specifying a transition profile and a duration. These are accessed by depressing the Auto Transition button 234 to give the display shown in Table 40. As before, both numeric keys 232 and shaft encoders 224 may be employed for data input (see Table 40) .
s
Figure imgf000038_0002
Table 40
Output Timing
The fully composed "composite" video, and the processed key signal both have an output timing control which may be reached by a depression of the Output Timing button 234. Selection is a two stage process. The first menu to be displayed is shown in Table 41. From here one chooses either the KEY or CMPST softkey 230, to give displays indicated in Tables 42 and 43 respectively.
OUTPUT TIMING
KEY CMPST
Table 41
OUTPUT TIMING - KEY []
Vertical = 2 Ins Horizontal = 3 pix
KEY CMPST [VERT] [HORIZ]
Table 42
OUTPUT TIMING - COMPOSITE []
Vertical = 6 Ins Horizontal = 5 pix
KEY CMPST [VERT] [HORIZ]
Table 43 Digital Output-2 Selection
The video compositing system 20 provides five digital output choices from "composite", foreground, background, key and mask signals. Selection is made by first depressing the Digital-2 Output selection button 234 and then the softkey 230 corresponding to signal required (see Table 44)
DIGITAL OUTPUT-2 Selection
Output: Com osite
CMPST FG BG KEY MASK
Figure imgf000040_0001
Table 44
Analog utput Selection An analog output is provided to monitor the outgoing signals. Choices are made via the Analog Output button 234 in the same way as for Digital-2 Output selection.
ANALOG OUTPUT Selection
Output: Composite
Figure imgf000040_0002
CMPST FG BG KEY MASK
Table 45
Attached as an appendix hereto and forming a part of this application is a source code listing in the C programming language of control programs for the microprocessor controller 62 (Figure 1) . These programs control all of the hardware elements shown in the detailed block diagrams of Figures 8, 9, 13 and 15, communications with the control panel of Figure 16 and communications with other systems, such as video editors, connected to the system 20. The software also provides the menus shown above in Tables 1-45.
In practice, the digital image compositing system of this invention is a component digital single-layer high accuracy compositing module that can stand alone or be an integral part of a large system. New techniques for combining layers provide a level of compositing performance and quality that has never been achieved before. The system can be used for all DI compositing applications as well as be the center of a very cost effective DI editing suite. The systemprovides advanced capabilities in layering performance. New digital processing techniques preserve foreground and background edge detail, thereby achieving unsurpassed layering realism. The system is easily integrated into digital and analog component systems where digital component editing is required. It can be the center of new DI editing and compositing suites by replacing more costly DI switchers. For graphics applications, it provides real-time layering at a lower price than alternative means. In telecine, the system replaces a switcher for preview or production compositing while pulling mattes.
Both linear and matte keying is provided. Matte keying simulates the optical matte process which leaves the edges of the foreground video unprocessed, thereby preserving the foreground's edge detail.
The system can be controlled through RS-422 and GPI ports. Serial RS-422 ports allow an external device to emulate the system's control panel and allow the system to be controlled as if it were a Grass Valley Group 100 series switcher. The functions of the GPIs can be assigned by the operator.
The system can be controlled using an internal time line. Events which are assigned to the system's time line are automatically initiated and can be stepped through on a field-by-field basis. The time line can be programmed from the control panel or by an external controller.
Even though the system is a single-layer module, it can be combinedwith additional systems to do parallel multi-layering. The delay through a single system is exactly one frame. When cascaded, the delay through each system is adjusted to maintain one frame delay through the completed cascaded array. A complete kit of tools is provided to modify the key signal. H and V position is adjustable in sub-pixel increments to match key to foreground. To improve the transitions between foreground andbackground, adjustable
H and V edge softening is provided. Upper and lower clip points can be set over the complete range of the key signal to adjust the keying effect. Used with the clip points, wide-range gain and offset adjustments also provide a powerful way to adjust the look of the keying process. To smooth edges from analog keys, noise reduction is available. Inversion of the key and mask is also provided.
Full adjustment of foreground and background parameters is provided, including luminance gain and offset and chrominance saturation and hue rotation. Timed events are provided for transitions between sources. These include cuts and dissolves between foreground, background and fill. Also, keys can be faded up or down. The foreground and background fill can be adjusted and programmed in RGB and YUV values. The system operates in both 525 and 625 4:2:2 standards. Selection is automatic or can be manually set by the operator.
Keyboard macros can be programmed and assigned to the time line or GPIs.
An analog luminance input is provided to accept analog key and mask sources. This input is converted to 10 bit 4:0:0 and is connectable to the internal key and mask busses through key and mask frame stores. A full quality 10 bit RGB, YUV or Betacam output is provided for monitoring and recording on analog VTRs.
It should now be readily apparent to those skilled in the art that a novel digital image compositing system and method capable of achieving the stated objects of the invention has been provided. The digital image compositing system and method incorporates flexible input and output crosspoint switching. Digital inputs and outputs are provided for all video signals. The digital image compositing system and method will accept analog key or mask input signals. The digital image compositing system and method will additionally provide any output video signal in analog form. The digital image compositing system and method can be cascaded to provide real time multiple layer compositing. The digital image compositing system and method incorporates a user interface that facilitates use of the flexibility provided by the system and method.
It should further be apparent to those skilled in the art that various changes in form and details of the invention as shown and described may be made. It is intended that such changes be included within the spirit and scope of the claims appended hereto.

Claims

WHAT IS CLAIMED IS:
1. A digital image compositing system, which comprises a plurality of digital video signal inputs, a key processing subsystem, a video image compositor, a first plurality of crosspoint switches for connecting said plurality of digital video signal inputs to said key processing subsystem and to said video image compositor, a key input connected through saidplurality of crosspoint switches to saidkeyprocessing subsystem, a plurality of digital video signal outputs, and a second plurality of crosspoint switches for connecting saidkey processing subsystem and said video image compositor to said plurality of digital video signal outputs.
2. The digital video image compositing system of Claim 1 in which said key input is an analog input, said key input being connected to said first plurality of crosspoint switches through an analog to digital converter.
3. The digital video image compositing system of Claim 1 additionally comprising an analog video signal output connected to said second plurality of crosspoint switches by a digital to analog converter.
4. The digital video image compositing system of Claim 1 in which said first plurality of crosspoint switches are connected to said key processing subsystem through at least one frame store.
5. The digital video image compositing system of Claim 4 in which said key input is connected by said first plurality of crosspoint switches to said key processing subsystem through a first frame store.
6. The digital video image compositing system of Claim 5 additionally comprising a second frame store comprising amask input connected by said first plurality of crosspoint switches to said key processing subsystem.
7. The digital video image compositing system of Claim 1 additionally comprising a controlmicroprocessor connected to said key processing subsystem, said video image compositor, said first plurality of crosspoint switches, and said second plurality of crosspoint switches.
8. The digital video image compositing system of Claim 7 in which said key processing subsystem includes a clipping circuit controlled by said microprocessor to select from a plurality of different clipping characteristics.
9. The digital video image compositing system of Claim 7 in which said key processing subsystem includes an edge softening filter/interpolating circuit controlled by said microprocessor for edge blending of a key signal supplied to said key input.
10. The digital video image compositing system of Claim 7 in which said video image compositor includes a keying multiplier controlled by said microprocessor and said key processing subsystem is connected to supply a processed key signal to said keying multiplier.
11. The digital video image compositing system of Claim 7 additionally comprising a keyboard and display connected to said key processing subsystem and to said video image compositor through said microprocessor to provide real time response to inputs through said keyboard.
12. The digital video image compositing system of Claim 11 in which said keyboard includes a plurality of function keys, a plurality of numeric keys and a plurality of shaft encoders.
13. The digital video image compositing system of Claim 12 in which said keyboard additionally includes a plurality of soft keys having software definable functions, said plurality of soft keys being positioned on said keyboard below said display, so that functions of said soft keys defined by software are displayed adjacent each of said plurality of soft keys.
14. The digital video image compositing system of Claim 13 in which said keyboard additionally includes a trackball.
15. The digital video image compositing system of Claim 10 additionally comprising a switch for selectively bypassing said keying multiplier.
16. In combination, the digital video image compositing system of Claim 1, a first digital video signal source connected to provide a foreground signal to said digital video image compositing system, a second digital video signal source connected to provide a background signal to said digital video image compositing system, a third video signal source connected to provide a key input to said digital video image compositing system, a video recording device connected to receive a digital program video signal from said digital video image compositing system, and a monitor connected to receive a display signal from said digital video image compositing system.
17. The digital video image compositing system of Claim 1 in which said key processing subsystem includes a lookup table containing a plurality of user selectable look up table input/output characteristics, said key processing subsystem being configured to allow selection of different look up table input/output characteristics for a television frame to provide different outputs for different portions of the television frame.
18. The digital video image compositing system of Claim l in which said video image compositor is configured to provide user selectable randomized rounding of fractional values of digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit.
19. The digital video image compositing system of Claim 1 in which said key processing subsystem is configured to provide user selectable randomized rounding of fractional values of the digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit.
20. The digital video image compositing system of Claim l in which said key processing subsystem and said video image compositor are configured to provide user selectable randomized rounding of fractional values of digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit.
21. The digital video image compositing system of Claim 1 in which said key processing subsystem is connected to said plurality of digital video signal outputs by at least one variable delay memory.
22. The digital video image compositing system of Claim 1 inwhich saidvideo image compositor is connected to said plurality of digital video signal outputs by at least one variable delay memory.
23. In combination, cascaded first and second digital video image compositing systems of Claim 22, at least one of said plurality of digital video signal outputs of said first digital video image compositing system being connected to at least one of said plurality of digital video signal inputs to said second digital video image compositing system, each of said first and second digital video image compositing systems having a buffer memory coupled to each of said plurality of digital video signal inputs, said second digital video image compositing system having said at least one variable delay memory having its variable delay set to provide at least one digital video signal output of said second digital video image compositing system one frame delay after at least one digital video signal input to said first digital video compositing system.
24. The digital video image compositing system of
Claim 23 in which said plurality of digital video signal inputs include a background video signal input, a foreground video signal input, a key signal input and amasksignal input, saidbackground and foregroundvideo signal inputs each being coupled to a first in, first out memory and said key and mask signal inputs each being coupled to a frame store.
25. The digital video image compositing system of Claim 1 additionally comprising a mask input connected by said first plurality of crosspoint switches to said key processing subsystem, a fade control input in said key processing system connected to modify a mask input signal from said mask input to produce a modified mask input signal, the modified mask input signal being connected to modify a key input signal from said key input.
26. The digital video image compositing system of Claim 25 in which the modified mask input signal is connected for selectively modifying the key input signal prior to and after processing of the key input signal by said key processing subsystem.
27. The digital video image compositing system of Claim 26 in which said digital video image compositing system includes a selectively operable inversion circuit connected between said key input and said key processing subsystem.
28. The digital video image compositing system of Claim 26 in which said digital video image compositing system includes a selectively operable inversion circuit connected between said key processing subsystem and said video image compositor.
29. The digital video image compositing system of Claim 26 in which said digital video image compositing system includes a key output and a selectively operable inversion circuit connected between said key processi subsystem and said key output.
30. The digital video image compositing system Claim 26 in which said digital video image compositi system includes a selectivelyoperable inversioncircu connectedbetweensaidmask input andsaidkeyprocessi subsystem.
31. The digital video image compositing system Claim 1 in which said system is configured to recei digital video signals at said plurality of digital vid signal inputs selectively in a plurality of vid standards and to provide digital video signals at sa plurality of digital video signal outputs selective in a plurality of video standards.
32. The digital video image compositing system o Claim 31 in which digital video signals supplied to sa plurality of digital video signal inputs include background signal and a foreground signal, said syst including independent means for supplying a fill signa to the background signal and to the foreground signal
33. A digital image compositing system, whic comprises a key processing subsystem including a clippin circuit, and means controlling said clipping circuit t select from a plurality of different clippin characteristics.
34. The digital image'compositing system of Clai 33 additionally comprising a means connected to sai means controlling said clipping circuit for providin a user selection from the different clippin characteristics.
35. A digital image compositing system, which comprises a key processing subsystem including an edge softening filter/interpolating circuit, and means controlling said edge softening filter/interpolating circuit for edge blending of a key signal in said key processing subsystem.
36. The digital image compositing system of Claim 35 in which said edge softening filter/interpolating circuit comprises a two-dimensional finite impulse response filter.
37. The digital image compositing system of Claim 35 additionally comprising a means connected to said means controlling said edge softening filter/interpolating circuit for providing a user selection of the edge blending.
38. A digital image compositing system, which comprises a key processing subsystem including a look up table containing a plurality of look up table input/output characteristics, and means controlling said look up table to allow selection of different look up table input/output characteristics for a television frame to provide different outputs for different portions of the television frame.
39. The digital image compositing system of Claim 38 additionally comprising a means connected to said means controlling said look up table for providing a user selection of the look up table input/output characteristics.
40. A digital image compositing system, which comprises a video image compositor and a key processing subsystem connected to said video image compositor, said video image compositor and said key processing subsystem being configured to provide randomized rounding of fractional values of digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit.
41. The digital image compositing system of Claim 40 additionally comprising a means for providing a user selection of the different ranges of fractional values.
42. A digital image compositing system comprising a key processing subsystem, a key input to said key processing subsystem, a mask input to saidkeyprocessing subsystem, a fade control input to said key processing subsystem connected to modify a mask input signal from said mask input, the modified mask input signal being connected to modify a key input signal from said key input selectively prior to and after processing of the key input signal by said key processing subsystem.
43. The digital image compositing system of Claim 42 additionally comprising a means for providing a user selection of modification of the key input signal by the modified mask input signal prior to or after processing of thekey input signal by saidkeyprocessing subsystem.
44. A digital image compositing system, which comprises a video image compositor and a key processing subsystem connected to said video image compositor, a key input to said key processing subsystem, and a selectively operable inversion circuit connected between said key input and said key processing subsystem.
45. The digital video image compositing system of Claim 44 in which said digital video image compositing system includes a selectively operable inversion circuit connected between said key processing subsystem and said video image compositor.
46. The digital video image compositing system of Claim 44 in which said digital video image compositing system includes a key output and a selectively operable inversion circuit connected between said key processing subsystem and said key output.
47. The digital video image compositing system of Claim 44 in which said digital video image compositing system includes a mask input and a selectively operable inversion circuit connected between said mask input and said key processing subsystem.
48. The digital image compositing system of Claim 44 additionally comprising a means for providing a user selection of inversion by said inversion circuit.
49. A digital image compositing system, which comprises a video image compositor, a key processing subsystem connected to said video image compositor, said video image compositor providing a foreground signal and a background signal, and independent means for generating a fill signal for each of the foreground and background signals.
50. A digital image compositing method, which comprises selectively providing a plurality of digital video input signals including a key signal to a key processing means and a video image compositor, processing the selected plurality of digital video input signals in the key processing means including selecting from a plurality of di ferent clipping characteristics or the key signal, processing the selected plurality of the digital video input signals in the video image compositor, and selectively providing a plurality of processed digital video output signals from the key processing means and the video image compositor.
51. A digital image compositing method, which comprises selectively providing a plurality of digital video input signals including a key signal to a key processing means and a video image compositor, processing the selected plurality of digital video input signals in the key processing means including sub-pixel positioning by interpolation for edge blending the key signal, processing the selected plurality of the digital video input signals in the video image compositor, and selectively providing a plurality of processed digital video output signals from the key processing means and the video image compositor.
52. A digital image compositing method, which comprises selectively providing a plurality of digital video input signals including a key signal to a key processing means and a video image compositor, processing the selected plurality of digital video input signals in the key processing means including selecting different look-up table input/output characteristics for a television frame to provide different outputs for different portions of the television frame, processing the selectedplurality of thedigital video input signals in the video image compositor, and selectivelyproviding a plurality of processed digital video output signals from the key processing means and the video image compositor.
53. A digital image compositing method, which comprises selectively providing a plurality of digital video input signals including a key signal to a key processing means and a video image compositor, processing the selected plurality of digital video input signals in the key processing means and in the video image compositor including selecting randomized rounding of fractional values of digital video signals to integers over different ranges of fractional values, with fractional values outside the different ranges being rounded upward or downward to the closest digit, and selectively providing a plurality of processed digital video output signals from the key processing means and the video image compositor.
54. A digital image compositing method, which comprises providing a key signal, a mask signal and a fade control signal to a keyprocessing means, processing the key signal in the key processing means, modifying the mask signal using the fade control signal to provide a modified mask signal, and selectively using the modified mask signal to modify the key signal prior to and after processing of the key input signal.
55. A digital image compositing method, which comprises selectively providing a plurality of digital video input signals including a key signal to a key processing means and a video image compositor, selectively inverting the key signal prior to supplying the key signal to the key processing means to give a selectively inverted key signal, processing the selectively inverted key signal in the key processing means to give a modified key signal, processing the selected plurality of digital video input signals in the video image compositor, and selectively providing a plurality of processed digital video output signals from the key processing means and the video image compositor.
56. The method of Claim 55 additionally comprising the steps of selectively inverting the modified key signal to give a selectively inverted modified key signal.
57. The method of Claim 56 additionally comprising the step of supplying the selectively inverted modified key signal to the video image compositor.
58. The method of Claim 56 additionally comprising the step of supplying the selectively inverted modified key signal as an output.
59. A digital image compositing method, which comprises providing a plurality of digital video input signals including a background signal and a foreground signal to a video image compositor, providing independent fill signals forthebackground signal and the foreground signal, and providing an output video image including a filled background and a filled foreground.
PCT/US1992/002849 1991-04-12 1992-04-07 Digital image compositing system and method WO1992018937A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US685,291 1991-04-12
US07/685,291 US5347622A (en) 1991-04-12 1991-04-12 Digital image compositing system and method

Publications (1)

Publication Number Publication Date
WO1992018937A1 true WO1992018937A1 (en) 1992-10-29

Family

ID=24751549

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1992/002849 WO1992018937A1 (en) 1991-04-12 1992-04-07 Digital image compositing system and method

Country Status (4)

Country Link
US (1) US5347622A (en)
EP (1) EP0579755A4 (en)
CA (1) CA2108258A1 (en)
WO (1) WO1992018937A1 (en)

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06242751A (en) * 1992-04-29 1994-09-02 Canon Inf Syst Res Australia Pty Ltd Information display system and information display network
JP2548497B2 (en) * 1992-10-09 1996-10-30 松下電器産業株式会社 Video editing equipment
US20020091850A1 (en) 1992-10-23 2002-07-11 Cybex Corporation System and method for remote monitoring and operation of personal computers
US5808682A (en) * 1993-06-14 1998-09-15 Sega Enterprises, Ltd. Picture data processing system for processing picture data representing foreground and background
US5535137A (en) * 1994-02-14 1996-07-09 Sony Corporation Of Japan Random access audio/video processor with compressed video resampling to allow higher bandwidth throughput
JPH07236151A (en) * 1994-02-23 1995-09-05 Sony Corp Digital video switcher
JPH08130750A (en) * 1994-10-31 1996-05-21 Sony Corp Fader device
EP1098527A1 (en) * 1994-11-04 2001-05-09 Matsushita Electric Industrial Co., Ltd. Picture coding apparatus and decoding apparatus
US5668639A (en) * 1995-03-21 1997-09-16 Comunicacion Integral Method for video editing
US5721842A (en) 1995-08-25 1998-02-24 Apex Pc Solutions, Inc. Interconnection system for viewing and controlling remotely connected computers with on-screen video overlay for controlling of the interconnection switch
NL1002614C2 (en) * 1996-03-14 1997-09-17 Nederland Ptt Mixing system for multiple video signals
US5963214A (en) * 1996-07-29 1999-10-05 Eastman Kodak Company Method of combining two digital images
GB9619119D0 (en) 1996-09-12 1996-10-23 Discreet Logic Inc Processing image
US6604242B1 (en) * 1998-05-18 2003-08-05 Liberate Technologies Combining television broadcast and personalized/interactive information
US6137919A (en) * 1997-04-04 2000-10-24 Avid Technology, Inc. Apparatus and methods for feathering a composite image
US6269195B1 (en) * 1997-04-04 2001-07-31 Avid Technology, Inc. Apparatus and methods for selectively feathering a composite image
US6400371B1 (en) * 1997-05-16 2002-06-04 Liberate Technologies Television signal chrominance adjustment
DE19748139A1 (en) * 1997-10-31 1999-05-06 Philips Patentverwaltung Arrangement for mixing and / or processing video signals
US6208351B1 (en) * 1997-12-22 2001-03-27 Adobe Systems Incorporated Conversion of alpha-multiplied color data
US6509935B1 (en) 1998-03-03 2003-01-21 Microsoft Corporation Method for improved chroma-key suppression
GB2336054B (en) 1998-04-01 2002-10-16 Discreet Logic Inc Processing image data
GB2336055B (en) 1998-04-01 2002-10-16 Discreet Logic Inc Image processing
JP4146089B2 (en) 1998-09-22 2008-09-03 アボセント ハンツヴィル コーポレーション System for remote access to personal computers
US9123380B2 (en) 1998-12-18 2015-09-01 Gvbb Holdings S.A.R.L. Systems, methods, and computer program products for automated real-time execution of live inserts of repurposed stored content distribution, and multiple aspect ratio automated simulcast production
US20030214605A1 (en) * 1998-12-18 2003-11-20 Snyder Robert J. Autokeying method, system, and computer program product
US6456300B1 (en) 1999-03-31 2002-09-24 Autodesk Canada Inc. Method and apparatus for processing image data to produce control data
AU3712300A (en) 1999-06-11 2001-01-02 Liberate Technologies Hierarchical open security information delegation and acquisition
US6825852B1 (en) * 2000-05-16 2004-11-30 Adobe Systems Incorporated Combining images including transparency by selecting color components
US6899273B2 (en) * 2001-05-02 2005-05-31 Hand Held Products, Inc. Optical reader comprising soft key including permanent graphic indicia
US20030035559A1 (en) * 2001-08-16 2003-02-20 Laurent Cohen Trackball controller for built-in effects
AU2003215829A1 (en) * 2002-03-29 2003-10-13 Koninklijke Philips Electronics N.V. Television receiving system
US20040189676A1 (en) * 2003-03-24 2004-09-30 Dischert Lee R. Method, apparatus , and system for selectively combining video signals
US8127242B1 (en) * 2010-08-12 2012-02-28 The United States Of America As Represented By The Secretary Of The Army Graphic user interface and software for processing large size signal data samples in a small buffer using automatically adjusted decimation ratio
US7956929B2 (en) * 2005-10-31 2011-06-07 Broadcom Corporation Video background subtractor system
US7673804B1 (en) 2005-11-14 2010-03-09 Hand Held Products, Inc. Handle for portable data terminal
US7748632B2 (en) * 2006-07-25 2010-07-06 Hand Held Products, Inc. Portable data terminal and battery therefor
EP2389768A4 (en) * 2009-01-21 2013-01-23 Gennum Corp Video specific built-in self test and system test for crosspoint switches
JP5646870B2 (en) * 2010-04-14 2014-12-24 一夫 市川 Color vision inspection apparatus, control program, and method for controlling color vision inspection apparatus
US20220035684A1 (en) * 2020-08-03 2022-02-03 Nvidia Corporation Dynamic load balancing of operations for real-time deep learning analytics

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4684990A (en) * 1985-04-12 1987-08-04 Ampex Corporation Method and apparatus for combining multiple video images in three dimensions
US4827253A (en) * 1987-05-18 1989-05-02 Dubner Computer Systems, Inc. Video compositing using a software linear keyer
US4991014A (en) * 1987-02-20 1991-02-05 Nec Corporation Key signal producing apparatus for video picture composition
US5046165A (en) * 1989-04-07 1991-09-03 Sony Corporation Controlling the combining of video signals

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4357624A (en) * 1979-05-15 1982-11-02 Combined Logic Company Interactive video production system
US4420770A (en) * 1982-04-05 1983-12-13 Thomson-Csf Broadcast, Inc. Video background generation system
GB8405947D0 (en) * 1984-03-07 1984-04-11 Quantel Ltd Video signal processing systems
US4827344A (en) * 1985-02-28 1989-05-02 Intel Corporation Apparatus for inserting part of one video image into another video image
US4954883A (en) * 1986-04-14 1990-09-04 Corporate Communications Consultants, Inc. Interweave picture comparison apparatus and method
US4872056A (en) * 1987-02-05 1989-10-03 Video Graphic Styling, Inc. Method for displaying selected hairstyles in video form
DE3842977C2 (en) * 1988-12-21 1996-03-28 Broadcast Television Syst Mixing device for video signals
GB2243047A (en) * 1990-04-11 1991-10-16 Rank Cintel Ltd A digital composite video encoder

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4684990A (en) * 1985-04-12 1987-08-04 Ampex Corporation Method and apparatus for combining multiple video images in three dimensions
US4991014A (en) * 1987-02-20 1991-02-05 Nec Corporation Key signal producing apparatus for video picture composition
US4827253A (en) * 1987-05-18 1989-05-02 Dubner Computer Systems, Inc. Video compositing using a software linear keyer
US5046165A (en) * 1989-04-07 1991-09-03 Sony Corporation Controlling the combining of video signals

Also Published As

Publication number Publication date
US5347622A (en) 1994-09-13
EP0579755A1 (en) 1994-01-26
EP0579755A4 (en) 1994-05-11
CA2108258A1 (en) 1992-10-13

Similar Documents

Publication Publication Date Title
US5347622A (en) Digital image compositing system and method
US4890162A (en) Adjustable antialias filters
US5594467A (en) Computer based display system allowing mixing and windowing of graphics and video
US5027212A (en) Computer based video/graphics display system
EP0463719B1 (en) Multiple video preview
US4660081A (en) Video signal processing apparatus having interpolators switchable between an input and an output of a field memory
US4853784A (en) Video switcher with independent processing of selected video signals
JPH0614705B2 (en) Video image forming method and circuit
JPS63198485A (en) Key signal generator
EP0327333B1 (en) Apparatus for generating a video signal representing a field of spatially varying color
JP2000138873A (en) Video processor
EP0395372B1 (en) Layered mix effects switcher architecture
US5684543A (en) Input and output signal converter with small-sized connection crosspoint
EP0215107A1 (en) Apparatus and method for producing a variety of video wipe borders.
US5282038A (en) Video signal processing system architecture
US5153728A (en) Image enlargement apparatus using plural diverse capacity memories
US4636838A (en) Generating a chroma keying signal
US5285283A (en) Depth signal processing in a video switcher
US4413273A (en) System for mixing two color television signals
EP0162507B1 (en) Video signal processing arrangement
JP2775126B2 (en) Video synthesis device
JPS60232787A (en) Picture synthesizing device
JP2007325112A (en) Image specific effect device, and control method thereof
Nasse et al. The All-Digital Studio is Here
Sheldon Component Digital Switching

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): CA

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FR GB GR IT LU MC NL SE

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2108258

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 1992910805

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 1992910805

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 1992910805

Country of ref document: EP