VERIFYING REACQUIRED VIDEO Background of the l ention The invention relates to verifying that reacquired video is the same as video initially acquired. In video editing systems source video is digitized (unless already digitized) and stored for random access in generating, editing and playing a video program made up of a plurality of sequences of frames for different scenes. The video sequences, also referred to as "clips," can be pieced together from different sources. The editing can involve adding effects to the video. In some instances it is desired to reenter the source video, which might. for example be implemented by an instruction sent to a tape operator to load a particular tape and play particular segments from the tape. Mistakes can arise in reacquiring source video, where, e.g.. an operator can load the wrong tape or access the wrong portion of a tape.
Summary of the Invention In one aspect, the invention features, in general, a method and apparatus for verifying that reacquired video is the same as originalh acquired video. A characteristic of the originally acquired video data is determined and stored during acquiring. When the same sequence of video frames is reacquired, the same characteristic is determined at the same time for the reacquired video data. The characteristic for the reacquired video data is then compared with the characteristic for the originally acquired video data.
In preferred embodiments, the acquired and reacquired video data can include luminance values, and the characteristic can be based on the luminance values or, e.g., pixel- to-pixel differences in luminance values. Alternatively the acquired and reacquired video data can include RGB intensity values, and the characteristic can be based on one or more of the RGB intensity values, or on one or more luminance values derived from the RGB intensity values. The comparison of characteristics of the acquired and reacquired video data can involve calculating differences of pixel-to-pixel differences of luminance values for the acquired and reacquired video data. The characteristic used for comparison can be based on portions of one or more video frames. The video frames can have time or key codes that can be monitored during reacquiring, and the comparison of characteristics can begin when a particular time or key code is detected.
In another aspect, the invention features, in general, a method or apparatus for locating a particular video frame in a series of video frames. First a coarse comparison of a characteristic of the video frames is carried out to determine that a frame being subjected to the coarse comparison is likely to be in the same clip as the particular video frame to be located. Then a fine comparison of the same or a different characteristic is carried out on the video frames to identify the particular frame.
In preferred embodiments, the characteristic used in the coarse comparison can. e.g.. be auxiliary data, an average luminance value for a plurality of pixels in a frame, a characteristic of JPEG encoded data, or an average luminance value for a plurality of pixels in a frame determined from MPEG encoded data. The characteristic used in the fine comparison can be luminance or one or more RGB intensity values. W avelet filtering can also be used to generate the characteristics for coarse and/or fine comparison.
Embodiments of the invention may include one or more of the following advantages. One can verify that video data being reacquired is the correct data at the time of reacquiring, and possibly even before one begins storing the data, as the data is being received. The verified data can thus be received in a single pass and without the need to store and then delete incorrect data.
Other features and advantages of the invention will be apparent from the following description of a preferred embodiment and from the claims.
Brief Description of the Drawing
Fig. 1 is a block diagram of a video editing system.
Fig. 2 is a block diagram of video editing circuitry of the Fig. 1 system.
Fig. 3 is a diagram of a stream of video frames processed by the Fig. 1 system.
Fig. 4 is a diagram of a frame of the Fig. 3 stream of frames. Fig. 5 is an illustration of a sum of the differences of the differences comparison carried out by the Fig. 1 system.
Description of the Preferred Embodiments
Referring to Fig. 1. video editing system 40 is implemented by computer 42. video editing software 44 running on computer 42. and video editing expansion card 46 plugged into computer 42. VTR 48 is a source of video segments that can be stored on disk or other mass storage 50 and randomly accessed by computer 42. The video can also be provided
other sources 60, e.g., network storage or backup storage. Keyboard 52 and mouse 54 are user input devices, and monitor 56 is used to provide a video editing interface including display of a program being created. An additional monitor (not shown) can also be used to play the video program. U.S. Patents Nos. 5.909,250: 5.506,932; 5,488.695; 5,471 ,577. which are hereby incorporated by reference, describe video editing systems implemented on a computer.
Referring to Fig. 2, video editing card 46 includes video input/output (I/O) 62. analog to digital converter (A/D) 64, processor 66. and buffer 68. Video I/O receives analog
video as an input and provides it to A/D 64, which outputs digital video in the Y. Cr. Cb format. These values are temporally stored in buffer 68 and sent on to the host buffer of computer 42 for storage on disk 50. The values may be compressed b\ components that are not shown on Fig. 2 before being transmitted to the host bus. As is shown in Fig. 3. the data for the video is contained in frames 70 that are sent one after the other. The system inputs video clips or "segments" 72 which each include a sequence of video frames 70. As is shown in Fig. 4, each video frame 70 includes lines 74 of pixel values.
Returning to Fig. 2. processor 66 receives the luminance (Y ) values, and computes an identification characteristic of the video sequence being input based upon the luminance values. Differences in luminance values can be used as the identification characteristic. One technique involves determining the pixel-to-pixel differences in luminance values, and then comparing the pixel-to-pixel differences of the reacquired video with those of the original video. One comparison could involve taking the differences of the differences, summing them up for a series of pixels, and seeing if the summation is less than a threshold value. The determination of characteristics and comparisons of characteristics could take place at processor 66.
Fig. 5 shows an example where four pixels of the same line 74 of a frame are compared for the original and reacquired video. Thus the original video has the luminance values 16, 16, 216, 216 for these pixels, and the reacquired video has 1 7. 1 8. 218, 219 for these pixels. The pixel-to-pixel differences for the original video are 16. 0, 200. 0, and the pixel-to-pixel differences for the reacquired video are 17, 1 , 200, 1 . The differences of the absolute values of the differences are 1 , 1 , 0, 1 , and the absolute sum of the differences is 3. The threshold could be done on a percentage basis. Here, the maximum luminance value is 219, and the sum value. 3. was obtained over 4 pixels; thus the percentage difference is (3/4)/219. or 0.3%. If the threshold were 1 %, the resulting 0.3% value w ould meet the threshold, and indicate that the reacquired video is from the same source.
If the video involves RGB video (instead of YUV). an RGB intensity value can be used for the comparison, or luminance values could be derived on the fly from the RGB intensity values.
One can determine the frames of the original video and the reacquired video that should be used in the comparison by using time codes or key codes (used for film to video transfer) to identify the same frame.
Alternatively, one could do a rough cut comparison to see if the reacquired video is getting close to the actual frame of the original, and then do the sum of the differences of the differences comparison as described with reference to Fig. 5 to see if there is a match. The rough cut approximation might involve the Fig. 5 type of comparison with a larger
threshold value, e.g.. 10%. Alternatively one could use another, simpler comparison for the coarse comparison.
Further techniques for rough cut comparisons to determine frames for a fine comparison include comparing compression coefficients (e.g., JPEG coefficients, converted to the frequency domain), comparing a displayable character, comparing auxiliary
(nonimage) data, and comparing average luminance values (either calculated or from MPEG data) for an entire frame. In the rough cut comparisons, wide tolerances could be used.
Wavelet filtering can also be used to generate the characteristics for coarse and/or fine comparison. The video data will be buffered as the reacquired video is being received and evaluated. The rough cut comparison can thus be used to decide to discard video prior to a match. Once a match has been met (by rough cut comparison), the data could continue to be buffered in buffer 68 while a fine cut comparison is then carried out on the pixels of interest.
Assuming that there is a match, the data will be stored in mass storage 50. In this way. the rough cut and fine comparisons can be carried out on the fly and prior to storage at the computer. Thus there is no need to store data until it has been verified, and there is no need to reinput the data after verification.
The luminance or other values being used for comparison could be filtered, with the comparisons being carried out on the filtered data, to remove noise (high frequencies) and the effects of overall gain changes (low frequencies).
Other embodiments of the invention are within the scope of the appended claims.
What is claimed is: