WO1998052356A1 - Methods and architecture for indexing and editing compressed video over the world wide web - Google Patents
Methods and architecture for indexing and editing compressed video over the world wide web Download PDFInfo
- Publication number
- WO1998052356A1 WO1998052356A1 PCT/US1997/008266 US9708266W WO9852356A1 WO 1998052356 A1 WO1998052356 A1 WO 1998052356A1 US 9708266 W US9708266 W US 9708266W WO 9852356 A1 WO9852356 A1 WO 9852356A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- frame
- video
- video information
- compressed
- coded
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 77
- 230000000694 effects Effects 0.000 claims abstract description 23
- 230000000873 masking effect Effects 0.000 claims abstract description 15
- 239000013598 vector Substances 0.000 claims description 79
- 230000000007 visual effect Effects 0.000 claims description 23
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 9
- 230000000877 morphologic effect Effects 0.000 claims description 5
- 238000005303 weighing Methods 0.000 claims description 4
- 238000013519 translation Methods 0.000 claims description 2
- 241000269627 Amphiuma means Species 0.000 claims 3
- 230000009466 transformation Effects 0.000 claims 2
- 238000013459 approach Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 238000001514 detection method Methods 0.000 description 8
- 230000008859 change Effects 0.000 description 6
- 230000006835 compression Effects 0.000 description 6
- 238000007906 compression Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 239000011159 matrix material Substances 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 238000012800 visualization Methods 0.000 description 4
- 238000010420 art technique Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000005562 fading Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000011946 reduction process Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7837—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
- G06F16/786—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content using motion, e.g. object motion or camera motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
- G06F16/7864—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content using domain-transform features, e.g. DCT or wavelet transform coefficients
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
- G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/34—Indicating arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/48—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/147—Scene change detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
- H04N5/92—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
- H04N5/926—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback by pulse code modulation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/79—Processing of colour television signals in connection with recording
- H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
- H04N9/804—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
- H04N9/806—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components with processing of the sound signal
Definitions
- video information is generally transmitted between systems in the digital environment the form of compressed bitstreams that are in a standard format, e.g., Motion JPEG, MPEG- 1 , MPEG-2, H.261 or H.263.
- DCT Discrete Cosine Transform
- Run-length encoding and entropy coding i.e., Huffman coding or arithmetic coding
- Huffman coding or arithmetic coding are applied to the quantized bitstream to produce a compressed bitstream which has a significantly reduced bit rate than the original uncompressed source signal.
- the process is assisted by additional side information, in the form of motion vectors, which are used to construct frame or field-based predictions from neighboring frames or fields by taking into account the inter-frame or inter-field motion that is typically present.
- bitstreams In order to be usable by a receiving system, such coded bitstreams must be both parsed and decoded.
- bitstream In the case of an MPEG-2 encoded bitstream, the bitstream must be parsed into slices and macroblocks before the information contained in the bitstream is usable by an MPEG-2 decoder. Parsed bitstream information may be used directly by an MPEG-2 decoder to reconstruct the original visual information, or may be subjected to further processing.
- further processing of video information can occur either in the normal, uncompressed domain or in the compressed domain. Indeed, there have been numerous attempts by others in the field to realize useful techniques for indexing and manipulating digital video information in both the uncompressed and compressed domains.
- Rivl uses group of pictures (GOPs) level direct copying whenever possible for "cut and paste” operations on MPEG video, it does not use operations in the compressed domain at frame and macroblock levels for special effects editing. Instead, most video effects in Rivl are done by decoding each frame into the pixel domain and then applying image library routines.
- GOPs group of pictures
- key content refers to key frames in video sequences, prominent video objects and their associated visual features (motion, shape, color, and trajectory), or special reconstructed video models for representing video content in a video scene.
- the technique must allow for video editing directly in the compressed domain to allow users to manipulate an specific object of interest in the video stream without having to fully decode the video. For example, the technique should permit a user to cut and paste any arbitrary segment from an existing video stream to produce a new video stream which conforms to the valid compression format.
- Another object of the present invention is to provide techniques for key content browsing and searching of compressed digital video without decoding and viewing the entire video stream.
- a further object of the present invention is to provide techniques which allow for video editing directly in the compressed domain
- a still further object of the present invention is to provide tools that permit users who want to manipulate compressed digital video information to extract a rich set of visual features associated with visual scenes and individual objects directly from compressed video.
- Yet another object of this invention is to provide an architecture which permits users to manipulate compressed video information over a distributed network, such as the Internet.
- the present invention provides a method for detecting moving video objects in a compressed digital bitstream which represents a sequence of fields or frames of video information for one or more previously captured scenes of video.
- the described method advantageously provides for analyzing a compressed bistream to locate scene cuts so that at least one sequence of fields or frames of video information which represents a single video scene is determined.
- the method also provides for estimating one or more operating parameters of a camera which initially captured the video scene,by analyzing a portion of the compressed bitstream which corresponds to the video scene, and for detecting one or more moving video objects represented in the compressed bitstream by applying global motion compensation with the estimated camera operating parameters.
- the compressed bitstream is a bitstream compressed in accordance with the MPEG-1, MPEG-2, H261, or H263 video standard.
- analyzing can beneficially be accomplished by parsing the compressed bitstream into blocks of video information and associated motion vector information for each field or frame of video information which comprises the determined sequence of fields or frames of video information representative of said single scene, performing inverse motion compensation on each of the parsed blocks of video information to derive discrete cosign transform coefficients for each of the parsed blocks of video information, counting the motion vector information associated with each of the parsed blocks of video information, and determining from the counted motion vector information and discrete cosign transform coefficient information whether one of the scene cuts has occurred.
- analyzing is performed by parsing the compressed bitstream into blocks of video information and associated motion vector information for each field or frame of video information which comprises the determined sequence of fields or frames of video information representative of the scene, and estimating is executed by approximating any zoom and any pan of the camera by determining a multi-parameter transform model applied to the parsed motion vector information.
- the frame difference due to camera pan and zoom motion may be modeled by a six-parameter affine transform which represents the global motion information representative of the zoom and pan of the camera.
- the detecting step advantageously provides for computing local object motion for one or more moving video objects based on the global motion information and on one or more motion vectors which correspond to the one or more moving video objects.
- thresholding and morphological operations are preferably applied to the determined local object motion values to eliminate any erroneously sensed moving objects.
- border points of the detected moving objects are determined to generate a bounding box for the detected moving object.
- the present invention also provides for an apparatus for detecting moving video objects in a compressed digital bitstream which represents a sequence of fields or frames of video information for one or more previously captured scenes of video.
- the apparatus includes means for analyzing the compressed bistream to locate scene cuts therein and to determine at least one sequence of fields or frames of video information which represents a single video scene, means for estimating one or more operating parameters for a camera which initially viewed the video scene by analyzing a portion of the compressed bitstream which corresponds to the video scene, and means for detecting one or more moving video objects represented in the compressed bitstream by applying global motion compensation to the estimated operating parameters.
- a different aspect of the present invention provides techniques for dissolving an incoming scene of video information which comprises a sequence of fields or frame of compressed video information to an outgoing scene of video information which comprises a sequence of fields or frame of compressed video information.
- This technique advantageously provides for applying DCT domain motion compensation to obtain DCT coefficients for all blocks of video information which make up a last frame of the outgoing video scene and the first frame of the incoming video scene, and for creating a frame in the dissolve region frame from the DCT coefficients of the last outgoing frame and the first incoming frame.
- an initial value for a weighting function is selected prior to the creation of a first frame in the dissolve region and is used in the creation of the first frame in the dissolve region.
- the weighting value is then incremented, and a second dissolve frame from the DCT coefficients is generated.
- the technique provides for examining motion vectors associated with block n to determine whether they point to blocks outside or on the mask region, and reencoding the block if a motion vector points to blocks outside the boundary, or on, the mask region.
- a technique for generating a frozen frame of video information from a sequence of frames of compressed video information is provided.
- the technique attractively provides for selecting a frame of compressed video information to be frozen, determining whether the frame to be frozen is intra-coded, predictive-coded or bi-directionally predictive-coded, and if the frame is not intra-coded, converting it to become intra-coded, creating duplicate predictive-coded frames, and arranging the intra-coded frame and the duplicate predictive-coded frames into a sequence of compressed frames of video information.
- a system for editing compressed video information over a distributed network includes a client computer, a network link for permitting said client computer to search for and locate compressed video information on said distributed network, and tools for editing a compressed bitstream of video information over the distributed network.
- Fig. 1 illustrates a system in accordance with one aspect of the present invention
- Fig. 2. is a flowchart which illustrates how a scene cut is detected in accordance with one aspect of the present invention
- Fig. 3. is a flowchart which illustrates how camera parameters are estimated in accordance with one aspect of the present invention
- Fig. 4 is a vector diagram which serves to explain global and local motion
- Fig. 5 depicts an exemplary frame of compressed video information and motion vectors for the frame
- Fig. 6 is a flowchart which illustrates global motion compensation in accordance with one aspect of the present invention.
- Fig. 9a depicts masking
- Fig 9(b) is a flowchart which illustrates masking
- Fig. 10 is a flowchart which illustrates the freezeframe effect
- Fig. 11 depicts two alternative techniques for the slow motion effect
- Fig. 12 is a system diagram of a distributed network in accordance with one aspect of the present invention.
- Fig. 13 depicts exemplarily techniques which may be executed in the distributed network illustrated in Fig. 12.
- FIG. 1 an exemplary embodiment of our invention which permits a user to edit and parse visual information in the compressed domain is provided.
- the architecture of the system 100 is broadly arranged into three functional modules, a parsing module 110, a visualization module 120, and an authoring and editing module 130.
- an incoming bitstream of compressed video information 111 which may be, for example, an MPEG-2 compressed bitstream is examined for scene cuts 112 and broken into shot segments, where each segment includes one or more fields or frames of compressed video information.
- the shot segments will be made of three types of fields or frames, i.e., Intra-coded ("I") fields which are coded independently and entirely without reference to other fields, Predictive-coded (“P") fields which are coded with reference to temporally preceding I or P fields in the sequence, and Bi- directionally predictive-coded (“B”) fields which are coded with reference to the nearest preceding and/or future I or P fields in the sequence.
- I Intra-coded
- P Predictive-coded
- B Bi- directionally predictive-coded
- the bistream will also include associated motion vector information for the P and B fields which "point" to blocks of video information in a preceding or succeeding field or frame which are needed to reconstruct the compressed field or frame of video information.
- the parsing module compiles a list of scene cuts which are useful for indexing the compressed video information.
- the individual shot segments are next analyzed 113 in order to derive camera operation parameters.
- the vector field 114 is representative of the pan of the camera which originally captured the video information which is being analyzed.
- Histogram 115 is used to detect the pan of the camera. Based on the derived operating parameters, moving objects 117 within the compressed video information 116 are detected and shape and trajectory features for such moving objects are extracted.
- the complied list of scene cuts and the derived camera zoom and pan information are used to extract key frames 121 which represent each video shot.
- the key frames 121 are placed in a hierarchical arrangement 125 so that they may be readily browsed with a hierarchical video scene browser, such as the browser described in D. Zhong et al., "Clustering Methods for Video Browsing and Annotation," Storage and Retrieval for Still Image and Video Databases IV, IS&T/SPIE's Electronic Images: Science & Tech. 96, Vol 2670 (1996).
- a content-based image query system 126 may then be used to index and retrieve key frames or video objects based on their visual features and spatial layout.
- An MPEG-1 or MPEG-2 compressed bitstream 201 that is received by the parsing module 110 is first subjected to parsing 210.
- Such a bitstream represents an arrangement of NxN blocks of video information that are broadly organized into macroblocks, where each macroblock is defined by four blocks of luminance information and one block of chrominance information, and further organized into slices which represent contigous sequences of macroblocks of video information in raster scan order.
- the NxN blocks of video information are preferably 8x8 blocks.
- the bitstream also represents associated motion vector information and prediction error difference information which are needed to reconstruct original blocks of video information.
- the bitstream is parsed down to the fundamental block level by parsing techniques known to those skilled in the art 211.
- the parsed blocks of video information are still in compressed format and are thus represented by Direct Cosign Trasform ("DCT") coefficients which have been quantized, Zig- Zag run-length encoded and variable length coded, as those skilled in the art will appreciate.
- DCT Direct Cosign Trasform
- the inverse motion compensation stage 220 the parsed blocks of video information which belong to P and B frames are then subjected to inverse motion compensation 220 by using the associated motion vector information to locate reference blocks of video information and reconstruct the DCT coefficients of blocks of video information in the B and P frames. In this step 220, only the first (the "DC") DCT coefficients are used. Motion vectors associated with the B and P frames are counted 222 for each frame in the sequence.
- three ratios i.e., the number of intra-coded macroblocks to the number of forward motion vectors, the number of backward motion vectors to the number of forward motion vectors, and the number of forward motion vectors to the number of backward motion vectors, are calculated 231 in order to detecting direct scene cuts in P, B, and I frames, respectively.
- the fact that most video shots of compressed MPEG video are formed by consecutive P, I and B frames that have a high degree of temporal correlation is taken advantage of. For P and B frames, this correlation is characterized by the ratio of the number of backward motion vectors, or intracoded macroblocks, to the number of forward motion vectors.
- the detection stage 240 the ratios calculated in 231 are compared to local adaptive thresholds in order to detect the peak values 241. For P frames, the ratio of the number of intra-coded macroblocks to the number of forward motion vectors is examined. For B frames, the ratio of the number of backward motion vectors to the number of forward motion vectors is examined.
- the variance of DCT DC coefficients calculated for I and P frames in 232 is used in two ways.
- this variance information is used together with the ratio of the number of forward motion vectors to the number of backward motion vectors determined in 231 in order to detect candidate scene changes 242. If the ratio is above a predetermined threshold, the frame is marked as containing a suspected scene cut. If the variable information is much different than was the variance information for the immediately preceding I frame, a suspected scene cut is likewise detectable.
- the variance information is also used directly to detect dissolve regions, i.e. regions where one scene is fading out and a second is fading in 243 by examining the parabolic curve of the variance information.
- Camera zoom and pan which give the global motion of the shot being analyzed, can be estimated with a 6-parameter affine transform model by using the actual motion vectors from the MPEG compressed stream.
- the motion vectors in MPEG are usually generated by block matching: finding a block in the reference frame so that the mean square error of prediction is minimized. Although the motion vectors do not represent the true optical flow, it is still good in most cases to estimate the camera parameters in sequences that do not contain large dark or uniform regions.
- a global parameter can be determined using Least Squares ("LS") estimation by finding a set of parameters to a minimize the error between the motion vectors estimated in (1) and the actual motion vectors obtained from the MPEG stream:
- S( ⁇ T) ⁇ [(" x.y xy y' + ( xy x . ] (2)
- step 310 each motion vector associated with the B and P frames contained in the shot are decoded.
- the intermediate variables N, A, B, C, D, and E are interactively calculated using the x and y coordinates for each macroblock in the current frame of video being analyzed 320.
- intermediate transform parameters u l5 u 2 , u 3 , v,, v 2 , v 3 are calculated using the decoded motion vectors and the x and y coordinates for such macroblocks 330.
- the vector a is solved for by solving for the matrix inverse operation 340.
- object motion can be extracted by applying global motion compensation.
- U X • a .
- GMC global motion compensation
- the moving objects themselves can be detected by comparing the magnitude of the local motion to a predetermined threshold value and by performing simple morphological operations to delete small false objects and to fill noisy spots.
- FIG 5(c) there is shown an extracted moving object.
- the DCT coefficients of the moving object are extracted from the compressed video information to provide for later querying of the object. Extraction of the DCT coefficient is done by the DCT domain motion compensation algorithm, as disclosed in U.S. Patent No. 5,408,274 to Chang et al., the disclosure of which is incorporated by reference herein.
- the outermost points of the extracted object are used to form a bounding box, as illustrated in figure 5(d).
- the location and size of each extracted bounding box is saved in a database 126 for later browsing and indexing by a user.
- visual features of extracted objects such as color, textures, and shape, are used to provide content-based visual querying 127 of these and associated video scenes.
- each globally motion compensated motion vector is compared to a predetermined threshold value 620 in order to eliminate non-moving parts of the frame from further consideration.
- the number of contiguous blocks are counted 630.
- the number of associated blocks are compared to a predetermined minimum threshold value 640 in order to eliminate false small objects from being detected.
- the border points for the remaining objects are identified and saved in a database, e.g. database 126, for later use 650, together with corresponding DCT coefficients for all blocks within the border. In this way, the important moving video object can be extracted and indexed for later viewing by a user.
- editing of compressed video is directed to permitting a user to cut a first segment of video 710 from a first video sequence 720 and a second segment of video 730 from a second sequence of video 740 to form a new bitstream of video information 750.
- Such techniques have been described in the art, including in the article by the present inventors, J. Meng et al., "Tools for Compressed-Domain Vide Indexing and Editing," SPIE Conf. on Storage and Retrieval for Image and Video Database, Vol. 2670 (1996), the disclosure of which is incorporated by reference herein.
- the DCT of the output video Y can be obtained by linear matrix operations of the input DCT, P i; as follows:
- H ( and W f are special filter coefficient matrices in the DCT domain.
- the compressed-domain manipulation functions can be implemented in two ways. First, transform-domain techniques can be used to convert B and P frames to intraframe DCT coefficients, on which the above techniques can be readily applied. An alternative approach is to keep the B or P structure (i.e., the DCT coefficients of residual errors and motion vectors) and develop algorithms directly utilizing these data.
- Several advanced visual effects which can be created in the compressed domain - dissolve, masking, freeze frame, variable speed, and strobe motion - are now particularly described. One of the most important tools used in film editing is dissolve.
- dissolve refers to the technique where an outgoing video scene 801 is faded out while an incoming video scene 802 is faded in.
- the actual DCT coefficients for each block of compressed video in the last frame of the outgoing video scene, and the DCT coefficients for each block of compressed video in the first frame of the incoming video scend must be extracted.
- One technique for extracting such DCT coefficients is described in the above-mentioned Chang et al. patent.
- the Chang et al. patent describes a technique which uses DCT domain inverse motion compensation to obtain the DCT coefficients for all blocks of video information which make up the needed frames of video.
- v and v are coordinates within a frame
- t is the frame index value which may range from 1 to N
- n being the total number of frames in the dissolved region
- ⁇ (t) is a weighing function that is variable from 100% to 0%
- F is the composite of the derived DCT coefficients for all blocks which make up the last frame of the outgoing video scene
- F 2 is the composite of the derived DCT coefficients for all blocks which make up the first frame of the incoming video scene.
- the resulting effect is a dissolve transition from a particular frozen frame of the outgoing video scene to another frozen frame of the incoming video scene.
- step 810 DCT domain motion compensation is used to obtain the DCT coefficients for all blocks of video information which make up the last frame of an outgoing video scene F, .
- step 820 DCT domain motion compensation is used to obtain the DCT coefficients for all blocks of video information which make up the first frame of an incoming video scene F 2 .
- step 830 the initial value for the weighing function, ⁇ (t), is chosen.
- equation 9 is applied to create a first frame in the dissolve region. The value oft is then incremented until a final value n is obtained 850.
- a second important tool used in film editing is masking.
- the film effect of masking video refers to transforming an original video scene having e.g. a 4:3 aspect ratio to different aspect ratios such as 1 : 1.66, 1 : 1.85, 1 :2.35, or 16:9.
- Masking can also be used to crop part of the frame region to a different frame size.
- the DCT coefficients for blocks outside of the desired region are set to 0, and the coefficients for blocks that lie on the masking boundaries can recalculated using a simplified DCT cropping algorithm:
- A is an original block located on the boundary
- B is the new masked block
- I h is the identity matrix with size h x h, as shown in figure 9a.
- step 910 the frame to be masked is examined in order to determine whether it is an I, P or B frame. If the frame is an I frame 911, all DCT coefficients for all blocks within the frame are extracted 920. Block n is examined 930 to determine where in the frame it is located. If the block is inside the mask region 931 , the DCT coefficients for the block are unchanged 941. If the block is outside the make region 932, the DCT coefficients for the block are set to zero 942. If the block is on the boundary of the masked region 933, the DCT cropping algorithm, i.e., equation 10, is applied 943. The block index n is incremented 950 and the process repeated 930-950 until all blocks have been examined.
- the DCT cropping algorithm i.e., equation 10
- motion vectors associated with block n are examined 960 to determine whether they point to blocks outside or on the mask region 970. If a motion vector points to blocks outside or on the mask region 972, the macroblock is reencoded 980. The block index n is incremented 990 and the process repeated 960-990 until all blocks have been examined.
- freeze effect A third important tool used in film editing is the freeze effect. Since the freeze effect is usually longer than 1 second, simply repeating duplicate frames is not desirable if interactive playback (e.g. random search) is desired. Instead, the film effect of freeze frame requires the use of periodic I frames.
- the frozen frame in order to ensure interactive playback capabilities, if the frozen frame is a P or B frame, it should be converted to an I frame.
- every block of video information for the frame that has an associated motion vector is converted into a pure array of DCT coefficients, without motion vector information.
- the above-mentioned DCT domain motion compensation algorithm described in the Chang et al. patent is advantageously used to effect such conversion.
- the converted blocks can be referred to as intracoded blocks.
- the group of pictures which represent the frozen frame are filled with duplicated P frames, with all macroblocks in the duplicated P frames being to Motion Compensation Not Coded (i.e., 0 motion vector, and 0 residue error).
- Duplicate P frames can easily be created independently and inserted after I or P frames, as those familiar with digital compression techniques will easily understand.
- step 1010 a user-defined frame of compressed video information is selected for the freeze- frame effect.
- step 1020 the frame is examined in order to determine whether it is an I, P or B frame. If the frame is not an I frame 1021, it is converted into an I frame 1030. Finally, original 1022 or converted 1030 I frames are then used as to create duplicate P frames 1040. Periodical I frames can be inserted to increase interactivity, and to maintain a compatible bitrate.
- a fourth important tool used in film editing is variable speed playback.
- Faster than normal playback e.g., fast forward
- slow motion depending on the slow motion rate, there are two possible approaches.
- one approach is to simply insert duplicate B frames (B') whenever an original B frame appears.
- Duplicate B frames are frames which copy all DCT coefficient, and associated motion vectors from previous B frame.
- this approach has a drawback since the I/P frame delay is increased by a factor equal to the inverse of the motion rate.
- the reference frame I 0 must be transmitted 4 frames earlier. This limitation makes this approach suitable for slow playback up to about Vi normal frame rate. This will require an increased decoder buffer size.
- a master server 1210 or a group of distributed servers is linked to distant servers 1220, 1230, 1240 and several clients 1250, 1260. From their client workstations, users are empowered to browse and edit compressed vide images in the manner described above.
- the master server 1210 acts as the content aggregator 1310 to collect video content, and as a content analyzer 1320 to analyze the visual content for efficient indexing 1330.
- the server 1210 may accept the functions performed by the parsing module 110 described above.
- the server 1210 also provides an editing engine 1340 with basic editing tools, such as the tools described above, for rendering basic editing functions and various special effects.
- Distributed clients 1250, 1260 access the video archives of the server through heterogeneous networks with interoperable interfaces, such as JAVA or MPEG MSDL manipulation tools.
- the video server 1210 can be linked with other distributed servers 1220, 1230, 1240 that have video search engines which search for image and video over a network, e.g., the World Wide Web.
- a video file Once a video file is found on any other hosts or online content providers, it will be downloaded and preprocessed by the video server 1210 to extract "keyframes," i.e., frames 121 stored in the above described visualization module 120, and associated visual features such as camera motion, moving objects, color, texture, and temporal features.
- the Universal Resource Location (“URL") of the video and the extracted features will be stored on the video server 1210.
- This client-server model empowers clients 1250, 1260 with much richer resources beyond that provided by the client's local system.
- the client 1260 may open any video shot stored at the server 1210 and browse 1330 the keyframes hierarchically using any one of numerous different viewing models, including sequential view, feature-based view, and story-based view.
- a sequential view arranges the video shots according to the sequential time order;
- a feature-based view clusters video shots with similar visual features to the same group;
- a story view presents video shots according to the story structures in the semantic level.
- the story model may use the anchorperson shot as the criterion for automatic segmentation of video into semantic-level stories.
- the client can also perform shot-level editing or frame level editing 1390 using the above discussed editing tools.
- the clients may also submit their own videos which may be analyzed and indexed by the server 1210.
- the client 1260 may choose different levels of resolution for the video shots sent by the server 1210. There are three broad levels of video rendering. At the first level, the client 1260 can render only straight cuts at low resolution without any special effects. At the second level, the client may send information to the server defining low-resolution video with desired special effects to be generated. Finally, when the client 1260 no longer wishes to perform editing, the master server 1210 will generate a full-resolution video with all the effects requested by the client from the highest quality source video which is located at either the master server 1210 or the distributed remote content servers 1220, 1230, 1240. In this way, the user is given the flexibility to tradeoff between quality and speed.
- a low resolution icon stream video In order to obtain a reduced resolution video sequence, for each video sequence stored on a server, a low resolution icon stream video must be extracted from the full resolution video.
- the icon stream is an 8:1 down sampled version of the original video.
- DCT DC coefficients are obtained directly.
- B and P frames DCT domain inverse motion compensation is applied to the DCT coefficients.
- the extracted coefficients are converted into a smaller sized, e.g., 8:2:1, 1 frame as those skilled in the art will appreciate.
Abstract
Description
Claims
Priority Applications (7)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CA002289757A CA2289757A1 (en) | 1997-05-16 | 1997-05-16 | Methods and architecture for indexing and editing compressed video over the world wide web |
PCT/US1997/008266 WO1998052356A1 (en) | 1997-05-16 | 1997-05-16 | Methods and architecture for indexing and editing compressed video over the world wide web |
US09/423,769 US6735253B1 (en) | 1997-05-16 | 1997-05-16 | Methods and architecture for indexing and editing compressed video over the world wide web |
JP54917698A JP2001526859A (en) | 1997-05-16 | 1997-05-16 | Instruction and editing method of compressed image on world wide web and architecture |
US10/728,345 US7817722B2 (en) | 1997-05-16 | 2003-12-04 | Methods and architecture for indexing and editing compressed video over the world wide web |
US12/874,337 US20110064136A1 (en) | 1997-05-16 | 2010-09-02 | Methods and architecture for indexing and editing compressed video over the world wide web |
US13/078,626 US9330722B2 (en) | 1997-05-16 | 2011-04-01 | Methods and architecture for indexing and editing compressed video over the world wide web |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US1997/008266 WO1998052356A1 (en) | 1997-05-16 | 1997-05-16 | Methods and architecture for indexing and editing compressed video over the world wide web |
Related Child Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09423769 A-371-Of-International | 1997-05-16 | ||
US09/423,769 A-371-Of-International US6735253B1 (en) | 1997-05-16 | 1997-05-16 | Methods and architecture for indexing and editing compressed video over the world wide web |
US10/728,345 Division US7817722B2 (en) | 1997-05-16 | 2003-12-04 | Methods and architecture for indexing and editing compressed video over the world wide web |
Publications (1)
Publication Number | Publication Date |
---|---|
WO1998052356A1 true WO1998052356A1 (en) | 1998-11-19 |
Family
ID=22260907
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1997/008266 WO1998052356A1 (en) | 1997-05-16 | 1997-05-16 | Methods and architecture for indexing and editing compressed video over the world wide web |
Country Status (3)
Country | Link |
---|---|
JP (1) | JP2001526859A (en) |
CA (1) | CA2289757A1 (en) |
WO (1) | WO1998052356A1 (en) |
Cited By (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2000051078A2 (en) * | 1999-02-22 | 2000-08-31 | Siemens Corporate Research, Inc. | Method and apparatus for authoring and linking video documents |
EP1100021A2 (en) * | 1999-11-09 | 2001-05-16 | Young Kyu Jang | Method and system for editing moving image data and providing object information based thereon |
WO2001041451A1 (en) * | 1999-11-29 | 2001-06-07 | Sony Corporation | Video/audio signal processing method and video/audio signal processing apparatus |
WO2001063937A2 (en) * | 2000-02-24 | 2001-08-30 | Sarnoff Corporation | Compressed video analysis |
WO2001069936A2 (en) * | 2000-03-13 | 2001-09-20 | Sony Corporation | Method and apparatus for generating compact transcoding hints metadata |
JP2001258038A (en) * | 2000-01-31 | 2001-09-21 | Canon Inc | Method for extracting key frame of video sequence, key frame extracting device for video sequence, storage medium and program software |
EP1154432A2 (en) * | 2000-05-12 | 2001-11-14 | Sony Corporation | Service providing apparatus, service providing method, receiving terminal apparatus, receiving method, and service providing system |
DE19944300C2 (en) * | 1999-09-15 | 2002-08-29 | Siemens Ag | Method, arrangement and computer program product for motion estimation when encoding an image object in an image |
EP1255251A1 (en) * | 2001-05-02 | 2002-11-06 | ClicknShoot | Method of remote processing of an article, particularly method of selective digitizing of video tapes |
EP1305944A2 (en) * | 2000-06-27 | 2003-05-02 | Bamboo Mediacasting, Inc. | Bandwidth scaling of a compressed video stream |
EP1318467A2 (en) * | 2001-12-05 | 2003-06-11 | Sony Corporation | Data searching and data editing |
AU762791B2 (en) * | 2000-01-31 | 2003-07-03 | Canon Kabushiki Kaisha | Extracting key frames from a video sequence |
WO2003077549A1 (en) * | 2002-03-13 | 2003-09-18 | Imax Corporation | Systems and methods for digitally re-mastering or otherwise modifying motion pictures or other image sequences data |
WO2003047268A3 (en) * | 2001-11-30 | 2003-10-09 | Ericsson Telefon Ab L M | Global motion compensation for video pictures |
EP1411522A3 (en) * | 2002-10-14 | 2004-09-08 | Samsung Electronics Co., Ltd. | Determining a scene change point |
WO2004100162A2 (en) * | 2003-04-30 | 2004-11-18 | Hewlett-Packard Development Company, L.P. | Methods and systems for video content browsing |
US6961060B1 (en) * | 1999-03-16 | 2005-11-01 | Matsushita Electric Industrial Co., Ltd. | Virtual space control data receiving apparatus,virtual space control data transmission and reception system, virtual space control data receiving method, and virtual space control data receiving program storage media |
EP1752891A2 (en) * | 1999-11-29 | 2007-02-14 | Sony Corporation | Method and apparatus for establishing and browsing a hierarchical video camera motion transition graph. |
CN100336390C (en) * | 1999-11-29 | 2007-09-05 | 索尼公司 | Step decomposition method and apparatus for extracting synthetic video selection for browsing |
WO2007102862A1 (en) | 2006-03-09 | 2007-09-13 | Thomson Licensing | Content access tree |
CN100348032C (en) * | 1999-11-29 | 2007-11-07 | 索尼公司 | Method and equipment for processing and browsing provided video/audio signal |
US7646437B1 (en) * | 2003-09-03 | 2010-01-12 | Apple Inc. | Look-ahead system and method for pan and zoom detection in video sequences |
US7738550B2 (en) | 2000-03-13 | 2010-06-15 | Sony Corporation | Method and apparatus for generating compact transcoding hints metadata |
US8842730B2 (en) | 2006-01-27 | 2014-09-23 | Imax Corporation | Methods and systems for digitally re-mastering of 2D and 3D motion pictures for exhibition with enhanced visual quality |
US9282313B2 (en) | 2006-06-23 | 2016-03-08 | Imax Corporation | Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101058673B1 (en) | 2004-02-10 | 2011-08-22 | 삼성전자주식회사 | Text image transmission method of mobile terminal |
JP7428855B2 (en) * | 2020-03-17 | 2024-02-07 | Lineヤフー株式会社 | Video analysis system, video analysis device, video analysis method, and program |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5191645A (en) * | 1991-02-28 | 1993-03-02 | Sony Corporation Of America | Digital signal processing system employing icon displays |
US5204706A (en) * | 1990-11-30 | 1993-04-20 | Kabushiki Kaisha Toshiba | Moving picture managing device |
US5262856A (en) * | 1992-06-04 | 1993-11-16 | Massachusetts Institute Of Technology | Video image compositing techniques |
US5353391A (en) * | 1991-05-06 | 1994-10-04 | Apple Computer, Inc. | Method apparatus for transitioning between sequences of images |
US5408274A (en) * | 1993-03-11 | 1995-04-18 | The Regents Of The University Of California | Method and apparatus for compositing compressed video data |
US5461679A (en) * | 1991-05-24 | 1995-10-24 | Apple Computer, Inc. | Method and apparatus for encoding/decoding image data |
US5606655A (en) * | 1994-03-31 | 1997-02-25 | Siemens Corporate Research, Inc. | Method for representing contents of a single video shot using frames |
-
1997
- 1997-05-16 JP JP54917698A patent/JP2001526859A/en active Pending
- 1997-05-16 WO PCT/US1997/008266 patent/WO1998052356A1/en active Application Filing
- 1997-05-16 CA CA002289757A patent/CA2289757A1/en not_active Abandoned
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5204706A (en) * | 1990-11-30 | 1993-04-20 | Kabushiki Kaisha Toshiba | Moving picture managing device |
US5191645A (en) * | 1991-02-28 | 1993-03-02 | Sony Corporation Of America | Digital signal processing system employing icon displays |
US5353391A (en) * | 1991-05-06 | 1994-10-04 | Apple Computer, Inc. | Method apparatus for transitioning between sequences of images |
US5461679A (en) * | 1991-05-24 | 1995-10-24 | Apple Computer, Inc. | Method and apparatus for encoding/decoding image data |
US5262856A (en) * | 1992-06-04 | 1993-11-16 | Massachusetts Institute Of Technology | Video image compositing techniques |
US5408274A (en) * | 1993-03-11 | 1995-04-18 | The Regents Of The University Of California | Method and apparatus for compositing compressed video data |
US5606655A (en) * | 1994-03-31 | 1997-02-25 | Siemens Corporate Research, Inc. | Method for representing contents of a single video shot using frames |
Cited By (53)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2000051078A2 (en) * | 1999-02-22 | 2000-08-31 | Siemens Corporate Research, Inc. | Method and apparatus for authoring and linking video documents |
WO2000051078A3 (en) * | 1999-02-22 | 2002-02-07 | Siemens Corp Res Inc | Method and apparatus for authoring and linking video documents |
US6961060B1 (en) * | 1999-03-16 | 2005-11-01 | Matsushita Electric Industrial Co., Ltd. | Virtual space control data receiving apparatus,virtual space control data transmission and reception system, virtual space control data receiving method, and virtual space control data receiving program storage media |
DE19944300C2 (en) * | 1999-09-15 | 2002-08-29 | Siemens Ag | Method, arrangement and computer program product for motion estimation when encoding an image object in an image |
EP1100021A2 (en) * | 1999-11-09 | 2001-05-16 | Young Kyu Jang | Method and system for editing moving image data and providing object information based thereon |
EP1100021A3 (en) * | 1999-11-09 | 2005-11-23 | Young Kyu Jang | Method and system for editing moving image data and providing object information based thereon |
JP2003516053A (en) * | 1999-11-29 | 2003-05-07 | ソニー株式会社 | Video / audio signal processing method and video / audio signal processing apparatus |
CN100348032C (en) * | 1999-11-29 | 2007-11-07 | 索尼公司 | Method and equipment for processing and browsing provided video/audio signal |
WO2001041451A1 (en) * | 1999-11-29 | 2001-06-07 | Sony Corporation | Video/audio signal processing method and video/audio signal processing apparatus |
US7356082B1 (en) | 1999-11-29 | 2008-04-08 | Sony Corporation | Video/audio signal processing method and video-audio signal processing apparatus |
EP1752891A3 (en) * | 1999-11-29 | 2007-02-21 | Sony Corporation | Method and apparatus for establishing and browsing a hierarchical video camera motion transition graph. |
CN100336390C (en) * | 1999-11-29 | 2007-09-05 | 索尼公司 | Step decomposition method and apparatus for extracting synthetic video selection for browsing |
EP1752891A2 (en) * | 1999-11-29 | 2007-02-14 | Sony Corporation | Method and apparatus for establishing and browsing a hierarchical video camera motion transition graph. |
JP4666784B2 (en) * | 2000-01-31 | 2011-04-06 | キヤノン株式会社 | Video sequence key frame extraction method and video sequence key frame extraction device |
JP2001258038A (en) * | 2000-01-31 | 2001-09-21 | Canon Inc | Method for extracting key frame of video sequence, key frame extracting device for video sequence, storage medium and program software |
AU762791B2 (en) * | 2000-01-31 | 2003-07-03 | Canon Kabushiki Kaisha | Extracting key frames from a video sequence |
WO2001063937A3 (en) * | 2000-02-24 | 2002-01-31 | Sarnoff Corp | Compressed video analysis |
WO2001063937A2 (en) * | 2000-02-24 | 2001-08-30 | Sarnoff Corporation | Compressed video analysis |
US7738550B2 (en) | 2000-03-13 | 2010-06-15 | Sony Corporation | Method and apparatus for generating compact transcoding hints metadata |
KR100776529B1 (en) * | 2000-03-13 | 2007-11-16 | 소니 가부시끼 가이샤 | Method and apparatus for generating compact transcoding hints metadata |
AU2005202313B2 (en) * | 2000-03-13 | 2008-06-19 | Sony Corporation | Method and apparatus for generating compact transcoding hints metadata |
AU780811B2 (en) * | 2000-03-13 | 2005-04-21 | Sony Corporation | Method and apparatus for generating compact transcoding hints metadata |
KR100844816B1 (en) * | 2000-03-13 | 2008-07-09 | 소니 가부시끼 가이샤 | Method and apparatus for generating compact transcoding hints metadata |
CN100338956C (en) * | 2000-03-13 | 2007-09-19 | 索尼公司 | Method and apapratus for generating compact transcoding hints metadata |
WO2001069936A3 (en) * | 2000-03-13 | 2002-02-28 | Sony Corp | Method and apparatus for generating compact transcoding hints metadata |
WO2001069936A2 (en) * | 2000-03-13 | 2001-09-20 | Sony Corporation | Method and apparatus for generating compact transcoding hints metadata |
US7359616B2 (en) | 2000-05-12 | 2008-04-15 | Sony Corporation | Service providing apparatus, service providing method, receiving terminal apparatus, receiving method, and service providing system |
EP1154432A3 (en) * | 2000-05-12 | 2004-09-29 | Sony Corporation | Service providing apparatus, service providing method, receiving terminal apparatus, receiving method, and service providing system |
EP1154432A2 (en) * | 2000-05-12 | 2001-11-14 | Sony Corporation | Service providing apparatus, service providing method, receiving terminal apparatus, receiving method, and service providing system |
EP1305944A2 (en) * | 2000-06-27 | 2003-05-02 | Bamboo Mediacasting, Inc. | Bandwidth scaling of a compressed video stream |
EP1305944A4 (en) * | 2000-06-27 | 2010-10-06 | Bamboo Mediacasting Ltd | Bandwidth scaling of a compressed video stream |
EP1255251A1 (en) * | 2001-05-02 | 2002-11-06 | ClicknShoot | Method of remote processing of an article, particularly method of selective digitizing of video tapes |
WO2003047268A3 (en) * | 2001-11-30 | 2003-10-09 | Ericsson Telefon Ab L M | Global motion compensation for video pictures |
EP2202981A1 (en) * | 2001-11-30 | 2010-06-30 | Telefonaktiebolaget L M Ericsson (Publ) | Global motion compensation for video pictures |
EP1318467A3 (en) * | 2001-12-05 | 2004-09-01 | Sony Corporation | Data searching and data editing |
EP1318467A2 (en) * | 2001-12-05 | 2003-06-11 | Sony Corporation | Data searching and data editing |
WO2003077549A1 (en) * | 2002-03-13 | 2003-09-18 | Imax Corporation | Systems and methods for digitally re-mastering or otherwise modifying motion pictures or other image sequences data |
US7856055B2 (en) | 2002-03-13 | 2010-12-21 | Imax Corporation | Systems and methods for digitally re-mastering or otherwise modifying motion pictures or other image sequences data |
EP1411522A3 (en) * | 2002-10-14 | 2004-09-08 | Samsung Electronics Co., Ltd. | Determining a scene change point |
US7305171B2 (en) | 2002-10-14 | 2007-12-04 | Samsung Electronics Co., Ltd. | Apparatus for recording and/or reproducing digital data, such as audio/video (A/V) data, and control method thereof |
WO2004100162A3 (en) * | 2003-04-30 | 2005-02-10 | Hewlett Packard Development Co | Methods and systems for video content browsing |
US7552387B2 (en) | 2003-04-30 | 2009-06-23 | Hewlett-Packard Development Company, L.P. | Methods and systems for video content browsing |
WO2004100162A2 (en) * | 2003-04-30 | 2004-11-18 | Hewlett-Packard Development Company, L.P. | Methods and systems for video content browsing |
US7646437B1 (en) * | 2003-09-03 | 2010-01-12 | Apple Inc. | Look-ahead system and method for pan and zoom detection in video sequences |
US20100110303A1 (en) * | 2003-09-03 | 2010-05-06 | Apple Inc. | Look-Ahead System and Method for Pan and Zoom Detection in Video Sequences |
US20130308058A1 (en) * | 2003-09-03 | 2013-11-21 | Apple Inc. | Look-ahead system and method for pan and zoom detection in video sequences |
US8625671B2 (en) | 2003-09-03 | 2014-01-07 | Apple Inc. | Look-ahead system and method for pan and zoom detection in video sequences |
US8902986B2 (en) | 2003-09-03 | 2014-12-02 | Apple Inc. | Look-ahead system and method for pan and zoom detection in video sequences |
US8842730B2 (en) | 2006-01-27 | 2014-09-23 | Imax Corporation | Methods and systems for digitally re-mastering of 2D and 3D motion pictures for exhibition with enhanced visual quality |
EP1991923A4 (en) * | 2006-03-09 | 2009-04-08 | Thomson Licensing | Content access tree |
EP1991923A1 (en) * | 2006-03-09 | 2008-11-19 | Thomson Licensing | Content access tree |
WO2007102862A1 (en) | 2006-03-09 | 2007-09-13 | Thomson Licensing | Content access tree |
US9282313B2 (en) | 2006-06-23 | 2016-03-08 | Imax Corporation | Methods and systems for converting 2D motion pictures for stereoscopic 3D exhibition |
Also Published As
Publication number | Publication date |
---|---|
JP2001526859A (en) | 2001-12-18 |
CA2289757A1 (en) | 1998-11-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6735253B1 (en) | Methods and architecture for indexing and editing compressed video over the world wide web | |
WO1998052356A1 (en) | Methods and architecture for indexing and editing compressed video over the world wide web | |
Meng et al. | CVEPS-a compressed video editing and parsing system | |
US7046731B2 (en) | Extracting key frames from a video sequence | |
US7469010B2 (en) | Extracting key frames from a video sequence | |
JP3719933B2 (en) | Hierarchical digital video summary and browsing method and apparatus | |
Meng et al. | Tools for compressed-domain video indexing and editing | |
Meng et al. | Scene change detection in an MPEG-compressed video sequence | |
JP4942883B2 (en) | Method for summarizing video using motion and color descriptors | |
US20080043848A1 (en) | Video/audio signal processing method and video/audio signal processing apparatus | |
JP2007533197A (en) | Encoding method applied to multimedia data | |
Faernando et al. | Scene change detection algorithms for content-based video indexing and retrieval | |
Nakajima | A video browsing using fast scene cut detection for an efficient networked video database access | |
Ratakonda et al. | Hierarchical video summarization | |
De Bruyne et al. | A compressed-domain approach for shot boundary detection on H. 264/AVC bit streams | |
Schonfeld et al. | Vortex: video retrieval and tracking from compressed multimedia databases—multiple object tracking from mpeg-2 bit stream | |
Yeo | On fast microscopic browsing of MPEG-compressed video | |
Gu et al. | Dissolve detection in MPEG compressed video | |
Meng et al. | A distributed system for editing and browsing compressed video over the network | |
Kaup et al. | Video analysis for universal multimedia messaging | |
Meng et al. | Searching and editing MPEG-compressed video in a distributed online environment | |
AU762791B2 (en) | Extracting key frames from a video sequence | |
US20090016441A1 (en) | Coding method and corresponding coded signal | |
Yi et al. | A motion-based scene tree for compressed video content management | |
Mithlesh et al. | Detection of Scene Change in Video |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): CA JP US |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
ENP | Entry into the national phase |
Ref document number: 2289757 Country of ref document: CA Ref country code: CA Ref document number: 2289757 Kind code of ref document: A Format of ref document f/p: F |
|
ENP | Entry into the national phase |
Ref country code: JP Ref document number: 1998 549176 Kind code of ref document: A Format of ref document f/p: F |
|
WWE | Wipo information: entry into national phase |
Ref document number: 09423769 Country of ref document: US |