WO2002071331A2 - Predictive edge extension into uncovered regions - Google Patents

Predictive edge extension into uncovered regions Download PDF

Info

Publication number
WO2002071331A2
WO2002071331A2 PCT/US2002/007220 US0207220W WO02071331A2 WO 2002071331 A2 WO2002071331 A2 WO 2002071331A2 US 0207220 W US0207220 W US 0207220W WO 02071331 A2 WO02071331 A2 WO 02071331A2
Authority
WO
WIPO (PCT)
Prior art keywords
edge
uncovered region
uncovered
point
trivalent
Prior art date
Application number
PCT/US2002/007220
Other languages
French (fr)
Other versions
WO2002071331A3 (en
Inventor
Adityo Prakash
Eniko F. Prakash
Edward Ratner
John S. Chen
Original Assignee
Pulsent Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pulsent Corporation filed Critical Pulsent Corporation
Priority to AU2002255684A priority Critical patent/AU2002255684A1/en
Publication of WO2002071331A2 publication Critical patent/WO2002071331A2/en
Publication of WO2002071331A3 publication Critical patent/WO2002071331A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00Image coding
    • G06T9/004Predictors, e.g. intraframe, interframe coding

Definitions

  • the present invention relates in general to image processing. More particularly, it relates to segment- or object-based image processing.
  • Information regarding segments or objects in an image may be used to facilitate digital image or video processing.
  • a previously hidden area of an image may become uncovered.
  • a frame may include various segments or objects.
  • one or more ofthe segments or objects may move.
  • a previously hidden area may become visible.
  • DCT discrete cosine transform
  • One embodiment ofthe invention comprises a method for image processing.
  • the method includes identifying an uncovered region in an image, determining at least one trivalent point, and extrapolating from the trivalent point to extend an edge of at least one of the image segments into the uncovered region.
  • Another embodiment ofthe invention comprises an apparatus for image processing.
  • the apparatus includes a predictor for predicting higher- frequency boundary information in newly uncovered regions.
  • Another embodiment ofthe invention comprises a system for efficiently communicating video information.
  • the system includes an encoder that encodes a video frame into an encoded frame, and a decoder that receives and decodes the encoded frame, wherein the encoder and decoder are both configured to identify uncovered regions and to extend edges from neighboring image segments into the uncovered region.
  • FIG. 1 is an illustration depicting a previously hidden area becoming visible (i.e. becoming an uncovered region) due to segment motion.
  • FIG. 2 is a flow chart depicting a method for image processing in accordance with an embodiment ofthe invention.
  • FIG. 3 is a diagram of an apparatus including a predictor for uncovered regions in accordance with an embodiment ofthe invention.
  • FIG. 4 is an illustration depicting a trivalent point adjacent to an uncovered region in accordance with an embodiment ofthe invention.
  • FIG. 5 is an illustration depicting predictive extension of an edge in accordance with an embodiment ofthe invention.
  • FIG. 6 is an illustration depicting the predictive extension of an edge terminating at a boundary ofthe uncovered region in accordance with an embodiment ofthe invention.
  • FIG. 7 is an illustration depicting multiple trivalent points adjacent to an uncovered region in accordance with an embodiment ofthe invention.
  • FIG. 8 is an illustration depicting predictive extension of multiple edges in accordance with an embodiment ofthe invention.
  • FIG. 9 is an illustration depicting the predictive extension of a first edge that terminates at a boundary ofthe uncovered region and of a second edge that terminates at the first edge in accordance with an embodiment ofthe invention.
  • FIG. 10 is an illustration depicting predictive extension of an edge that skips intermediary points in accordance with an embodiment ofthe invention.
  • FIG. 11 is a diagram of a system for communicating video in accordance with an embodiment ofthe invention.
  • FIGS. 12A-C depict another example of a trivalent point that may be used as a starting point in accordance with an embodiment ofthe invention.
  • FIG. 1 is an illustration depicting a previously hidden or occluded area becoming visible (i.e. becoming an uncovered region) due to segment motion. As shown in FIG. 1, as the vehicle segment(s) moves, three previously hidden areas (101, 102, and 103) become visible. These uncovered regions are: the region 101 behind the back ofthe car; the region 102 behind the rear wheels; and the region 103 behind the front wheels.
  • FIG. 2 is a flow chart depicting a method for image processing in accordance with an embodiment ofthe invention.
  • the method 200 depicted in FIG. 2 includes six steps (202, 203, 204, 206, 208, and 210).
  • the first step 202 an uncovered region in a frame is identified. Examples of such uncovered regions are depicted in FIG. 1. Uncovered regions may be considered as part ofthe residue after motion prediction. (Residue information after motion prediction includes information from uncovered regions, information from inexact matches, and newly appearing information.) Segment-based motion prediction and residues are discussed further in International Publication No.
  • a surrounding segment must share a common boundary with the newly uncovered region.
  • the common boundary must be of a certain minimum size.
  • a starting point is determined.
  • the starting point comprises a trivalent point.
  • a trivalent point is a point at the intersection of three image "objects.” The objects may be either image segments or an uncovered region.
  • a trivalent point may be at the intersection between two image segments and an uncovered region.
  • a trivalent point is depicted, for example, in FIG. 4 (described below).
  • FIG. 7 depicts an example of multiple trivalent points adjacent to an uncovered region.
  • extrapolation from the starting point occurs to extend a segment edge.
  • the extrapolation extends the edge from the starting point into the uncovered region.
  • the extrapolation may be performed using directional information from the edge to be extended.
  • An edge extension from a starting point into an uncovered region is depicted, for example, in FIG. 5 (described below).
  • FIG. 8 depicts an example of edge extension from multiple starting points into an uncovered region.
  • the extrapolation continues until the edge extension reaches a termination point.
  • the termination point may be, for example, a point on the boundary ofthe uncovered region.
  • the uncovered region is divided such that each division may be associated with an adjacent image segment.
  • FIG. 9 depicts an example of multiple edge extensions until termination and the resulting division of an uncovered region.
  • the divisions ofthe uncovered regions may be predictively filled.
  • the predictive filling may be accomplished by applying a predictive filling routine to expand lower- frequency information from adjacent segments into the associated divisions ofthe uncovered region.
  • FIG. 3 is a diagram of an apparatus including a predictor for uncovered regions in accordance with an embodiment ofthe invention.
  • the predictor 320 may receive as input information or data relating to uncovered regions in an image frame.
  • the predictor 320 may output predicted high-frequency information for the uncovered regions in the frame.
  • the predictor 320 depicted in FIG. 3 includes three components working in cooperation with each other (321, 322, 324).
  • the input into the predictor 320 may be received from a hidden area locator 310.
  • the hidden area locator 310 is configured to locate uncovered regions (previously hidden or occluded areas) in a frame. As mentioned above, the uncovered regions may be considered as part ofthe residue after motion prediction.
  • the hidden area locator 302 analyzes the residue after motion prediction and determines which residues correspond to uncovered areas. Other residues (besides those corresponding to uncovered areas) may correspond to inexact matches or newly appearing information.
  • the hidden area locator 310 may be integrated with the predictor 320 or may be integrated with another component.
  • the first component ofthe predictor 320 comprises a surrounding segment determinator 321.
  • the surrounding segment determinator 321 is configured to determine the image segments surrounding the uncovered region.
  • a surrounding segment must share a common boundary with the newly uncovered region.
  • the common boundary must be of a certain minimum size.
  • the second component ofthe predictor 320 comprises a junction identifier 322.
  • Thejunction identifier 322 is configured to determine the starting points.
  • the junction identifier 322 locates trivalent points adjacent to the uncovered region and designates those trivalent points as starting points.
  • a trivalent point is a point at the intersection of three image "objects.”
  • the objects may be either image segments or an uncovered region.
  • a trivalent point may be at the intersection between two image segments and an uncovered region.
  • a trivalent point is depicted, for example, in FIG. 4 (described below).
  • FIG. 7 depicts an example of multiple trivalent points adjacent to an uncovered region.
  • the third component comprises an edge extender 324.
  • the edge extender 324 extends an edge by extrapolation from the starting point into the uncovered region. The extrapolation may be performed using directional information from the edge to be extended.
  • An edge extension from a starting point into an uncovered region is depicted, for example, in FIG. 5 (described below).
  • FIG. 8 depicts an example of edge extension from multiple starting points into an uncovered region.
  • the edge extender 324 also terminates extension ofthe edges at appropriate termination points.
  • the termination point may be, for example, a point on the boundary of the uncovered region.
  • the uncovered region is divided such that each division may be associated with an adjacent segment.
  • Edge extension until termination and the resulting division of an uncovered region are depicted, for example, in FIG. 6 (described below).
  • FIG. 9 depicts an example of multiple edge extensions until termination and the resulting division of an uncovered region.
  • a segment expander may be configured to apply a predictive filling routine. Such a routine may be used to expand lower-frequency information from the adjacent segments to the associated division of the uncovered region.
  • FIG. 4 is an illustration depicting a trivalent point adjacent to an uncovered region in accordance with an embodiment ofthe invention.
  • the illustration shows three adjacent image segments 402, 404, and 406.
  • the third segment 406 moves from first image frame to a second image frame.
  • the position ofthe third segment 406 in the second frame is shown by the solid white region labeled 406.
  • Previously hidden areas that are exposed by movement ofthe third segment 406 from the first to second frames is shown by the diagonally-striped areas (408A, 408B, and 408C). These previously hidden areas correspond to uncovered regions.
  • the following discussion focuses on the first uncovered region labeled 408A.
  • the first uncovered region 408A is adjacent to the first and second image segments 402 and 404. At the intersection ofthe uncovered region 408 A and those two segments 402 and 404 is shown a trivalent point 410.
  • the trivalent point 410 is a starting point for the edge extension.
  • FIG. 5 is an illustration depicting predictive extension (or extrapolation) of an edge in accordance with an embodiment ofthe invention.
  • the illustration shows the extrapolation 502 beginning at the starting point 410 and extending the edge between the first and second segments 402 and 404 into the uncovered region 408A.
  • the extrapolation 502 may be performed using directional information from previous points ofthe edge.
  • the extrapolation 502 depends on an average directional (tangent) vector from previous edge points. For example, directional vectors from the previous ten (10) edge points may be averaged to determine the average directional vector that indicates the extrapolated location ofthe next edge point. Of course, more or less than ten (10) edge points may be used.
  • the extrapolation uses a polynomial predictive algorithm. Polynomial predictive algorithms may be used to extrapolate signals. Here polynomical prediction may be used to extrapolate the edges. Polynomial prediction may involve taking the portion ofthe edge to be extended (that close to the trivalent point) and modeling it by an nth-degree polyomial. Typically, a relatively low-degree (4 th degree or less) would be practical. Other algorithms may also be used and are contemplated to be within the scope ofthe invention.
  • FIG. 6 is an illustration depicting the predictive extension of an edge terminating at a boundary ofthe uncovered region in accordance with an embodiment ofthe invention.
  • the example depicted in FIG. 6 shows the predictive extension 602 continuing from the starting point 410 until it reaches a termination point 604 at a boundary ofthe uncovered region 408A.
  • the extension 602 is depicted as a straight line in FIG. 6, it need not be such a straight line. Instead, as described above in relation to FIG. 5, the extension may be a non-straight line determined using, for example, an average directional vector or a polynomial predictive algorithm.
  • the completed extension ofthe edge divides the uncovered region 408A into two divisions, one associated with adjacent segment 402 and the other associated with adjacent segment 404.
  • FIG. 7 is an illustration depicting multiple trivalent points adjacent to an uncovered region in accordance with an embodiment ofthe invention.
  • the example of FIG. 7 differs from the example of FIG. 4 in that there are depicted three image segments (702, 704, and 706) adjacent to the uncovered region 408A. Since there are three adjacent segments, there are now two starting points 708 and 710. In accordance to an embodiment of the invention, the two starting points may each be located at a trivalent point between the uncovered region 408 A and two ofthe segments. In other examples, of course, there may be more than three adjacent segments and hence more than two starting points.
  • FIG. 8 is an illustration depicting predictive extension (or extrapolation) of multiple edges in accordance with an embodiment ofthe invention.
  • the illustration shows the extrapolations 802 and 804 beginning at the starting points 708 and 710, respectively.
  • the first extrapolation 802 extends the edge between the first and second segments 702 and 704 into the uncovered region 408A.
  • the second extrapolation 804 extends the edge between the second and third segments 704 and 706 into the uncovered region 408A.
  • each extrapolation (802 or 804) may be performed using directional information from previous points of its respective edge.
  • FIG. 9 is an illustration depicting the predictive extension of a first edge that terminates at a boundary ofthe uncovered region and of a second edge that terminates at the first edge in accordance with an embodiment ofthe invention.
  • the example depicted in FIG. 6 shows the first and second predictive extensions (902 and 904, respectively), continuing in parallel from the first and second starting points (708 and 710, respectively), until they reach the first and second termination points (906 and 908, respectively).
  • the first termination point 906 occurs at a boundary ofthe uncovered region 408 A. This first type of termination point is discussed above in relation to FIG. 6.
  • the second termination point 908 is not at a boundary ofthe uncovered region 408A. Instead, the second termination point 908 occurs when the second predictive extension 904 intersects the first predictive extension 902. In other words, the predictive extension ofthe second edge terminates at the first edge.
  • the completed extensions ofthe edges divide the uncovered region 408 A into three divisions. A first division is associated with first adjacent segment 702. A second division is associated with the second adjacent segment 704. A third division is associated with the third adjacent segment 706.
  • extensions 902 and 904 are depicted as straight lines in FIG. 6, they need not be straight lines. Instead, as described above in relation to FIG. 5, the extensions 902 and 904 may be a non-straight line determined using, for example, an average directional vector or a polynomial predictive algorithm.
  • FIG. 10 is an illustration depicting predictive extension of an edge that skips intermediary points in accordance with an embodiment ofthe invention.
  • the extrapolation may not predict every point in the extensions 902 and 904 by an extrapolating algorithm. Instead, only certain points such as, for example, every nth point, may be predicted.
  • the "intermediary" points in between the predicted points may be filled in later, for example, by interpolation between predicted points. In other words, the intermediary points are subsequently determined by filling in between the dots ofthe dotted line.
  • the predictive filling process may be performed for each surrounding segment until all ofthe segment boundaries have been extended into the newly uncovered image region.
  • Such a process provides a prediction for the image content in the uncovered region.
  • this prediction may vary from the actual content ofthe uncovered region.
  • FIG. 11 is a diagram of a system for communicating video in accordance with an embodiment ofthe invention.
  • the system 1100 may include an encoder 1102, a decoder 1104, and a communications channel 1106.
  • the communications channel 1106 connects the encoder 1 102 to the decoder 1104 and provides a channel for transmitting information between the two devices.
  • an encoder receives a raw, i.e., uncompressed, image frame, manipulates and compresses the image frame, and transmits the compressed image to an decoder.
  • the decoder then decompresses, i.e., reconstructs, the image frame.
  • the encoder manipulates portions of image frames called image segments. Often, as segments change location from frame to frame, there are newly uncovered regions that come into view. The encoder, having the original raw image, knows the complete structure ofthe newly uncovered image regions. However, the decoder likely has no information regarding the image data within the newly uncovered image regions.
  • the encoder 1102 receives image data and, in accordance with an embodiment ofthe invention, is configured to process the image data using a predictor 320, such as one described above in relation to FIG. 3.
  • the encoder 1102 subsequently transmits encoded image information across the communications channel 1106 to the decoder 1104.
  • the decoder 1104 receives the encoded video information and uses it to reconstruct the image.
  • the decoder 1104 may perform the reconstruction utilizing a same or equivalent predictor 302 algorithm as the encoder 1102 uses.
  • the encoder 1102 does not need to transmit the predicted filling ofthe uncovered regions. This is because the encoder 1102 and the decoder 1104 are able to use the same or equivalent predictive filling.
  • FIGS. 12A-C depict another example of a trivalent point that may be used as a starting point in accordance with an embodiment ofthe invention.
  • FIG. 12A depicts frame N-l
  • FIG. 12B depicts frame N, where frame N-l represents a preceding frame in a video sequence and frame N represents the current frame.
  • FIG. 12B the region to the left ofthe ball is shown as blank because that region is an uncovered region (newly exposed hidden area) in this example.
  • the present invention may be applied to such an uncovered region.
  • the trivalent point 1202 from FIG. 12A (the prior frame) is used as the starting point.
  • FIG. 12C shows the extrapolation 1204 beginning from that point 1202. In such a case, directional information from the edge in FIG.
  • This directional information from the preceding frame may be either determined from the preceding frame or received by a decoder from an encoder.
  • the encoder and the decoder both determine the image segments surrounding an uncovered region so that this information does not need to be transmitted.
  • the information about the surrounding segments may be transmitted from the encoder to the decoder since relatively few bits of information may be required to do so. (In the latter embodiment, the decoder may not need a surrounding segment determinator 321.)
  • the encoder and the decoder both determine the starting points so that the starting points do not need to be transmitted. In another embodiment, the locations ofthe starting points may be transmitted from the encoder to the decoder since relatively few bits of information may be required to do so. (In the latter embodiment, the decoder may not need a junction identifier 322.)
  • the encoder and decoder may make identical predictions as to the location ofthe next point on the segment boundary contained within the newly uncovered region. Again, since the encoder and decoder are using the same image, the prediction or guess will be identical.
  • the encoder may also determine corrective information by comparing the predicted filling ofthe uncovered regions with the actual image data in the uncovered regions.
  • the corrective information relates to the difference between the predicted and actual data.
  • the corrective information is transmitted from the encoder to the decoder. This provides the information needed for the decoder to accurately reconstruct the image frame with only a relatively low amount of information needing to be transmitted over the communications channel 1106.
  • the encoder knowing the prediction that the decoder will make, and knowing the correct location ofthe subsequent edge location, can send corrective information to the decoder.
  • the encoder and the decoder may determine every N l extrapolated boundary point, the encoder may send corrective information for every N l point, and the decoder can interpolate the intermediary points.
  • one embodiment according to the present invention provides for an efficient method by which the encoder transmits the high frequency image data ofthe newly uncovered image region to the decoder.
  • the high frequency image data corresponds to sha ⁇ color transitions (fore example, segment edges) within the image data.
  • the encoder could transmit the pixel locations corresponding to the high frequency image data, but that method would be bandwidth expensive. Bandwidth usage can be reduced by assuming that the structure ofthe newly uncovered region will very likely be an extension of surrounding segments. Therefore, a decoder may instead attempt to extend the high frequency information ofthe surrounding segments into the newly uncovered region.
  • the high frequency image information generally corresponds to segment boundaries or edges.

Abstract

One embodiment comprises a method for image processing. The method includes identifying an uncovered region in an image, determining at least one trivalent point, and extrapolating from the trivalent point to extend an edge of at least one of the image segments into the uncovered region. Another embodiment comprises an apparatus for image processing. The apparatus includes a predictor for predicting higher-frequency boundary information in newly uncovered regions. Another embodiment comprises a system for efficiently communicating video information. The system includes an encoder that encodes a video frame into an encoded frame, and a decoder that receives and decodes the encoded frame, wherein the encoder and decoder are both configured to identify uncovered regions and to extend edges from neighboring image segments into the uncovered region.

Description

PREDICTIVE EDGE EXTENSION INTO UNCOVERED REGIONS
Adityo Prakash, Eniko F. Prakash, Edward Ratner, and John Chen
CROSS-REFERENCES TO RELATED APPLICATIONS
This patent application claims priority from U.S. Provisional Patent Application No. 60/274,129, filed March 7, 2001 and entitled "Method and apparatus for raw edge extension," the disclosure of which is incorporated herein by reference.
FIELD OF THE INVENTION
The present invention relates in general to image processing. More particularly, it relates to segment- or object-based image processing.
BACKGROUND OF THE INVENTION It is often desirable to identify and delineate segments or objects in an image.
Information regarding segments or objects in an image may be used to facilitate digital image or video processing.
In certain circumstances, a previously hidden area of an image may become uncovered. For example, in video processing, a frame may include various segments or objects. In a subsequent frame, one or more ofthe segments or objects may move. Where a segment or object used to be (prior to moving), a previously hidden area may become visible.
Processing such newly uncovered areas can be problematic and done inefficiently. For example, when similar areas become newly uncovered in MPEG encoding (due to movement of blocks), MPEG applies discrete cosine transform (DCT) coding to the newly uncovered areas and transmits the entire newly uncovered image region to the decoding device. This approach is disadvantageous in that it results in inefficient image compression.
SUMMARY One embodiment ofthe invention comprises a method for image processing.
The method includes identifying an uncovered region in an image, determining at least one trivalent point, and extrapolating from the trivalent point to extend an edge of at least one of the image segments into the uncovered region. Another embodiment ofthe invention comprises an apparatus for image processing. The apparatus includes a predictor for predicting higher- frequency boundary information in newly uncovered regions.
Another embodiment ofthe invention comprises a system for efficiently communicating video information. The system includes an encoder that encodes a video frame into an encoded frame, and a decoder that receives and decodes the encoded frame, wherein the encoder and decoder are both configured to identify uncovered regions and to extend edges from neighboring image segments into the uncovered region.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is an illustration depicting a previously hidden area becoming visible (i.e. becoming an uncovered region) due to segment motion.
FIG. 2 is a flow chart depicting a method for image processing in accordance with an embodiment ofthe invention. FIG. 3 is a diagram of an apparatus including a predictor for uncovered regions in accordance with an embodiment ofthe invention.
FIG. 4 is an illustration depicting a trivalent point adjacent to an uncovered region in accordance with an embodiment ofthe invention.
FIG. 5 is an illustration depicting predictive extension of an edge in accordance with an embodiment ofthe invention.
FIG. 6 is an illustration depicting the predictive extension of an edge terminating at a boundary ofthe uncovered region in accordance with an embodiment ofthe invention.
FIG. 7 is an illustration depicting multiple trivalent points adjacent to an uncovered region in accordance with an embodiment ofthe invention.
FIG. 8 is an illustration depicting predictive extension of multiple edges in accordance with an embodiment ofthe invention.
FIG. 9 is an illustration depicting the predictive extension of a first edge that terminates at a boundary ofthe uncovered region and of a second edge that terminates at the first edge in accordance with an embodiment ofthe invention.
FIG. 10 is an illustration depicting predictive extension of an edge that skips intermediary points in accordance with an embodiment ofthe invention. FIG. 11 is a diagram of a system for communicating video in accordance with an embodiment ofthe invention.
FIGS. 12A-C depict another example of a trivalent point that may be used as a starting point in accordance with an embodiment ofthe invention.
DETAILED DESCRIPTION FIG. 1 is an illustration depicting a previously hidden or occluded area becoming visible (i.e. becoming an uncovered region) due to segment motion. As shown in FIG. 1, as the vehicle segment(s) moves, three previously hidden areas (101, 102, and 103) become visible. These uncovered regions are: the region 101 behind the back ofthe car; the region 102 behind the rear wheels; and the region 103 behind the front wheels.
FIG. 2 is a flow chart depicting a method for image processing in accordance with an embodiment ofthe invention. The method 200 depicted in FIG. 2 includes six steps (202, 203, 204, 206, 208, and 210). In the first step 202, an uncovered region in a frame is identified. Examples of such uncovered regions are depicted in FIG. 1. Uncovered regions may be considered as part ofthe residue after motion prediction. (Residue information after motion prediction includes information from uncovered regions, information from inexact matches, and newly appearing information.) Segment-based motion prediction and residues are discussed further in International Publication No. WO 00/64167, entitled "Method and Apparatus for Efficient Video Processing," publication date October 26, 2000, the disclosure of which is incorporated herein by reference. Digital image segmentation is discussed further in International Publication No. WO 00/77735 Al, entitled "Method and Apparatus for Digital Image Segmentation," publication date December 21, 2000, the disclosure of which is incorporated herein by reference.
In the second step 203, a determination is made as to the image segments surrounding the uncovered region. In one embodiment, a surrounding segment must share a common boundary with the newly uncovered region. In another embodiment, the common boundary must be of a certain minimum size. In the third step 204, a starting point is determined. In accordance with an embodiment ofthe invention, the starting point comprises a trivalent point. A trivalent point is a point at the intersection of three image "objects." The objects may be either image segments or an uncovered region. In particular, a trivalent point may be at the intersection between two image segments and an uncovered region. A trivalent point is depicted, for example, in FIG. 4 (described below). FIG. 7 (described below) depicts an example of multiple trivalent points adjacent to an uncovered region.
In the fourth step 206, extrapolation from the starting point occurs to extend a segment edge. The extrapolation extends the edge from the starting point into the uncovered region. The extrapolation may be performed using directional information from the edge to be extended. An edge extension from a starting point into an uncovered region is depicted, for example, in FIG. 5 (described below). FIG. 8 (described below) depicts an example of edge extension from multiple starting points into an uncovered region. Finally, in the fifth step 208, the extrapolation continues until the edge extension reaches a termination point. The termination point may be, for example, a point on the boundary ofthe uncovered region. As a result ofthe edge extension until termination, the uncovered region is divided such that each division may be associated with an adjacent image segment. Edge extension until termination, the resulting division of an uncovered region, and the associations ofthe divisions and surrounding segments are depicted, for example, in FIG. 6 (described below). FIG. 9 depicts an example of multiple edge extensions until termination and the resulting division of an uncovered region.
In one embodiment, using the output ofthe method 200, the divisions ofthe uncovered regions may be predictively filled. The predictive filling may be accomplished by applying a predictive filling routine to expand lower- frequency information from adjacent segments into the associated divisions ofthe uncovered region.
FIG. 3 is a diagram of an apparatus including a predictor for uncovered regions in accordance with an embodiment ofthe invention. The predictor 320 may receive as input information or data relating to uncovered regions in an image frame. The predictor 320 may output predicted high-frequency information for the uncovered regions in the frame. The predictor 320 depicted in FIG. 3 includes three components working in cooperation with each other (321, 322, 324).
In one embodiment, the input into the predictor 320 may be received from a hidden area locator 310. The hidden area locator 310 is configured to locate uncovered regions (previously hidden or occluded areas) in a frame. As mentioned above, the uncovered regions may be considered as part ofthe residue after motion prediction. The hidden area locator 302 analyzes the residue after motion prediction and determines which residues correspond to uncovered areas. Other residues (besides those corresponding to uncovered areas) may correspond to inexact matches or newly appearing information. Of course, in other embodiments, the hidden area locator 310 may be integrated with the predictor 320 or may be integrated with another component.
The first component ofthe predictor 320 comprises a surrounding segment determinator 321. The surrounding segment determinator 321 is configured to determine the image segments surrounding the uncovered region. In one embodiment, a surrounding segment must share a common boundary with the newly uncovered region. In another embodiment, the common boundary must be of a certain minimum size.
The second component ofthe predictor 320 comprises a junction identifier 322. Thejunction identifier 322 is configured to determine the starting points. In accordance with an embodiment ofthe invention, the junction identifier 322 locates trivalent points adjacent to the uncovered region and designates those trivalent points as starting points. As mentioned above, a trivalent point is a point at the intersection of three image "objects." The objects may be either image segments or an uncovered region. In particular, a trivalent point may be at the intersection between two image segments and an uncovered region. A trivalent point is depicted, for example, in FIG. 4 (described below). FIG. 7 (described below) depicts an example of multiple trivalent points adjacent to an uncovered region.
The third component comprises an edge extender 324. The edge extender 324 extends an edge by extrapolation from the starting point into the uncovered region. The extrapolation may be performed using directional information from the edge to be extended. An edge extension from a starting point into an uncovered region is depicted, for example, in FIG. 5 (described below). FIG. 8 (described below) depicts an example of edge extension from multiple starting points into an uncovered region.
The edge extender 324 also terminates extension ofthe edges at appropriate termination points. The termination point may be, for example, a point on the boundary of the uncovered region. As a result ofthe edge extension until termination, the uncovered region is divided such that each division may be associated with an adjacent segment. Edge extension until termination and the resulting division of an uncovered region are depicted, for example, in FIG. 6 (described below). FIG. 9 depicts an example of multiple edge extensions until termination and the resulting division of an uncovered region.
In one embodiment, using the output ofthe predictor 320, a segment expander may be configured to apply a predictive filling routine. Such a routine may be used to expand lower-frequency information from the adjacent segments to the associated division of the uncovered region.
FIG. 4 is an illustration depicting a trivalent point adjacent to an uncovered region in accordance with an embodiment ofthe invention. The illustration shows three adjacent image segments 402, 404, and 406. The third segment 406 moves from first image frame to a second image frame. The position ofthe third segment 406 in the second frame is shown by the solid white region labeled 406. Previously hidden areas that are exposed by movement ofthe third segment 406 from the first to second frames is shown by the diagonally-striped areas (408A, 408B, and 408C). These previously hidden areas correspond to uncovered regions. The following discussion focuses on the first uncovered region labeled 408A.
The first uncovered region 408A is adjacent to the first and second image segments 402 and 404. At the intersection ofthe uncovered region 408 A and those two segments 402 and 404 is shown a trivalent point 410. In accordance with an embodiment of the invention, the trivalent point 410 is a starting point for the edge extension.
FIG. 5 is an illustration depicting predictive extension (or extrapolation) of an edge in accordance with an embodiment ofthe invention. The illustration shows the extrapolation 502 beginning at the starting point 410 and extending the edge between the first and second segments 402 and 404 into the uncovered region 408A. The extrapolation 502 may be performed using directional information from previous points ofthe edge.
In accordance with one embodiment, the extrapolation 502 depends on an average directional (tangent) vector from previous edge points. For example, directional vectors from the previous ten (10) edge points may be averaged to determine the average directional vector that indicates the extrapolated location ofthe next edge point. Of course, more or less than ten (10) edge points may be used. In accordance with another embodiment, the extrapolation uses a polynomial predictive algorithm. Polynomial predictive algorithms may be used to extrapolate signals. Here polynomical prediction may be used to extrapolate the edges. Polynomial prediction may involve taking the portion ofthe edge to be extended (that close to the trivalent point) and modeling it by an nth-degree polyomial. Typically, a relatively low-degree (4th degree or less) would be practical. Other algorithms may also be used and are contemplated to be within the scope ofthe invention.
FIG. 6 is an illustration depicting the predictive extension of an edge terminating at a boundary ofthe uncovered region in accordance with an embodiment ofthe invention. The example depicted in FIG. 6 shows the predictive extension 602 continuing from the starting point 410 until it reaches a termination point 604 at a boundary ofthe uncovered region 408A. While the extension 602 is depicted as a straight line in FIG. 6, it need not be such a straight line. Instead, as described above in relation to FIG. 5, the extension may be a non-straight line determined using, for example, an average directional vector or a polynomial predictive algorithm. The completed extension ofthe edge divides the uncovered region 408A into two divisions, one associated with adjacent segment 402 and the other associated with adjacent segment 404.
FIG. 7 is an illustration depicting multiple trivalent points adjacent to an uncovered region in accordance with an embodiment ofthe invention. The example of FIG. 7 differs from the example of FIG. 4 in that there are depicted three image segments (702, 704, and 706) adjacent to the uncovered region 408A. Since there are three adjacent segments, there are now two starting points 708 and 710. In accordance to an embodiment of the invention, the two starting points may each be located at a trivalent point between the uncovered region 408 A and two ofthe segments. In other examples, of course, there may be more than three adjacent segments and hence more than two starting points.
FIG. 8 is an illustration depicting predictive extension (or extrapolation) of multiple edges in accordance with an embodiment ofthe invention. The illustration shows the extrapolations 802 and 804 beginning at the starting points 708 and 710, respectively. The first extrapolation 802 extends the edge between the first and second segments 702 and 704 into the uncovered region 408A. The second extrapolation 804 extends the edge between the second and third segments 704 and 706 into the uncovered region 408A. As described above in relation to FIG. 5, each extrapolation (802 or 804) may be performed using directional information from previous points of its respective edge. FIG. 9 is an illustration depicting the predictive extension of a first edge that terminates at a boundary ofthe uncovered region and of a second edge that terminates at the first edge in accordance with an embodiment ofthe invention. The example depicted in FIG. 6 shows the first and second predictive extensions (902 and 904, respectively), continuing in parallel from the first and second starting points (708 and 710, respectively), until they reach the first and second termination points (906 and 908, respectively).
The first termination point 906 occurs at a boundary ofthe uncovered region 408 A. This first type of termination point is discussed above in relation to FIG. 6. On the other hand, the second termination point 908 is not at a boundary ofthe uncovered region 408A. Instead, the second termination point 908 occurs when the second predictive extension 904 intersects the first predictive extension 902. In other words, the predictive extension ofthe second edge terminates at the first edge. This is a second type of termination point in accordance with an embodiment ofthe invention. The completed extensions ofthe edges divide the uncovered region 408 A into three divisions. A first division is associated with first adjacent segment 702. A second division is associated with the second adjacent segment 704. A third division is associated with the third adjacent segment 706.
Of course, while the extensions 902 and 904 are depicted as straight lines in FIG. 6, they need not be straight lines. Instead, as described above in relation to FIG. 5, the extensions 902 and 904 may be a non-straight line determined using, for example, an average directional vector or a polynomial predictive algorithm.
FIG. 10 is an illustration depicting predictive extension of an edge that skips intermediary points in accordance with an embodiment ofthe invention. As illustrated in FIG. 10, the extrapolation may not predict every point in the extensions 902 and 904 by an extrapolating algorithm. Instead, only certain points such as, for example, every nth point, may be predicted. The "intermediary" points in between the predicted points may be filled in later, for example, by interpolation between predicted points. In other words, the intermediary points are subsequently determined by filling in between the dots ofthe dotted line.
Using the above-described techniques, the predictive filling process may be performed for each surrounding segment until all ofthe segment boundaries have been extended into the newly uncovered image region. Such a process provides a prediction for the image content in the uncovered region. Of course, this prediction may vary from the actual content ofthe uncovered region.
FIG. 11 is a diagram of a system for communicating video in accordance with an embodiment ofthe invention. As depicted, the system 1100 may include an encoder 1102, a decoder 1104, and a communications channel 1106. The communications channel 1106 connects the encoder 1 102 to the decoder 1104 and provides a channel for transmitting information between the two devices.
Generally, an encoder receives a raw, i.e., uncompressed, image frame, manipulates and compresses the image frame, and transmits the compressed image to an decoder. The decoder then decompresses, i.e., reconstructs, the image frame. The encoder manipulates portions of image frames called image segments. Often, as segments change location from frame to frame, there are newly uncovered regions that come into view. The encoder, having the original raw image, knows the complete structure ofthe newly uncovered image regions. However, the decoder likely has no information regarding the image data within the newly uncovered image regions.
As depicted in FIG. 1 1 , the encoder 1102 receives image data and, in accordance with an embodiment ofthe invention, is configured to process the image data using a predictor 320, such as one described above in relation to FIG. 3. The encoder 1102 subsequently transmits encoded image information across the communications channel 1106 to the decoder 1104. The decoder 1104 receives the encoded video information and uses it to reconstruct the image.
In accordance with one embodiment, the decoder 1104 may perform the reconstruction utilizing a same or equivalent predictor 302 algorithm as the encoder 1102 uses. Advantageously in such a system 1100, the encoder 1102 does not need to transmit the predicted filling ofthe uncovered regions. This is because the encoder 1102 and the decoder 1104 are able to use the same or equivalent predictive filling.
FIGS. 12A-C depict another example of a trivalent point that may be used as a starting point in accordance with an embodiment ofthe invention. In this example, FIG. 12A depicts frame N-l, and FIG. 12B depicts frame N, where frame N-l represents a preceding frame in a video sequence and frame N represents the current frame.
Consider, for example, that these figures illustrate a ball rolling from left to right and that the line behind the ball represents a horizon in the image. Consider also that the camera is panning from left to right following the ball's motion. In FIG. 12B, the region to the left ofthe ball is shown as blank because that region is an uncovered region (newly exposed hidden area) in this example. In accordance with an embodiment, the present invention may be applied to such an uncovered region. In particular, the trivalent point 1202 from FIG. 12A (the prior frame) is used as the starting point. FIG. 12C shows the extrapolation 1204 beginning from that point 1202. In such a case, directional information from the edge in FIG. 12A (the edge ending at the starting point 1202) may be used in determining the direction of extrapolation 1204. This directional information from the preceding frame may be either determined from the preceding frame or received by a decoder from an encoder. In one embodiment, the encoder and the decoder both determine the image segments surrounding an uncovered region so that this information does not need to be transmitted. In another embodiment, the information about the surrounding segments may be transmitted from the encoder to the decoder since relatively few bits of information may be required to do so. (In the latter embodiment, the decoder may not need a surrounding segment determinator 321.)
In one embodiment, the encoder and the decoder both determine the starting points so that the starting points do not need to be transmitted. In another embodiment, the locations ofthe starting points may be transmitted from the encoder to the decoder since relatively few bits of information may be required to do so. (In the latter embodiment, the decoder may not need a junction identifier 322.)
Once the starting point has been determined, the encoder and decoder may make identical predictions as to the location ofthe next point on the segment boundary contained within the newly uncovered region. Again, since the encoder and decoder are using the same image, the prediction or guess will be identical.
The encoder may also determine corrective information by comparing the predicted filling ofthe uncovered regions with the actual image data in the uncovered regions. The corrective information relates to the difference between the predicted and actual data. In one embodiment, the corrective information is transmitted from the encoder to the decoder. This provides the information needed for the decoder to accurately reconstruct the image frame with only a relatively low amount of information needing to be transmitted over the communications channel 1106. In other words, the encoder, knowing the prediction that the decoder will make, and knowing the correct location ofthe subsequent edge location, can send corrective information to the decoder. In one embodiment where the encoder and the decoder may determine every Nl extrapolated boundary point, the encoder may send corrective information for every Nl point, and the decoder can interpolate the intermediary points.
Therefore, one embodiment according to the present invention provides for an efficient method by which the encoder transmits the high frequency image data ofthe newly uncovered image region to the decoder. The high frequency image data corresponds to shaφ color transitions (fore example, segment edges) within the image data. The encoder could transmit the pixel locations corresponding to the high frequency image data, but that method would be bandwidth expensive. Bandwidth usage can be reduced by assuming that the structure ofthe newly uncovered region will very likely be an extension of surrounding segments. Therefore, a decoder may instead attempt to extend the high frequency information ofthe surrounding segments into the newly uncovered region. In the preferred embodiment, the high frequency image information generally corresponds to segment boundaries or edges.

Claims

WHAT IS CLAIMED IS:
1. A method for image processing, the method comprising:
locating an uncovered region in an image;
identifying a trivalent point adjacent to the uncovered region;
extrapolating from the trivalent point to extend an edge of edge points into the uncovered region; and
using the extended edge to predictively fill the uncovered region,
wherein prior to extension the edge terminated at the trivalent point.
2. The method of claim 1 , wherein the trivalent point comprises a point which lies at a junction of at least three objects from a group of objects including image segments and the uncovered region.
3. The method of claim 2, wherein the trivalent point comprises a ternary junction previously between three image segments, and wherein at least one ofthe three image segments has moved so that the trivalent point becomes adjacent to the uncovered region.
4. The method of claim 2, wherein the trivalent point is part of a common boundary between two adjacent image segments, and wherein the trivalent point terminates the common boundary at the uncovered region.
5. The method of claim 1, wherein shared boundaries between the uncovered region and each ofthe image segments are at least of a threshold length.
6. The method of claim 1, wherein the extrapolating depends on an average directional vector based on preceding edge points.
7. The method of claim 1 , wherein the extrapolating uses a polynomial predictive routine based on preceding edge points.
8. The method of claim 1 , wherein multiple trivalent points are determined.
9. The method of claim 8, wherein the extrapolating extends multiple edges one at a time, and wherein the extrapolating of each edge terminates if a boundary of the uncovered region is reached.
10. The method of claim 8, wherein the extrapolating extends multiple edges in parallel, and wherein the extrapolating of each edge terminates if another edge is intersected.
11. The method of claim 8, wherein the extrapolating skips intermediary points in extending the edge, and wherein the intermediary points are determined by a form of inteφolation.
12. An apparatus for image processing, the apparatus comprising:
a hidden area locator for locating an uncovered region in an image;
a junction identifier for identifying a starting point adjacent to the uncovered region; and
an edge extender for extrapolating from the starting point to extend an edge of edge points into the uncovered region,
wherein prior to extension the edge terminated at the starting point.
13. The apparatus of claim 12, wherein junction identifier identifies the starting point as a trivalent point at a junction of at least three objects from a group of objects including image segments and uncovered region.
14. The apparatus of claim 13, wherein junction identifier identifies the trivalent point at ternary junction previously between three image segments, and wherein at least one ofthe three image segments has moved so that the trivalent point becomes adjacent to the uncovered region.
15. The apparatus of claim 13, wherein junction identifier identifies the trivalent point from a common boundary between two adjacent image segments, and wherein the trivalent point terminates the common boundary at the uncovered region.
16. The apparatus of claim 12, further comprising:
a boundary length checker which verifies that shared boundaries between the uncovered region and each ofthe image segments are at least of a threshold length.
17. The apparatus of claim 12, wherein the edge extender extrapolates the edge using an average directional vector based on preceding edge points.
18. The apparatus of claim 12, wherein the edge extender extrapolates the edge using a polynomial predictive routine based on preceding edge points.
19. The apparatus of claim 12, wherein the junction identifier identifies multiple starting points adjacent to the uncovered region.
20. The apparatus of claim 19, wherein the edge extender extrapolates multiple edges one at a time, and wherein the extrapolating of each edge terminates if a boundary of the uncovered region is reached.
21. The apparatus of claim 19, wherein the edge extender extrapolates multiple edges in parallel, and wherein the extrapolating of each edge terminates if another edge is intersected.
22. The apparatus of claim 19, wherein the edge extender skips intermediary points in extending the edge, and wherein the intermediary points are determined by a form of interpolation.
23. The apparatus of claim 12, wherein the apparatus comprises an encoder.
24. The apparatus of claim 23, wherein the encoder is configured to transmit compressed video information to a decoder, and wherein the decoder also includes the junction identifier and the edge extender.
25. The apparatus of claim 23, wherein the encoder is configured to transmit locations of starting points for the uncovered region to a decoder, and wherein the decoder includes the edge extender.
26. The apparatus of claim 23, wherein the encoder calculates corrective data between a predicted content ofthe uncovered region and an actual content ofthe uncovered region, and wherein the compressed video information includes the corrective data.
27. The apparatus of claim 12, wherein the apparatus comprises a decoder.
28. The apparatus of claim 27, wherein the decoder is configured to receive compressed video information from an encoder, and wherein the encoder also includes the hidden area locator, the junction identifier, and the edge extender.
29. The apparatus of claim 28, wherein the compressed video information includes corrective data between a predicted content ofthe uncovered region and an actual content ofthe uncovered region, and wherein the decoder applies the corrective information to reconstitute the more accurate content ofthe uncovered region.
30. A system for efficiently communicating video information, the system comprising:
an encoder that encodes a video frame into an encoded frame; and
a decoder that receives and decodes the encoded frame,
wherein the encoder is configured to locate uncovered regions and neighboring image segments in the video frame,
wherein the encoder predictively extends edges into the uncovered region according to a specific method, and
wherein the decoder also predictively extends edges the uncovered region according to the specific method.
31. The system of claim 30, wherein the specific method comprises: determining a starting point; and
extrapolating from the starting point to extend an edge of a neighboring image segment into the uncovered region.
32. The system of claim 31 , wherein the starting point comprises a trivalent point at an intersection between the uncovered region and two neighboring image segments that are adjacent to each other and adjacent to the uncovered region.
33. The system of claim 32, wherein the edge begins at the trivalent point bordering the uncovered region and terminates at the boundary ofthe uncovered region.
34. The system of claim 31 , wherein the extrapolating depends on an average directional vector based on preceding edge points.
35. The system of claim 31 , wherein the extrapolating uses a polynomial predictive routine based on preceding edge points.
PCT/US2002/007220 2001-03-07 2002-03-07 Predictive edge extension into uncovered regions WO2002071331A2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2002255684A AU2002255684A1 (en) 2001-03-07 2002-03-07 Predictive edge extension into uncovered regions

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US27412901P 2001-03-07 2001-03-07
US60/274,129 2001-03-07

Publications (2)

Publication Number Publication Date
WO2002071331A2 true WO2002071331A2 (en) 2002-09-12
WO2002071331A3 WO2002071331A3 (en) 2004-03-18

Family

ID=23046901

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2002/007220 WO2002071331A2 (en) 2001-03-07 2002-03-07 Predictive edge extension into uncovered regions

Country Status (3)

Country Link
US (1) US6898240B2 (en)
AU (1) AU2002255684A1 (en)
WO (1) WO2002071331A2 (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7050503B2 (en) * 1999-04-17 2006-05-23 Pts Corporation Segment-based encoding system using residue coding by basis function coefficients
US7502525B2 (en) * 2003-01-27 2009-03-10 Boston Scientific Scimed, Inc. System and method for edge detection of an image
EP1654705A1 (en) * 2003-08-07 2006-05-10 Koninklijke Philips Electronics N.V. Image object processing
KR101035930B1 (en) * 2007-01-24 2011-05-23 후지쯔 가부시끼가이샤 Image reading device, recording medium having image reading program, and image reading method
KR101366093B1 (en) * 2007-03-28 2014-02-21 삼성전자주식회사 Method and apparatus for video encoding and decoding
KR101142739B1 (en) * 2009-10-29 2012-05-03 삼성전자주식회사 Apparatus and method for inpainting image by restricting reference image region

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4689671A (en) * 1985-06-27 1987-08-25 Nec Corporation Coding apparatus for moving object image
JPH09214899A (en) * 1996-01-31 1997-08-15 Nippon Hoso Kyokai <Nhk> Image prediction processing method an its device

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100186979B1 (en) * 1994-12-28 1999-05-01 김주용 Image data encoding method for uncovered background of object
US5917943A (en) * 1995-03-31 1999-06-29 Canon Kabushiki Kaisha Image processing apparatus and method
US6453069B1 (en) * 1996-11-20 2002-09-17 Canon Kabushiki Kaisha Method of extracting image from input image using reference image
US6600786B1 (en) * 1999-04-17 2003-07-29 Pulsent Corporation Method and apparatus for efficient video processing
DE60003032T2 (en) 1999-06-11 2004-04-01 Pulsent Corp., Milpitas METHOD FOR IMAGE SEGMENTATION
IL146978A0 (en) * 1999-06-11 2002-08-14 Pulsent Corp Method and apparatus for digital image segmentation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4689671A (en) * 1985-06-27 1987-08-25 Nec Corporation Coding apparatus for moving object image
JPH09214899A (en) * 1996-01-31 1997-08-15 Nippon Hoso Kyokai <Nhk> Image prediction processing method an its device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
GERKEN P: "OBJECT-BASED ANALYSIS-SYNTHESIS CODING OF IMAGE SEQUENCES AT VERY LOW BIT RATES" IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, IEEE INC. NEW YORK, US, vol. 4, no. 3, 1 June 1994 (1994-06-01), pages 228-235, XP000460755 ISSN: 1051-8215 *
KAUP A ET AL: "Efficient prediction of uncovered background in interframe coding using spatial extrapolation" ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 1994. ICASSP-94., 1994 IEEE INTERNATIONAL CONFERENCE ON ADELAIDE, SA, AUSTRALIA 19-22 APRIL 1994, NEW YORK, NY, USA,IEEE, 19 April 1994 (1994-04-19), pages V-501-V-504, XP010133695 ISBN: 0-7803-1775-0 *
RAMBARUTH R ET AL: "Interframe texture coding of uncovered regions for a region-based scheme" ELECTRONICS LETTERS, IEE STEVENAGE, GB, vol. 34, no. 17, 20 August 1998 (1998-08-20), pages 1655-1657, XP006010179 ISSN: 0013-5194 *

Also Published As

Publication number Publication date
US6898240B2 (en) 2005-05-24
WO2002071331A3 (en) 2004-03-18
AU2002255684A1 (en) 2002-09-19
US20020131639A1 (en) 2002-09-19

Similar Documents

Publication Publication Date Title
EP1057341B1 (en) Motion vector extrapolation for transcoding video sequences
US6600786B1 (en) Method and apparatus for efficient video processing
US4703350A (en) Method and apparatus for efficiently communicating image sequences
US6324214B2 (en) Method and apparatus for controlling a target amount of code and for compressing video data
US5760846A (en) Apparatus for estimating motion vectors for feature points of a video signal
US4849810A (en) Hierarchial encoding method and apparatus for efficiently communicating image sequences
KR100478558B1 (en) Outline Coding Method Using Error Strips
JP5130381B2 (en) Method and apparatus for efficient video processing
US5835147A (en) Method for compression and decompression of a video signal
US6898240B2 (en) Predictive edge extension into uncovered regions
US7305033B2 (en) Method of encoding and decoding motion picture, motion picture encoding device and motion picture decoding device
US6480544B1 (en) Encoding apparatus and encoding method
KR100196874B1 (en) Apparatus for selectively approximating contour of image
US6885706B2 (en) Method for promoting temporal resolution of sequential images
JPH0244883A (en) Moving picture transmission system
KR0176079B1 (en) Method of dividing variable boundarys in a parallel high definition tv decoder
WO1998049830A3 (en) Method of estimation of motion between images
KR100223170B1 (en) The optimization method and apparatus for region partition and motion estimation in moving image encoder
WO2006137020A2 (en) Video signal compression
JPH07274175A (en) Method and apparatus for coding moving picture of low transmission rate by dynamic movement evaluation
Servais et al. Affine motion compensation using a content-based mesh
KR100388802B1 (en) apparatus and method for concealing error
Ostermann et al. Coding of color parameters in an analysis-synthesis coder based on moving 3D-objects
KR0180161B1 (en) Apparatus and method for processing a motion vector
KR960028487A (en) Shape Information Reduction Method Using Motion Estimation

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP