US20060002463A1 - Method, apparatus and program products for retrieving moving image - Google Patents
Method, apparatus and program products for retrieving moving image Download PDFInfo
- Publication number
- US20060002463A1 US20060002463A1 US10/866,164 US86616404A US2006002463A1 US 20060002463 A1 US20060002463 A1 US 20060002463A1 US 86616404 A US86616404 A US 86616404A US 2006002463 A1 US2006002463 A1 US 2006002463A1
- Authority
- US
- United States
- Prior art keywords
- moving image
- information
- retrieving
- frame
- feature quantity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
- G06F16/785—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content using colour or luminescence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
- G06F16/786—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content using motion, e.g. object motion or camera motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/7847—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content
- G06F16/7864—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using low-level visual features of the video content using domain-transform features, e.g. DCT or wavelet transform coefficients
Definitions
- This invention relates to a method and an apparatus for retrieving moving image, which retrieve and identify fixed moving image from some moving image. And more particularly to, a method and an apparatus for retrieving moving image, in which the information of the unique identification of moving image which depends on the content of the moving image is extracted and registered, the registered moving image is retrieved and identified earlier than video rate from moving image such as moving image archives sent in broadcasting and the like.
- the conventional method for retrieving moving image there are several methods for retrieving moving image, for example a method using the keyword retrieval, a method using the visual discrimination by the visual observation of the user, and a method using the number of continuous frames on the shot of moving image disclosed in Japanese Patent Publication No. JP7-retrieving moving image by detecting the cut point of the moving image.
- this apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publication No. JP7-236153 the time variation of luminance difference between frames of the reduction image which is made from prepared reduction image and the mean value component data of the information of compressed and encoded moving image and the time variation of color difference histogram correlation value are used in order to obtain a cut point at high speed from moving image.
- the low frequency part of DCT Discrete Cosine Transform
- index information for example a cut point frame, a motionless frame, a character frame, a specific color frame, etc.
- an apparatus for retrieving moving image to retrieve pre-determined moving image from object moving image comprises first extracting means for extracting the information of the feature quantity of each frame from the pre-determined moving image, second extracting means for extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by the first extracting means, and retrieving means for retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the second extracting means, with the information of the unique identification prepared in accordance with the object moving image.
- a method for retrieving moving image to retrieve pre-determined moving image from object moving image comprises the steps of (A) extracting the information of the feature quantity of each frame from the pre-determined moving image, (B) extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by said step (A), and (C) retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the step (B), with the information of the unique identification prepared in accordance with the object moving image.
- a program for retrieving moving image to retrieve predetermined moving image from object moving image comprises the steps of (A) extracting the information of the feature quantity of each frame from the pre-determined moving image, (B) extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by the step (A), and (C) retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the step (B), with the information of the unique identification prepared in accordance with the object moving image.
- FIG. 1 is a block diagram showing the apparatus for retrieving moving image according to the present invention
- FIG. 2A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 2B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 3 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 4A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 4B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 5 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 6A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 6B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 7 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 8A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 8B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 9 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 10A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 10B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 11 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 12A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 12B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 13 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 14 is a conceptual view showing the process of DCT (discrete cosine transform) operation in MPEG;
- FIG. 15A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 15B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 16 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 17A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 17B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 18 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 19A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 19B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 20 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 21A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 21B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 22 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 23A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 23B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 24 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 25A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 25B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 26 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 27A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 27B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 28 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 29A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 29B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 30 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 31A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 31B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 32 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 33A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 33B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 34 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 35A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 35B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 36 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 37A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 37B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 38 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 39A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 39B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 40 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 41A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 41B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 42 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 43A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 43B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 44 is a block diagram showing the apparatus for retrieving moving image according to the present invention.
- FIG. 45A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 45B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIG. 46 is a block diagram showing the apparatus for monitoring television commercials according to the present invention.
- FIG. 47A is a flowchart of the process in the apparatus for monitoring television commercials according to the present invention.
- FIG. 47B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention.
- FIGS. 1, 2A and 2 B An apparatus and a method for retrieving moving image in the first preferred embodiment according to the invention will be explained in FIGS. 1, 2A and 2 B.
- FIG. 1 is a block diagram showing an apparatus for retrieving moving image according to the present invention.
- the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of the feature quantity of moving image, which includes physical information of the feature quantity of moving image such as information of luminance and color difference (Y, Cb, Cr) of each frame and/or information of color (R, G, B) of each frame, from moving image data 101 , an extracting unit 104 for arranging at least one physical information included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 105 of the unique identification of moving image to identify the moving image from the arrangement of the physical information on the time axis, a storing unit 106 for storing the information 105 of the unique identification of the moving image extracted by the extracting unit 104 , a retrieving unit 109 for retrieving
- FIGS. 2A and 2B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 1 .
- the extracting unit 102 for extracting the information of the feature quantity of the moving image fetches the moving image data 101 to be a retrieval key for identifying the moving image (in the step 201 ).
- the extracting unit 102 extracts the information 103 of the feature quantity of the moving image, which includes the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame, from the moving image data 101 (in the step 202 ).
- the extracting unit 104 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on the information 103 , mean value of the values, total value of the values and/or difference value of the values on the time axis. Then, the extracting unit 104 picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 104 extracts the picked out values as the information 105 of the unique identification of the moving image to be used for identifying the moving image (in the step 203 ).
- the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R
- the storing unit 106 for storing the information of the unique identification of the moving image stores the information 105 of the unique identification of the moving image extracted by the extracting unit 104 (in the step 204 ).
- the information 108 of the unique identification of the moving image is extracted from other moving image data (the object moving image data for retrieving) (in the step 205 ).
- This extracting process can be executed by the processing equal to the foregoing steps 201 to 204 .
- the retrieving unit 109 for retrieving the information of the unique identification of the moving image compares the information 105 of the unique identification of the moving image stored in the storing unit 106 with the information 108 of the unique identification of the moving image extracted from the other moving image data (in the step 206 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the above mentioned extracting unit 104 arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and the information of the color (R, G, B) of each frame) based on the information 103 of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values on the time axis, and picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals, and then extracts the picked out values as the information 105 of the unique identification of the moving image to be used for identifying the moving image.
- the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and the information of the color (R, G, B) of each frame
- the above mentioned extracting unit 104 may arrange at least one value from values of a part or multiple parts of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and the information of the color (R, G, B) of each frame) based on the information 103 of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values on the time axis, and pick out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals, and then extract the picked out values as the information 105 of the unique identification of the moving image to be used for identifying the moving image.
- the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and the information of the color (R, G, B) of each frame
- values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and the information 105 of the unique identification of the moving image may be extracted by using all of the above mentioned values of the information of the feature quantity of the moving image. And also, the information 105 of the unique identification of the moving image may be extracted by using some values of the information of the feature quantity of the moving image.
- values of the information of the feature quantity of the moving image which are calculated from blocks that are located on the part except for the part of time display and the part of around frame (the part with unevenness), are used.
- the extracting unit 104 may calculate values after executing the weighting on the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on the information 103 , then may apply the values, mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 104 may calculate values of the physical information of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values, and then may quantize the calculated values.
- information of the unique identification of moving image for retrieving moving image depends on the content of the moving image can be extracted, consequently, identical moving image can be retrieved and identified from the moving image by using the information of the unique identification of the moving image at high speed.
- FIGS. 3, 4A and 4 B An apparatus and a method for retrieving moving image in the second preferred embodiment according to the invention will be explained in FIGS. 3, 4A and 4 B.
- FIG. 3 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 3 with FIG. 1 .
- the apparatus for retrieving moving image comprises an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are included in the information 303 , on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304
- DCT
- FIGS. 4A and 4B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 3 . Still, an identical step number is appended to the step having the identical process in FIGS. 4A and 4B with FIGS. 2A and 2B .
- the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ).
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402 ).
- the extracting unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 401 to 404 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the above mentioned extracting unit 304 calculates values of the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values, and values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on the whole frame from the information of the motion compensation. And the above mentioned extracting unit 304 arranges at least one value from the above mentioned values and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals.
- the above mentioned extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image.
- the above mentioned extracting unit 304 may calculate values of a part or multiple parts of the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values, and values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on the whole frame from a part or multiple parts of the information of the motion compensation.
- values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and the information 305 of the unique identification of the moving image may be extracted by using all of the above mentioned values of the information of the feature quantity of the moving image. And also, the information 305 of the unique identification of the moving image may be extracted by using some values of the information of the feature quantity of the moving image.
- values of the information of the feature quantity of the moving image which are calculated from blocks that are located on the part except for the part of time display and the part of around frame (the part with unevenness), are used.
- the extracting unit 304 may execute the weighting on the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , then may calculate values from the weighted information of the feature quantity of the compressed moving image, mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 304 may execute the weighting on the information of the motion compensation and may calculate a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the weighted information of the motion compensation.
- values provided by quantizing the above mentioned values of the physical information of the feature quantity of the compressed moving image and the like, and values provided by quantizing the above mentioned motion vectors, and the like may be applied.
- information of the unique identification of moving image for retrieving moving image depends on the content of the compressed moving image provided by the standard compression encoding process for moving image such as MPEG can be extracted, consequently, identical moving image can be retrieved and identified from the moving image by using the information of the unique identification of the moving image at high speed.
- FIGS. 5, 6A and 6 B An apparatus and a method for retrieving moving image in the third preferred embodiment according to the invention will be explained in FIGS. 5, 6A and 6 B.
- FIG. 5 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 5 with FIG. 1 .
- the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of the feature quantity of moving image, which includes physical information of the feature quantity of moving image such as information of luminance and color difference (Y, Cb, Cr) of each frame and/or information of color (R, G, B) of each frame, from moving image data 101 , a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 503 for arranging at least one physical information included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 505 of the unique identification of moving image to identify the moving image in
- FIGS. 6A and 6B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 5 . Still, an identical step number is appended to the step having the identical process in FIGS. 6A and 6B with FIGS. 2A and 2B .
- the extracting unit 102 for extracting the information of the feature quantity of the moving image and the detecting unit 501 for detecting the cut point fetch the moving image data 101 to be a retrieval key for identifying the moving image (in the step 601 ).
- the extracting unit 102 extracts the information 103 of the feature quantity of the moving image, which includes the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame, from the moving image data 101 (in the step 202 ).
- the detecting unit 501 detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- This cut point of the moving image data 101 shows a point where the scene of the moving image data 101 changes or a point where the camera for photographing the moving image data 101 switches.
- the extracting unit 503 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on the information 103 , mean value of the values, total value of the values and/or difference value of the values on the time axis. Then, the extracting unit 503 picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals.
- the extracting unit 503 extracts the picked out values in each shot unit of the moving image data 101 divided by the information 502 of the cut point as the information 505 of the unique identification of the moving image to be used for identifying the moving image (in the step 603 ).
- the storing unit 106 for storing the information of the unique identification of the moving image stores the information 505 of the unique identification of the moving image extracted by the extracting unit 503 (in the step 604 ).
- the information 508 of the unique identification of the moving image is extracted from other moving image data (the object moving image data for retrieving) (in the step 205 ).
- This extracting process can be executed by the processing equal to the foregoing steps 601 to 604 .
- the retrieving unit 109 for retrieving the information of the unique identification of the moving image compares the information 505 of the unique identification of the moving image stored in the storing unit 106 with the information 508 of the unique identification of the moving image extracted from the other moving image data (in the step 206 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the above mentioned extracting unit 503 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on the information 103 of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values on the time axis.
- the extracting unit 503 may arranges at least one value from values of a part or multiple parts of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on the information 103 of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values on the time axis. That is to say, values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and the information 505 of the unique identification of the moving image may be extracted by using all of the above mentioned values of the information of the feature quantity of the moving image.
- the information 505 of the unique identification of the moving image may be extracted by using some values of the information of the feature quantity of the moving image.
- values of the information of the feature quantity of the moving image which are calculated from blocks that are located on the part except for the part of time display and the part of around frame (the part with unevenness), are used.
- the extracting unit 503 may calculate values after executing the weighting on the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on the information 103 of the feature quantity of the moving image, then may apply the values, mean value of the values, total value of the values and/or difference value of the values.
- the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame
- the extracting unit 503 may calculate values of the physical information of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values, and then may quantize the calculated values.
- the information 505 of the unique identification of the moving image for retrieving the moving image depends on the content of the moving image can be extracted by the less information quantity with utilizing the time concept of a shot according to the information 502 of the cut point. And, identical moving image can be retrieved and identified from the moving image by using the information 505 of the unique identification of the moving image at the higher speed.
- FIGS. 7, 8A and 8 B An apparatus and a method for retrieving moving image in the fourth preferred embodiment according to the invention will be explained in FIGS. 7, 8A and 8 B.
- FIG. 7 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 7 with FIG. 3 .
- the apparatus for retrieving moving image comprises an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are included in the information 303 , on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in
- FIGS. 8A and 8B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 7 . Still, an identical step number is appended to the step having the identical process in FIGS. 8A and 8B with FIGS. 4A and 4B .
- the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 801 ).
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402 ). And, the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802 ).
- the extracting unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 703 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 703 extracts the picked out values in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 803 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 801 to 804 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the above mentioned extracting unit 703 calculates values of the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values, and values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 703 may calculate values of a part or multiple parts of the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values, and values/such as motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on the whole frame from a part or multiple parts of the information of the motion compensation.
- values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and the information 705 of the unique identification of the moving image may be extracted by using all of the above mentioned values of the information of the feature quantity of the moving image. And also, the information 705 of the unique identification of the moving image may be extracted by using some values of the information of the feature quantity of the moving image.
- values of the information of the feature quantity of the moving image which are calculated from blocks that are located on the part except for the part of time display and the part of around frame (the part with unevenness), are used.
- the extracting unit 703 may execute the weighting on the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , then may calculate the above mentioned values from the weighted information of the feature quantity of the compressed moving image.
- the extracting unit 703 may execute the weighting on the information of the motion compensation and may calculate the above mentioned motion vectors from the weighted information of the motion compensation.
- values provided by quantizing the above mentioned values of the physical information of the feature quantity of the compressed moving image and values provided by quantizing the above mentioned motion vectors, and the like may be applied.
- the information 705 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the compressed moving image provided by the standard compression encoding process for moving image such as MPEG can be extracted by the less information quantity with utilizing the time concept of the shot according to the information 702 of the cut point. And, identical moving image can be retrieved and identified from the moving image by using the information 705 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 9, 10A and 10 B An apparatus and a method for retrieving moving image in the fifth preferred embodiment according to the invention will be explained in FIGS. 9, 10A and 10 B.
- FIG. 9 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 9 with FIG. 1 and FIG. 3 .
- the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 , a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels unit of the moving image data 101 , an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation
- FIGS. 10 A and 10 B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 9 . Still, an identical step number is appended to the step having the identical process in FIGS. 10A and 10B with FIGS. 4A and 4B .
- 10A and 10 B at first whether inputted moving image data is compressed is judged. (in the step 1001 ).
- the extracting unit 901 for extracting the information of the motion compensation and the DCT operation unit 902 fetch the moving image data 101 to be a retrieval key for identifying the moving image (in the step 1002 ).
- the extracting unit 901 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M ⁇ N pixels unit (in the step 1003 ).
- the DCT operation unit 902 divides each frame of the moving image data 101 into the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M′ ⁇ N′ pixels unit (in the step 1004 ).
- the information of the motion compensation in the M ⁇ N pixels unit provided by the extracting unit 901 and the DCT coefficient in the M′ ⁇ N′ pixels unit provided by the DCT operation unit 902 are sent to the extracting unit 304 for extracting the information of the unique identification of the compressed moving image as the information 303 of the feature quantity of the compressed moving image.
- the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ). And the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402 ).
- the extracting unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 extracted by extracting unit 302 , or the extracting unit 901 and the DCT operation unit 902 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the M ⁇ N pixels unit and the M′ ⁇ N′ pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- the information 305 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the moving image 101 and the compressed moving image 301 can be extracted by the equal processing (the step 403 ). And identical moving image can be retrieved and identified from the moving image by using the information 305 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 11, 12A and 12 B An apparatus and a method for retrieving moving image in the sixth preferred embodiment according to the invention will be explained in FIGS. 11, 12A and 12 B.
- FIG. 11 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 11 with FIG. 5 , FIG. 7 and FIG. 9 .
- the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M x N (M and N are natural numbers) pixels unit of moving image data 101 , a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels unit of the moving image data 101 , a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such
- FIGS. 12A and 12B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 11 . Still, an identical step number is appended to the step having the identical process in FIGS. 12A and 12B with FIGS. 6A, 6B , 8 A, 8 B, 10 A and 10 B.
- FIGS. 11, 12A and 12 B at first whether inputted moving image data is compressed is judged (in the step 1001 ).
- the extracting unit 901 for extracting the information of the motion compensation, the DCT operation unit 902 and the detecting unit 501 for detecting the cut point fetch the moving image data 101 to be a retrieval key for identifying the moving image (in the step 1201 ).
- the extracting unit 901 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M ⁇ N pixels unit (in the step 1003 ).
- the DCT operation unit 902 divides each frame of the moving image data 101 into the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M′ ⁇ N′ pixels unit (in the step 1004 ).
- the detecting unit 501 detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- the information of the motion compensation in the M ⁇ N pixels unit provided by the extracting unit 901 and the DCT coefficient in the M′ ⁇ N′ pixels unit provided by the DCT operation unit 902 are sent to the extracting unit 703 for extracting the information of the unique identification of the compressed moving image with the information 502 of the cut point detected by the detecting unit 501 as the information 303 of the feature quantity of the compressed moving image.
- the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 801 ).
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step, 402 ).
- the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802 ).
- the extracting unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 703 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 703 extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 or in each shot unit of the compressed moving image data 301 divided by using the information 702 as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 803 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the M ⁇ N pixels unit and the M′ ⁇ N′ pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- the information 705 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the moving image 101 and the compressed moving image 301 can be extracted by the equal processing (the step 803 ). And, the information 705 of the unique identification of the compressed moving image for retrieving moving image can be extracted by the less information quantity with utilizing the time concept of the shot according to the information 502 and the information 702 . Identical moving image can be retrieved and identified from the moving image by using the information 705 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 13 to 15 B An apparatus and a method for retrieving moving image in the seventh preferred embodiment according to the invention will be explained in FIGS. 13 to 15 B.
- FIG. 13 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 13 with FIG. 3 and FIG. 9 .
- the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 , an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, Cr), in the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 1304 for arranging at
- FIG. 14 is a conceptual scheme showing the relationship between the DCT (Discrete Cosine Transform) operation and the IDCT (Inverse Discrete Cosine Transform) operation, which are used in the international standard encoding system such as MPEG, and frame image.
- DCT Discrete Cosine Transform
- IDCT Inverse Discrete Cosine Transform
- N is showing a block size of intra-code
- f (x,y) is showing a pixel value of the coordinate (x, y) in a block
- F (u, v) is showing a DCT coefficient value of the coordinate (u, v) after the DCT conversion
- F (u, v) in the EXPRESSION 1 is shown in the following EXPRESSION 3.
- the DC component F (u, v) of the DCT coefficient becomes a value provided by multiplying the average pixel value in the block by the quantization number (N) of the block. That is to say, the DC components F (u, v) of the DCT coefficients extracted by the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image and the average pixel value provided by the extracting unit 1301 for extracting the average value of the block unit may be handled as identical information data considering the quantization number of the block.
- the information 1303 extracted by the extracting unit 1302 and the information 1303 depends on the information of the motion compensation extracted by the extracting unit 901 and the average pixel value extracted by the extracting unit 1301 may be considered information of data having an identical content composition.
- FIGS. 15A and 15B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 13 . Still, an identical step number is appended to the step having the identical process in FIGS. 15A and 15B with FIGS. 4A, 4B , 10 A and 10 B.
- FIGS. 13, 15A and 15 B at first whether inputted moving image data is compressed is judged. (in the step 1001 ).
- the extracting unit 901 for extracting the information of the motion compensation and the extracting unit 1301 for extracting the average value of the block unit fetch the moving image data 101 to be a retrieval key for identifying the moving image (in the step 1501 ).
- the extracting unit 901 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M ⁇ N pixels unit (in the step 1003 ).
- the extracting unit 1301 divides each frame of the moving image data 101 into the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels units, then calculates the average pixel value in the M′ ⁇ N′ pixels unit (in the step 1502 ).
- the information of the motion compensation in the M ⁇ N pixels unit provided by the extracting unit 901 and the average pixel value in the M′ ⁇ N′ pixels unit provided by the extracting unit 1301 are sent to the extracting unit 1304 for extracting the information of the unique identification of the compressed moving image as the information 1303 of the feature quantity of the compressed moving image.
- the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ). And the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 1503 ).
- the extracting unit 1304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 1304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 1304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1504 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the above mentioned extracting unit 1304 calculates values of physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/or difference value of the values. And the extracting unit 1304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 1304 may calculate the above mentioned values by using a part or multiple parts of the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or the average pixel value of each frame, which are included in the information 1303 , and the above mentioned vectors by using a part or multiple parts of the information of the motion compensation.
- the extracting unit 1304 may execute the weighting on the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or on the average pixel value of each frame, then may calculate the above mentioned values from the weighted information of the feature quantity of the compressed moving image and/or the weighted average pixel value.
- the extracting unit 1304 may execute the weighting on the information of the motion compensation and may calculate the above mentioned vectors from the weighted information of the motion compensation.
- values provided by quantizing the above mentioned values of the physical information or of the average pixel value or the like, and values provided by quantizing the above mentioned motion vectors and the like may be applied.
- the M ⁇ N pixels unit and the M′ ⁇ N′ pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- the information 305 of the unique identification of the compressed moving image for retrieving the moving image depends on the content of the moving image data 101 and the compressed moving image data 301 can be extracted by the less information quantity and by executing the equal process for the moving image data 101 and the compressed moving image data 301 . And, identical moving image can be retrieved and identified from the moving image by using the information 305 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 16, 17A and 17 B An apparatus and a method for retrieving moving image in the eighth preferred embodiment according to the invention will be explained in FIGS. 16, 17A and 17 B.
- FIG. 16 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 16 with FIG. 5 , FIG. 7 and FIG. 13 .
- the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M x N (M and N are natural numbers) pixels unit of moving image data 101 , an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, Cr), in the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels unit, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each
- DC
- FIGS. 17A and 17B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 16 . Still, an identical step number is appended to the step having the identical process in FIGS. 17A and 17B with FIGS. 6A, 6B , 8 A, 8 B, 15 A and 15 B.
- FIGS. 16, 17A and 17 B at first whether inputted moving image data is compressed is judged (in the step 1001 ).
- the extracting unit 901 for extracting the information of the motion compensation
- the extracting unit 1301 for extracting the average value of the block unit
- the detecting unit 501 for detecting the cut point fetch the moving image data 101 to be a retrieval key for identifying the moving image (in the step 1701 ).
- the extracting unit 901 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M ⁇ N pixels unit (in the step 1003 ). And, the extracting unit 1301 divides each frame of the moving image data 101 into the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels units, then calculates the average pixel value in the M′ ⁇ N′ pixels unit (in the step 1502 ). And more, the detecting unit 501 detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- the information of the motion compensation in the M ⁇ N pixels unit provided by the extracting unit 901 and the average pixel value in the M′ ⁇ N′ pixels unit provided by the extracting unit 1301 are sent to the extracting unit 1603 for extracting the information of the unique identification of the compressed moving image with the information 502 of the cut point detected by the detecting unit 501 as the information 1303 of the feature quantity of the compressed moving image.
- the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 801 ).
- the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 1503 ).
- the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information of the cut point 702 (in the step 802 ).
- the extracting unit 1603 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 1603 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 1603 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1603 extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the M ⁇ N pixels unit and the M′ ⁇ N′ pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- information 1303 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the moving image data 101 and the compressed moving image data 301 can be extracted by the less information quantity and by executing the equal process for the moving image data 101 and the compressed moving image data 301 .
- identical moving image can be retrieved and identified from the moving image by using the information 1303 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 18, 19A and 19 B An apparatus and a method for retrieving moving image in the ninth preferred embodiment according to the invention will be explained in FIGS. 18, 19A and 19 B.
- FIG. 18 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 18 with FIG. 3 and FIG. 9 .
- the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 as information 303 of the feature quantity of compressed moving image, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes the DCT coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or by the extracting unit 302 , on the time axis and for extracting information 305 of the
- FIGS. 19A and 19B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 18 . Still, an identical step number is appended to the step having the identical process in FIGS. 19A and 19B with FIGS. 10A and 10B .
- FIGS. 18, 19A and 19 B at first whether inputted moving image data is compressed is judged. (in the step 1001 ).
- the DCT operation unit 902 fetches the moving image data 101 to be a retrieval key for identifying the moving image (in the step 1901 ).
- the DCT operation unit 902 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M ⁇ N pixels unit as the information 303 of the feature quantity of the compressed moving image (in the step 1902 ).
- the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ). And the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed moving image data 301 as (in the step 1903 ).
- the extracting unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 extracted by the extracting unit 902 , mean value of the values, total value of the values and/or difference value of the values. And the extracting unit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1904 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the M ⁇ N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- the information 305 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the moving image data 101 and the compressed moving image data 301 can be extracted by executing the equal process. And, identical moving image can be retrieved and identified from the moving image by using the information 305 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 20, 21A and 21 B An apparatus and a method for retrieving moving image in the tenth preferred embodiment according to the invention will be explained in FIGS. 20, 21A and 21 B.
- FIG. 20 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 20 with FIG. 11 .
- the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient as information 303 of the feature quantity of compressed moving image by executing a DCT operation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 , a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at
- FIGS. 21A and 21B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 20 . Still, an identical step number is appended to the step having the identical process in FIGS. 21A and 21B with FIGS. 12A, 12B , 19 A and 19 B.
- FIGS. 20, 21A and 21 B at first whether inputted moving image data is compressed is judged. (in the step 1001 ).
- the DCT operation unit 902 and the detecting unit 501 for detecting the cut point fetch the moving image data 101 to be a retrieval key for identifying the moving image (in the step 2101 ).
- the DCT operation unit 902 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M ⁇ N pixels unit as the information 303 of the feature quantity of the compressed moving image (in the step 1902 ).
- the detecting unit 501 for detecting the cut point detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- the DCT coefficient in the M ⁇ N pixels unit provides by the DCT operation unit 902 is sent to the extracting unit 703 for extracting the information of the unique identification of the compressed moving image with the information 502 detected by the detecting unit 501 as the information 303 of the feature quantity of the compressed moving image.
- the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 801 ).
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1903 ).
- the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802 ).
- the extracting unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values. And the extracting unit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals.
- the extracting unit 703 extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 or in each shot unit of the compressed moving image data 301 divided by using the information 702 as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 2102 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the M ⁇ N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- the information 705 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the moving image 101 and the compressed moving image 301 can be extracted by the equal processing. And, the information 705 for retrieving moving image can be extracted by the less information quantity with utilizing the time concept of the shot according to the information 502 and the information 702 . Identical moving image can be retrieved and identified from the moving image by using the information 705 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 22, 23A and 23 B An apparatus and a method for retrieving moving image in the eleventh preferred embodiment according to the invention will be explained in FIGS. 22, 23A and 23 B.
- FIG. 22 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 22 with FIG. 13 .
- the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, Cr), in the M ⁇ N (M and N are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 1304 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame or the average pixel value, which is included in the information 1303 extracted by the extracting unit 1301 and/or the extracting unit 1302
- FIGS. 23A and 23B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 22 . Still, an identical step number is appended to the step having the identical process in FIGS. 23A and 23B with FIGS. 15A and 15B .
- FIGS. 22, 23A and 23 B at first whether inputted moving image data is compressed is judged. (in the step 1001 ).
- the extracting unit 1301 for extracting the average value of the block unit fetches the moving image data 101 to be a retrieval key for identifying the moving image (in the step 2301 ).
- the extracting unit 1301 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the average pixel value in the M ⁇ N pixels unit as the information 1303 of the feature quantity of the compressed moving image (in the step 1502 ).
- the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 401 ). Then, the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1503 ).
- the extracting unit 1304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and i or difference value of the values. And the extracting unit 1304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1504 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or sprinting the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the above mentioned extracting unit 1304 calculates values of the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame or values of the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/ or difference value of the values.
- the extracting unit 1304 may calculate the above mentioned values by using a part or multiple parts of the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame or the average pixel value of each frame, which are included in the information 1303 .
- the extracting unit 1304 may execute the weighting on the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame or the average pixel value of each frame, which are included in the information 1303 , then may calculate the above mentioned values by the weighted information of the feature quantity of the compressed moving image or the weighted average pixel value.
- values provided by quantizing the above mentioned physical information or average pixel value or the like may be applied.
- the M ⁇ N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- the information 305 of the unique identification of the compressed moving image for retrieving the moving image depends on the content of the moving image data 101 and the compressed moving image data 301 can be extracted by the less information quantity and by executing the equal process for the moving image data 101 and the compressed moving image data 301 . And, identical moving image can be retrieved and identified from the moving image by using the information 305 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 24, 25A and 25 B An apparatus and a method for retrieving moving image in the twelfth preferred embodiment according to the invention will be explained in FIGS. 24, 25A and 25 B.
- FIG. 24 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 24 with FIG. 16 .
- the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, Cr), in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 as information 1303 of the feature quantity of compressed moving image, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point
- DC
- FIGS. 25A and 25B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 24 . Still, an identical step number is appended to the step having the identical process in FIGS. 25A and 25B with FIGS. 17A and 17B .
- 25A and 25 B at first whether inputted moving image data is compressed is judged (in the step 1001 ).
- the extracting unit 1301 for extracting the average value of the block unit and the detecting unit 501 for detecting the cut point fetch the moving image data 101 to be a retrieval key for identifying the moving image (in the step 2501 ).
- the extracting unit 1301 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the average pixel value in the M ⁇ N pixels unit (in the step 1502 ). And, the detecting unit 501 detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point fetch the compressed moving image data 301 to be a retrieval key for identifying the compressed moving image (in the step 801 ). Then, the extracting unit 1302 extracts the information 1303 .of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1503 ). And, the detecting unit 701 detests the cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802 ).
- the extracting unit 1603 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/or difference value of the values. And the extracting unit 1603 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals.
- the extracting unit 1603 extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804 .
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the apparatus and the method for retrieving the moving image according to the invention are explained the above.
- the M ⁇ N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- the information 1303 of the unique identification of the compressed moving image for retrieving the moving image depends on the content of the moving image data 101 and the compressed moving image data 301 can be extracted by the less information quantity and by executing the equal process for the moving image data 101 and the compressed moving image data 301 .
- identical moving image can be retrieved and identified from the moving image by using the information 1303 of the unique identification of the compressed moving image at the higher speed.
- FIGS. 26, 27A and 27 B An apparatus and a method for retrieving moving image in the thirteenth preferred embodiment according to the invention will be explained in FIGS. 26, 27A and 27 B.
- FIG. 26 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 26 with FIG. 1 and FIG. 3 .
- the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of the feature quantity of moving image, which includes physical information of the feature quantity of moving image such as information of luminance and color difference (Y, Cb, Cr) of each frame and/or information of color (R, G, B) of each frame, from moving image data 101 , an extracting unit 104 for arranging at least one physical information included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 105 of the unique identification of moving image to identify the moving image from the arrangement of the physical information on the time axis, a storing unit 106 for storing the information
- FIGS. 27A and 27B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 26 . Still, an identical step number is appended to the step having the identical process in FIGS. 27A and 27B with FIGS. 2A, 2B , 4 A and 4 B.
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the digital moving image data 2601 is the uncompressed moving image data 101 .
- the following process (the steps 202 to 210 ) is executed.
- the extracting unit 102 extracts the information 103 of the feature quantity of the moving image, which includes the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame, from the moving image data 101 (in the step 202 ).
- the extracting unit 104 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on the information 103 , mean value of the values, total value of the values and/or difference value of the values on the time axis. Then, the extracting unit 104 picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 104 extracts the picked out values as the information 105 of the unique identification of the moving image to be used for identifying the moving image (in the step 203 ).
- the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R
- the storing unit 106 for storing the information of the unique identification of the moving image stores the information 105 of the unique identification of the moving image extracted by the extracting unit 104 (in the step 204 ).
- the information 108 of the unique identification of the moving image is extracted from other moving image data (the object moving image data for retrieving) (in the step 205 ).
- This extracting process can be executed by the processing equal to the foregoing steps 201 to 204 (in FIGS. 2A and 2B ).
- the retrieving unit 109 for retrieving the information of the unique identification of the moving image compares the information 105 of the unique identification of the moving image stored in the storing unit 106 with the information 108 of the unique identification of the moving image extracted from the other moving image data (in the step 206 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the digital moving image data 2601 is the compressed moving image data 301 .
- the following process (the steps 402 to 410 ) is executed.
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402 ).
- the extracting unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 401 to 404 (in FIGS. 4A and 4B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the thirteenth embodiment of the invention whether the inputted digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of image data to be retrieved.
- FIGS. 28, 29A and 29 B An apparatus and a method for retrieving moving image in the fourteenth preferred embodiment according to the invention will be explained in FIGS. 28, 29A and 29 B.
- FIG. 28 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 28 with FIG. 5 and FIG. 7 .
- the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of the feature quantity of moving image, which includes physical information of the feature quantity of moving image such as information of luminance and color difference (Y, Cb, Cr) of each frame and/or information of color (R, G, B) of each frame, from moving image data 101 , a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 503 for arranging at least one physical information included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 505 of the unique identification of moving image to identify
- FIGS. 29A and 29B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 28 . Still, an identical step number is appended to the step having the identical process in FIGS. 29A and 29B with FIGS. 6A, 6B , 8 A and 8 B.
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the digital moving image data 2601 is the uncompressed moving image data 101 .
- the following process (the steps 202 to 210 ) is executed.
- the extracting unit 102 extracts the information 103 of the feature quantity of the moving image, which includes the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame, from the moving image data 101 (in the step 202 ).
- the detecting unit 501 detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- This cut point of the moving image data 101 shows a point where the scene of the moving image data 101 changes or a point where the camera for photographing the moving image data 101 switches.
- the extracting unit 503 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on the information 103 , mean value of the values, total value of the values and/or difference value of the values on the time axis. Then, the extracting unit 503 picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals.
- the extracting unit 503 extracts the picked out values in each shot unit of the moving image data 101 divided by the information 502 of the cut point as the information 505 of the unique identification of the moving image to be used for identifying the moving image (in the step 603 ).
- the storing unit 106 for storing the information of the unique identification of the moving image stores the information 505 of the unique identification of the moving image extracted by the extracting unit 503 (in the step 604 ).
- the information 508 of the unique identification of the moving image is extracted from other moving image data (the object moving image data for retrieving) (in the step 205 ).
- This extracting process can be executed by the processing equal to the foregoing steps 601 to 604 (in FIGS. 6A and 6B ).
- the retrieving unit 109 for retrieving the information of the unique identification of the moving image compares the information 505 of the unique identification of the moving image stored in the storing unit 106 with the information 508 of the unique identification of the moving image extracted from the other moving image data (in the step 206 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or ⁇ printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the digital moving image data 2601 is the compressed moving image data 301 .
- the following process (the steps 402 to 410 ) is executed.
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402 ). And, the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802 ).
- the extracting unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 703 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 703 extracts the picked out values in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 803 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 801 to 804 (in FIGS. 8A and 8B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the fourteenth embodiment of the invention whether the inputted digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data.
- FIGS. 30, 31A and 31 B An apparatus and a method for retrieving moving image in the fifteenth preferred embodiment according to the invention will be explained in FIGS. 30, 31A and 31 B.
- FIG. 30 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 30 with FIG. 9 .
- the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 , a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels unit of the moving image data 101 , an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame,
- FIGS. 31A and 31B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 30 . Still, an identical step number is appended to the step having the identical process in FIGS. 31A and 31B with FIGS. 10A and 10B .
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the compressed/uncompressed judging unit 2602 outputs the moving image data 101 to the extracting unit 901 for extracting the information of the motion compensation and the DCT operation unit 902 . And, the following process (the steps 1003 to 410 ) is executed.
- the extracting unit 901 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M ⁇ N pixels unit (in the step 1003 ).
- the DCT operation unit 902 divides each frame of the moving image data 101 into the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M′ ⁇ N′ pixels unit (in the step 1004 ).
- the compressed/uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image. And, the following process (the steps 402 to 410 ) is executed.
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402 ).
- the extracting unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 extracted by extracting unit 302 , or the extracting unit 901 and the DCT operation unit 902 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404 (in FIGS. 10A and 10B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the fifteenth embodiment of the invention whether the inputted digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data.
- FIGS. 32, 33A and 33 B An apparatus and a method for retrieving moving image in the sixteenth preferred embodiment according to the invention will be explained in FIGS. 32, 33A and 33 B.
- FIG. 32 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 32 with FIG. 11 .
- the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 , a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels unit of the moving image data 101 , a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 70
- FIGS. 33A and 33B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 32 . Still, an identical step number is appended to the step having the identical process in FIGS. 33A and 33B with FIGS. 12A and 12B .
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the compressed/uncompressed judging unit 2602 outputs the moving image data 101 to the extracting unit 901 for extracting the information of the motion compensation, the DCT operation unit 902 and the detecting unit 501 for detecting the cut point. And, the following process (the steps 1003 to 410 ) is executed.
- the extracting unit 901 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M ⁇ N pixels unit (in the step 1003 ).
- the DCT operation unit 902 divides each frame of the moving image data 101 into the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M′ ⁇ N′ pixels unit (in the step 1004 ).
- the detecting unit 501 detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- the compressed/uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point. And, the following process (the steps 402 to 410 ) is executed.
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402 ). And, the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802 ).
- the extracting unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 703 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 703 extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 or in each shot unit of the compressed moving image data 301 divided by using the information 702 as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 803 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804 (in FIGS. 12A and 12B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the sixteenth embodiment of the invention whether the inputted digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data.
- FIGS. 34, 35A and 35 B An apparatus and a method for retrieving moving image in the seventeenth preferred embodiment according to the invention will be explained in FIGS. 34, 35A and 35 B.
- FIG. 34 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 34 with FIG. 13 .
- the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 , an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B), and/or information of average luminance and average color difference (Y, Cb, Cr), in the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 1304 for arranging at least one
- DC
- FIGS. 35A and 35B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 34 . Still, an identical step number is appended to the step having the identical process in FIGS. 35A and 35B with FIGS. 15A and 15B .
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the compressed/uncompressed judging unit 2602 outputs the moving image data 101 to the extracting unit 901 for extracting the information of the motion compensation and the extracting unit 1301 for extracting the average value of the block unit. And, the following process (the steps 1003 to 410 ) is executed.
- the extracting unit 901 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M ⁇ N pixels unit (in the step 1003 ).
- the extracting unit 1301 divides each frame of the moving image data 101 into the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels units, then calculates the average pixel value in the M′ ⁇ N′ pixels unit (in the step 1502 ).
- the compressed/uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image. And, the following process (the steps 1503 to 410 ) is executed.
- the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 1503 ).
- the extracting unit 1304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 1304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 1304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1504 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404 (in FIGS. 15A and 15B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the seventeenth embodiment of the invention whether the inputted digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data.
- FIGS. 36, 37A and 37 B An apparatus and a method for retrieving moving image in the eighteenth preferred embodiment according to the invention will be explained in FIGS. 36, 37A and 37 B.
- FIG. 36 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 36 with FIG. 16 .
- the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 , an extracting unit 1301 for calculating an average pixel value such as information of average color (R, G, B) and/or information of average luminance and average color difference (Y, Cb, Cr) in the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels unit, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301
- DC
- FIGS. 37A and 37B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 36 . Still, an identical step number is appended to the step having the identical process in FIGS. 37A and 37B with FIGS. 17A and 17B .
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the compressed/uncompressed judging unit 2602 outputs the moving image data 101 to the extracting unit 901 for extracting the information of the motion compensation, the extracting unit 1301 for extracting the average value of the block unit and the detecting unit 501 for detecting the cut point. And, the following process (the steps 1003 to 410 ) is executed.
- the extracting unit 901 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M ⁇ N pixels unit (in the step 1003 ). And, the extracting unit 1301 divides each frame of the moving image data 101 into the M′ ⁇ N′ (M′ and N′ are natural numbers) pixels units, then calculates the average pixel value in the M′ ⁇ N′ pixels unit (in the step 1502 ). And more, the detecting unit 501 detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- the compressed/uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point. And, the following process (the steps 1503 to 410 ) is executed.
- the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 1503 ). And then, the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information of the cut point 702 (in the step 802 ).
- the extracting unit 1603 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/or difference value of the values.
- the extracting unit 1603 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation.
- the extracting unit 1603 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1603 extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804 (in FIGS. 17A and 17B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the eighteenth embodiment of the invention whether the inputted digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data.
- FIGS. 38, 39A and 39 B An apparatus and a method for retrieving moving image in the nineteenth preferred embodiment according to the invention will be explained in FIGS. 38, 39A and 39 B.
- FIG. 38 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 38 with FIG. 18 .
- the apparatus for retrieving moving image comprises a DCT (.Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 as information 303 of the feature quantity of compressed moving image, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or by the extracting unit 302 , on the time axis and for extracting information 305 of the unique identification of compressed
- FIGS. 39A and 39B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 38 . Still, an identical step number is appended to the step having the identical process in FIGS. 39A and 39B with FIGS. 19A and 19B .
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the compressed/uncompressed judging unit 2602 outputs the moving image data 101 to the DCT operation unit 902 .
- the following process (the steps 1902 to 410 ) is executed.
- the DCT operation unit 902 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M ⁇ N pixels unit as the information 303 of the feature quantity of the compressed moving image (in the step 1902 ).
- the compressed/uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image.
- the following process (the steps 1903 to 410 ) is executed.
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed moving image data 301 as (in the step 1903 ).
- the extracting unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 extracted by the extracting unit 902 , mean value of the values, total value of the values and/or difference value of the values. And the extracting unit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1994 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404 (in FIGS. 19A and 19B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the nineteenth embodiment of the invention whether the inputted digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data.
- FIGS. 40, 41A and 41 B An apparatus and a method for retrieving moving image in the twentieth preferred embodiment according to the invention will be explained in FIGS. 40, 41A and 41 B.
- FIG. 40 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 40 with FIG. 20 .
- the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient as information 303 of the feature quantity of compressed moving image by executing a DCT operation in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 , a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at
- FIGS. 41A and 41B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 40 . Still, an identical step number is appended to the step having the identical process in FIGS. 41A and 41B with FIGS. 21A and 21B .
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the compressed/uncompressed judging unit 2602 outputs the moving image data 101 to the DCT operation unit 902 and the detecting unit 501 for detecting the cut point. And, the following process (the steps 1902 to 410 ) is executed.
- the DCT operation unit 902 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M ⁇ N pixels unit as the information 303 of the feature quantity of the compressed moving image (in the step 1902 ).
- the detecting unit 501 for detecting the cut point detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- the compressed/uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting unit 302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point.
- the following process (the steps 1903 to 410 ) is executed.
- the extracting unit 302 extracts the information 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1903 ). And, the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802 ).
- the extracting unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in the information 303 , mean value of the values, total value of the values and/or difference value of the values. And the extracting unit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals.
- the extracting unit 703 extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 or in each shot unit of the compressed moving image data 301 divided by using the information 702 as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 2102 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804 (in FIGS. 21A and 21B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the inputted digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data.
- FIGS. 42, 43A and 43 B An apparatus and a method for retrieving moving image in the twenty-first preferred embodiment according to the invention will be explained in FIGS. 42, 43A and 43 B.
- FIG. 42 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 42 with FIG. 22 .
- the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B), and/or information of average luminance and average color difference (Y, Cb, Cr), in the M ⁇ N (M and N are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 1304 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame or the average pixel value, which is included in the information 1303 extracted by the extracting unit 1301 and/or the extracting unit
- FIGS. 43A and 43B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 42 . Still, an identical step number is appended to the step having the identical process in FIGS. 43A and 43B with FIGS. 23A and 23B .
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the compressed/uncompressed judging unit 2602 outputs the moving image data 101 to the extracting unit 1301 for extracting the average value of the block unit.
- the following process (the steps 1502 to 410 ) is executed.
- the extracting unit 1301 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the average pixel value in the M ⁇ N pixels unit as the information 1303 of the feature quantity of the compressed moving image (in the step 1502 ).
- the compressed/uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image.
- the following process (the steps 1503 to 410 ) is executed.
- the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1503 ).
- the extracting unit 1304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/or difference value of the values. And the extracting unit 1304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extracting unit 1304 extracts the picked out values as the information 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1504 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304 (in the step 404 ).
- the information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 404 (in FIGS. 23A and 23B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- FIGS. 44, 45A and 45 B An apparatus and a method for retrieving moving image in the twenty-second preferred embodiment according to the invention will be explained in FIGS. 44, 45A and 45 B.
- FIG. 44 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 44 with FIG. 24 .
- the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value such as information of average color (R, G, B) 5 and/or information of average luminance and average color difference (Y, Cb, Cr) in the M ⁇ N (M and N are natural numbers) pixels unit of moving image data 101 as information 1303 of the feature quantity of compressed moving image, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 1302 for extracting 10 information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting
- DC Direct
- FIGS. 45A and 45B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in FIG. 44 . Still, an identical step number is appended to the step having the identical process in FIGS. 45A and 45B with FIGS. 25A and 25B .
- the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701 ). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital moving image data 2601 is the uncompressed moving image data 101 or the compressed moving image data 301 (in the step 2702 ).
- the compressed/uncompressed judging unit 2602 outputs the moving image data 101 to the extracting unit 1301 for extracting the average value of the block unit and the detecting unit 501 for detecting the cut point. And, the following process (the steps 1502 to 410 ) is executed.
- the extracting unit 1301 divides each frame of the moving image data 101 into the M ⁇ N (M and N are natural numbers) pixels units, then calculates the average pixel value in the M ⁇ N pixels unit (in the step 1502 ). And, the detecting unit 501 detects the cut point of the moving image data 101 as the information 502 of the cut point (in the step 602 ).
- the compressed/uncompressed judging unit 2602 outputs the compressed moving image data 301 to the extracting unit 1302 for extracting the information of the feature quantity of the compressed moving image and the detecting unit 701 for detecting the cut point. And, the following process (the steps 1503 to 410 ) is executed.
- the extracting unit 1302 extracts the information 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1503 ). And, the detecting unit 701 detects the cut point of the compressed moving image data 301 as the information 702 of the cut point (in the step 802 ).
- the extracting unit 1603 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or the average pixel value of each frame, which are included in the information 1303 , mean value of the values, total value of the values and/or difference value of the values. And the extracting unit 1603 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals.
- the extracting unit 1603 extracts the picked out values in each shot unit of the moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point as the information 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702 ).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603 (in the step 804 ).
- the information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 1001 to 804 (in FIGS. 25A and 25B ).
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 with the information 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- FIGS. 46, 47A and 47 B An apparatus and a method for retrieving moving image in the twenty-third preferred embodiment according to the invention will be explained in FIGS. 46, 47A and 47 B.
- FIG. 46 is a block diagram showing an apparatus for monitoring television commercials, which is a specific example of the second embodiment according to the present invention. Still, an identical code is appended to the thing having the identical composition in FIG. 46 with FIG. 3 .
- the apparatus for monitoring television commercials comprises an extracting unit 302 A for extracting information (a DC component) 303 A of the feature quantity of compressed moving image, which includes the only DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from MPEG-2 stream (television broadcasting) 301 A compressed by MPEG-2 process, an extracting unit (a calculating unit for calculating average color information) 304 A for arranging an average value of the DC components of each frame, which is included in the information 303 A extracted by the extracting unit 302 A, on the time axis and for extracting the arrangement of the average values on the time axis as information (time series information of average color information) 305 A of the unique identification of compressed moving image to identify the compressed moving image, a storing unit 306 for storing the information 305 A of the unique identification of the compressed moving image extracted by the extracting unit 304 A, a retrieving unit 309 for retrieving identical information with the information 305 A of the unique identification of the compressed moving image
- the MPEG-2 stream (the television broadcasting) 301 A is provided by encoding the moving image of the current television broadcasting.
- FIGS. 47A and 47B are flowcharts of the process in the apparatus for monitoring television commercials to the present invention described in FIG. 46 . Still, an identical step number is appended to the step having the identical process in FIGS. 47A and 47B with FIGS. 4A and 4B .
- the extracting unit 302 A for extracting the DC component fetches the MPEG-2 stream (the television broadcasting) 301 A to be a retrieval key for identifying the compressed moving image (in the step 401 A).
- the extracting unit 302 A extracts the information (the DC component) 303 A of the feature quantity of the compressed moving image, which includes the only DC components of the DCT coefficients of each frame, from the MPEG-2 stream 301 A (in the step 402 A).
- the extracting unit (the calculating unit for calculating the average color information) 304 A for extracting the information of the unique identification of the compressed moving image calculates an average value by using the DC components of each frame, which is included in the information 303 A. Then, the extracting unit 304 A arranges the average values on the time axis and extracts the arrangement of the average values on the time axis as the information (the time series information of the average color information) 305 A of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403 A).
- the storing unit 306 for storing the information of the unique identification of the compressed moving image stores the information 305 A extracted by the extracting unit 304 A (in the step 404 ).
- the information (the time series information of the average color information) 305 A of the unique identification of the compressed moving image consists of three kinds of time series information, which are information of luminance and color difference (Y, Cb, Cr).
- time series information which are information of luminance and color difference (Y, Cb, Cr).
- the information (the time series information of the average color information of the television commercial) 308 A of the unique identification of the compressed moving image is extracted from the MPEG-2 stream of the television commercial (the object compressed moving image data for retrieving) (in the step 405 ).
- This extracting process can be executed by the processing equal to the foregoing steps 401 A to 404 .
- the MPEG-2 stream of the television commercial is provided by encoding the current television broadcasting equal to the above mentioned MPEG-2 stream 301 A.
- the retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image compares the information 305 A of the unique identification of the compressed moving image stored in the storing unit 306 with the information 308 A of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406 ).
- the apparatus for retrieving the moving image of the twenty-third embodiment is considered that is realized using the simple facility as PC (Personal Computer). Consequently, in the present television broadcasting, ghost phenomenon, in which image is multiply reflected by reflection and multipath of the television radio wave, and color slippage and the like state arise. A little effect is also generated in the information 305 A extracted using the television broadcasting.
- PC Personal Computer
- the information 305 A is shown by 3 graphs, and effect is also generated in the graph.
- the information 305 A is shown by 3 graphs, and effect is also generated in the graph.
- This comparing result is stored in the storing unit 110 for storing the retrieving result (in the step 207 ). In the case of displaying and/or printing the comparing result (in the step 208 ), it is outputted by the output unit 111 for displaying and/or printing the retrieving result (in the step 209 ).
- the experimental system shown in FIG. 46 was mounted, and the experiment testing the effectiveness of this invention was executed. In the following, the result of the experiment is shown.
- a process for dividing an image into a plurality of frames is executed. It is also possible to execute a process for dividing each of the frames into “n” (“n” is a natural number 2 or over) regions in addition to the above mentioned process.
- n is a natural number 2 or over
- the information of the unique identification of the moving image and/or the information of the unique identification of the compressed moving image for identifying the moving image are extracted respectively from the divided regions, and then, the extracted information is applied to the information 105 of the unique identification of the moving image and/or the information 305 and 305 A of the unique identification of the compressed moving image.
- a retrieving unit 109 for retrieving the information of the unique identification of the moving image and/or a retrieving unit 309 for retrieving the information of the unique identification of the compressed moving image retrieve the identical information with the information 105 and or the information 305 and 305 A from the information 108 of the unique identification of the moving image and/or the information 308 and 308 A of the unique identification of the compressed moving image beforehand extracted.
- the identical information with the information 105 and/or the information 305 and 305 A may be extracted with corresponding in all regions of the information 105 and/or the information 305 and 305 A.
- the precision of the retrieving process may be further improved.
- the information 105 and/or the information 305 and 305 A are extracted from “n” regions of each of the frames and a process of extracting moving image is executed by using the information 105 and/or the information 305 and 305 A of limited regions over “n-m” (1 ⁇ m ⁇ (n-1)), it is possible to retrieve similar moving image with the difference in a part of regions. That is to say, the retrieving process may be executed with corresponding in the part of the regions selected as objects for retrieving, consequently, it is possible to retrieve similar moving image.
- a program for executing processes of the above mentioned methods for retrieving the moving image may be executed by a computer system.
- both the uncompressed moving image data and the compressed moving image data may be applied as the object moving image data for retrieving.
- the information 105 or 305 was extracted by using over one information from each of the frames.
- the information 105 or 305 will be extracted by using over two information from each of the frames, consequently, the precision of the retrieving process may be improved.
- the retrieving unit 109 or 309 may retrieve similar moving image according to comparing the information 105 or 305 by using a part of the information values from each of the frames.
Abstract
An apparatus for retrieving moving image according to the present invention has an extracting unit for extracting the information of the feature quantity of the moving image from the fixed moving image data, an extracting unit for extracting the information of the unique identification of the moving image to identify the moving image according to the information of the feature quantity of the moving image, and a retrieving unit for retrieving and identifying the fixed moving image from the object moving image for retrieving by comparing the information of the unique identification of the moving image with the information of the unique identification of the moving image beforehand extracted from the object moving image. Therefore, the information of the unique identification of the moving image for retrieving the moving image depends on the content of the moving image can be extracted, consequently, the identical moving image can be retrieved and identified from the moving image by using the information of the unique identification of the moving image at high speed.
Description
- This invention relates to a method and an apparatus for retrieving moving image, which retrieve and identify fixed moving image from some moving image. And more particularly to, a method and an apparatus for retrieving moving image, in which the information of the unique identification of moving image which depends on the content of the moving image is extracted and registered, the registered moving image is retrieved and identified earlier than video rate from moving image such as moving image archives sent in broadcasting and the like.
- It is well known that there are various technologies on the international standard encoding system such as MPEG (Motion Picture Experts Group) as the information compressing technology for storing and transmitting moving image. It becomes possible that moving image is sufficiently handled even in the PC (Personal Computer), if the encoded moving image compressed by this international standard encoding system is used. The technology for monitoring the specific moving image by using the PC, for example monitoring a television commercial (identifying moving image), is ever studied and is developed.
- As the conventional method for retrieving moving image, there are several methods for retrieving moving image, for example a method using the keyword retrieval, a method using the visual discrimination by the visual observation of the user, and a method using the number of continuous frames on the shot of moving image disclosed in Japanese Patent Publication No. JP7-retrieving moving image by detecting the cut point of the moving image. In this apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publication No. JP7-236153, the time variation of luminance difference between frames of the reduction image which is made from prepared reduction image and the mean value component data of the information of compressed and encoded moving image and the time variation of color difference histogram correlation value are used in order to obtain a cut point at high speed from moving image.
- And, there is an apparatus for processing moving image disclosed in Japanese Patent Publication No. JP7-152779 as an apparatus for detecting the characteristic frame of moving image. In this apparatus for processing moving image disclosed in Japanese Patent Publication No. JP7-152779, the low frequency part of DCT (Discrete Cosine Transform) component is used in order to detect characteristic frame that is used for as index information (retrieving information), for example a cut point frame, a motionless frame, a character frame, a specific color frame, etc., from compressed moving image data including the DCT operation.
- In the conventional methods and apparatuses for retrieving moving image, however, there are disadvantages as follows.
- That is to say, in the conventional method for retrieving moving image by the keyword retrieval, there is a disadvantage that the considerable cost for registering keyword index information is taken, because to beforehand register the keyword index information is necessary. And more, there is a disadvantage with the case in which keyword index information differs even in the case of retrieving of identical moving image by the user, because the feeling of moving image of each user who retrieves moving image and the way of understanding are different. Moreover, there is a disadvantage that the time for choosing the keyword index of the desire is taken, even the system for inputting keyword index in which the user who retrieves moving image from keyword index information presented by a list type is made to choose the keyword index of the desire.
- And, in the conventional method for retrieving moving image using the visual discrimination by the visual observation of the user, there is a disadvantage that hands are taken and the cost is high, because it is the method in which the user who retrieves moving image memorizes moving image that the user intends to retrieve and identify, and the user retrieves and identifies the moving image of the desire by observing all moving images.
- And, in the method for retrieving moving image using the number of continuous frames on the shot which constitutes moving image disclosed in Japanese Patent Publication JP7-46517, there is a disadvantage that it might be misunderstood that moving image is identical with moving image for retrieving, if the number of continuous frames of the moving image agrees even if the content of the moving image differs from the moving image for retrieving, because the number of continuous frames is not index information reflecting the content of the meaning of moving image.
- And, in the conventional real time discrimination method of image scene based on the time series coding of cutting, there is a disadvantage that the time for the extracting process of a cut point frame is taken and the cost also rises, because it is necessary to accurately extract a cut point frame in order to obtain a right collation result.
- And, in the apparatus for retrieving the cut point of moving image disclosed in Japanese Patent Publication No. JP7-236153, there is a disadvantage that it is impossible to retrieve and identify moving image of the desire automatically, because the detection of a cut point in moving image is a purpose, and it is only to detect a cut point by regarding the time variation of luminance difference and the time variation of color difference histogram correlation value.
- In the apparatus for processing moving image disclosed in Japanese Patent Publication No. JP7-152779, there is a disadvantage that time is taken due to the complicated process and the cost is high. Because the interim result which corresponds to the result of DCT operation by the process for reconstructing image data from compressed image data is made to be signals of the information of the feature quantity of moving image index, at least a kind of moving image index is detected from this signals of the information of the feature quantity of moving image index, and the sum of difference value with the moving image index of examined image is processed at the threshold, and then moving image is detected as index image by the result of this threshold processing.
- Accordingly, it is an object of the invention to provide a method and an apparatus for retrieving moving image, in which the information of the unique identification of moving image used for retrieving in accordance with the content of the moving image can be automatically extracted even in the computer with small resources such as PC, and in which moving image can be retrieved and identified automatically by using the information of the unique identification of moving image.
- According to the first feature of the invention, an apparatus for retrieving moving image to retrieve pre-determined moving image from object moving image, comprises first extracting means for extracting the information of the feature quantity of each frame from the pre-determined moving image, second extracting means for extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by the first extracting means, and retrieving means for retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the second extracting means, with the information of the unique identification prepared in accordance with the object moving image.
- According to the second feature of the invention, a method for retrieving moving image to retrieve pre-determined moving image from object moving image, comprises the steps of (A) extracting the information of the feature quantity of each frame from the pre-determined moving image, (B) extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by said step (A), and (C) retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the step (B), with the information of the unique identification prepared in accordance with the object moving image.
- According to the third feature of the invention, a program for retrieving moving image to retrieve predetermined moving image from object moving image, which is executed by computer system, comprises the steps of (A) extracting the information of the feature quantity of each frame from the pre-determined moving image, (B) extracting the information of the unique identification of moving image to identify the moving image in accordance with the information of the feature quantity, extracted by the step (A), and (C) retrieving the pre-determined moving image from the object moving image by comparing the information of the unique identification, extracted by the step (B), with the information of the unique identification prepared in accordance with the object moving image.
- The invention will be described in more detail in conjunction with the appended drawings, wherein:
-
FIG. 1 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 2A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 2B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 3 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 4A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 4B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 5 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 6A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 6B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 7 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 8A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 8B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 9 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 10A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 10B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 11 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 12A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 12B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 13 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 14 is a conceptual view showing the process of DCT (discrete cosine transform) operation in MPEG; -
FIG. 15A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 15B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 16 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 17A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 17B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 18 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 19A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 19B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 20 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 21A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 21B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 22 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 23A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 23B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 24 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 25A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 25B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 26 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 27A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 27B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 28 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 29A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 29B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 30 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 31A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 31B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 32 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 33A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 33B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 34 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 35A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 35B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 36 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 37A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 37B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 38 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 39A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 39B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 40 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 41A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 41B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 42 is a block diagram showing the apparatus for retrieving moving image according to the present invention; -
FIG. 43A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 43B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 44 is a block diagram showing the apparatus for retrieving moving image according to the present invention; and -
FIG. 45A is a flowchart of the process in the apparatus for retrieving moving image according to the present invention. -
FIG. 45B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; -
FIG. 46 is a block diagram showing the apparatus for monitoring television commercials according to the present invention; and -
FIG. 47A is a flowchart of the process in the apparatus for monitoring television commercials according to the present invention. -
FIG. 47B is a flowchart of the process in the apparatus for retrieving moving image according to the present invention; - In the following, a method and an apparatus for retrieving moving image, and a recording medium according to the invention are explained in details referring to the drawings.
- An apparatus and a method for retrieving moving image in the first preferred embodiment according to the invention will be explained in
FIGS. 1, 2A and 2B. -
FIG. 1 is a block diagram showing an apparatus for retrieving moving image according to the present invention. InFIG. 1 , the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of the feature quantity of moving image, which includes physical information of the feature quantity of moving image such as information of luminance and color difference (Y, Cb, Cr) of each frame and/or information of color (R, G, B) of each frame, from moving image data 101, an extracting unit 104 for arranging at least one physical information included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 105 of the unique identification of moving image to identify the moving image from the arrangement of the physical information on the time axis, a storing unit 106 for storing the information 105 of the unique identification of the moving image extracted by the extracting unit 104, a retrieving unit 109 for retrieving identical information with the information 105 of the unique identification of the moving image stored in the storing unit 106 from information 108 of the unique identification of moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 109, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 2A and 2B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 1 . InFIGS. 1, 2A and 2B, at first the extractingunit 102 for extracting the information of the feature quantity of the moving image fetches the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 201). Next, the extractingunit 102 extracts theinformation 103 of the feature quantity of the moving image, which includes the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame, from the moving image data 101 (in the step 202). - Next, the extracting
unit 104 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on theinformation 103, mean value of the values, total value of the values and/or difference value of the values on the time axis. Then, the extractingunit 104 picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 104 extracts the picked out values as theinformation 105 of the unique identification of the moving image to be used for identifying the moving image (in the step 203). - The storing
unit 106 for storing the information of the unique identification of the moving image stores theinformation 105 of the unique identification of the moving image extracted by the extracting unit 104 (in the step 204). - On the other hand, the
information 108 of the unique identification of the moving image is extracted from other moving image data (the object moving image data for retrieving) (in the step 205). This extracting process can be executed by the processing equal to the foregoingsteps 201 to 204. - Then, the retrieving
unit 109 for retrieving the information of the unique identification of the moving image compares theinformation 105 of the unique identification of the moving image stored in thestoring unit 106 with theinformation 108 of the unique identification of the moving image extracted from the other moving image data (in the step 206). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 206 to 209 is executed for all of theinformation 108 of the unique identification of the moving image extracted from the other moving image data (in the step 210). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The above mentioned extracting
unit 104 arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and the information of the color (R, G, B) of each frame) based on theinformation 103 of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values on the time axis, and picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals, and then extracts the picked out values as theinformation 105 of the unique identification of the moving image to be used for identifying the moving image. However, the above mentioned extractingunit 104 may arrange at least one value from values of a part or multiple parts of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and the information of the color (R, G, B) of each frame) based on theinformation 103 of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values on the time axis, and pick out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals, and then extract the picked out values as theinformation 105 of the unique identification of the moving image to be used for identifying the moving image. That is to say, values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and theinformation 105 of the unique identification of the moving image may be extracted by using all of the above mentioned values of the information of the feature quantity of the moving image. And also, theinformation 105 of the unique identification of the moving image may be extracted by using some values of the information of the feature quantity of the moving image. As such embodiment, for example, values of the information of the feature quantity of the moving image, which are calculated from blocks that are located on the part except for the part of time display and the part of around frame (the part with unevenness), are used. - And more, the extracting
unit 104 may calculate values after executing the weighting on the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on theinformation 103, then may apply the values, mean value of the values, total value of the values and/or difference value of the values. - Moreover, the extracting
unit 104 may calculate values of the physical information of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values, and then may quantize the calculated values. - As mentioned above, according to the first embodiment of the invention, information of the unique identification of moving image for retrieving moving image depends on the content of the moving image can be extracted, consequently, identical moving image can be retrieved and identified from the moving image by using the information of the unique identification of the moving image at high speed.
- An apparatus and a method for retrieving moving image in the second preferred embodiment according to the invention will be explained in
FIGS. 3, 4A and 4B. -
FIG. 3 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 3 withFIG. 1 . InFIG. 3 , the apparatus for retrieving moving image comprises an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are included in the information 303, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 4A and 4B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 3 . Still, an identical step number is appended to the step having the identical process inFIGS. 4A and 4B withFIGS. 2A and 2B . InFIGS. 3, 4A and 4B, at first the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 401). Next, the extractingunit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402). - Next, the extracting
unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 401 to 404. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The above mentioned extracting
unit 304 calculates values of the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values, and values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on the whole frame from the information of the motion compensation. And the above mentioned extractingunit 304 arranges at least one value from the above mentioned values and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the above mentioned extractingunit 304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image. However, the above mentioned extractingunit 304 may calculate values of a part or multiple parts of the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values, and values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on the whole frame from a part or multiple parts of the information of the motion compensation. That is to say, values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and theinformation 305 of the unique identification of the moving image may be extracted by using all of the above mentioned values of the information of the feature quantity of the moving image. And also, theinformation 305 of the unique identification of the moving image may be extracted by using some values of the information of the feature quantity of the moving image. As such embodiment, for example, values of the information of the feature quantity of the moving image, which are calculated from blocks that are located on the part except for the part of time display and the part of around frame (the part with unevenness), are used. - And more, the extracting
unit 304 may execute the weighting on the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, then may calculate values from the weighted information of the feature quantity of the compressed moving image, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 304 may execute the weighting on the information of the motion compensation and may calculate a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the weighted information of the motion compensation. - Moreover, in the extracting
unit 304, values provided by quantizing the above mentioned values of the physical information of the feature quantity of the compressed moving image and the like, and values provided by quantizing the above mentioned motion vectors, and the like may be applied. - As mentioned above, according to the second embodiment of the invention, information of the unique identification of moving image for retrieving moving image depends on the content of the compressed moving image provided by the standard compression encoding process for moving image such as MPEG can be extracted, consequently, identical moving image can be retrieved and identified from the moving image by using the information of the unique identification of the moving image at high speed.
- An apparatus and a method for retrieving moving image in the third preferred embodiment according to the invention will be explained in
FIGS. 5, 6A and 6B. -
FIG. 5 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 5 withFIG. 1 . InFIG. 5 , the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of the feature quantity of moving image, which includes physical information of the feature quantity of moving image such as information of luminance and color difference (Y, Cb, Cr) of each frame and/or information of color (R, G, B) of each frame, from moving image data 101, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 503 for arranging at least one physical information included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 505 of the unique identification of moving image to identify the moving image in each shot unit of the moving image data 101 divided by using the information 502 of the cut point from the arrangement of the physical information on the time axis, a storing unit 106 for storing the information 505 of the unique identification of the moving image extracted by the extracting unit 503, a retrieving unit 109 for retrieving identical information with the information 505 of the unique identification of the moving image stored in the storing unit 106 from information 508 of the unique identification of moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 109, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 6A and 6B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 5 . Still, an identical step number is appended to the step having the identical process inFIGS. 6A and 6B withFIGS. 2A and 2B . InFIGS. 5, 6A and 6B, at first the extractingunit 102 for extracting the information of the feature quantity of the moving image and the detectingunit 501 for detecting the cut point fetch the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 601). - Next, the extracting
unit 102 extracts theinformation 103 of the feature quantity of the moving image, which includes the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame, from the moving image data 101 (in the step 202). - And, the detecting
unit 501 detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). This cut point of the movingimage data 101 shows a point where the scene of the movingimage data 101 changes or a point where the camera for photographing the movingimage data 101 switches. - Next, the extracting
unit 503 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on theinformation 103, mean value of the values, total value of the values and/or difference value of the values on the time axis. Then, the extractingunit 503 picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 503 extracts the picked out values in each shot unit of the movingimage data 101 divided by theinformation 502 of the cut point as theinformation 505 of the unique identification of the moving image to be used for identifying the moving image (in the step 603). - The storing
unit 106 for storing the information of the unique identification of the moving image stores theinformation 505 of the unique identification of the moving image extracted by the extracting unit 503 (in the step 604). - On the other hand, the
information 508 of the unique identification of the moving image is extracted from other moving image data (the object moving image data for retrieving) (in the step 205). This extracting process can be executed by the processing equal to the foregoingsteps 601 to 604. - Then, the retrieving
unit 109 for retrieving the information of the unique identification of the moving image compares theinformation 505 of the unique identification of the moving image stored in thestoring unit 106 with theinformation 508 of the unique identification of the moving image extracted from the other moving image data (in the step 206). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 206 to 209 is executed for all of theinformation 508 of the unique identification of the moving image extracted from the other moving image data (in the step 210). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The above mentioned extracting
unit 503 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on theinformation 103 of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values on the time axis. However, the extractingunit 503 may arranges at least one value from values of a part or multiple parts of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on theinformation 103 of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values on the time axis. That is to say, values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and theinformation 505 of the unique identification of the moving image may be extracted by using all of the above mentioned values of the information of the feature quantity of the moving image. And also, theinformation 505 of the unique identification of the moving image may be extracted by using some values of the information of the feature quantity of the moving image. As such embodiment, for example, values of the information of the feature quantity of the moving image, which are calculated from blocks that are located on the part except for the part of time display and the part of around frame (the part with unevenness), are used. - And more, the extracting
unit 503 may calculate values after executing the weighting on the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on theinformation 103 of the feature quantity of the moving image, then may apply the values, mean value of the values, total value of the values and/or difference value of the values. - Moreover, the extracting
unit 503 may calculate values of the physical information of the feature quantity of the moving image, mean value of the values, total value of the values and/or difference value of the values, and then may quantize the calculated values. - As mentioned above, according to the third embodiment of the invention, the
information 505 of the unique identification of the moving image for retrieving the moving image depends on the content of the moving image can be extracted by the less information quantity with utilizing the time concept of a shot according to theinformation 502 of the cut point. And, identical moving image can be retrieved and identified from the moving image by using theinformation 505 of the unique identification of the moving image at the higher speed. - An apparatus and a method for retrieving moving image in the fourth preferred embodiment according to the invention will be explained in
FIGS. 7, 8A and 8B. -
FIG. 7 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 7 withFIG. 3 . InFIG. 7 , the apparatus for retrieving moving image comprises an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are included in the information 303, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 8A and 8B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 7 . Still, an identical step number is appended to the step having the identical process inFIGS. 8A and 8B withFIGS. 4A and 4B . InFIGS. 7, 8A and 8B, at first the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point fetch the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 801). Next, the extractingunit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402). And, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as theinformation 702 of the cut point (in the step 802). - Next, the extracting
unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 703 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 703 extracts the picked out values in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 of the cut point as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 803). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 801 to 804. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The above mentioned extracting
unit 703 calculates values of the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values, and values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. However, the extractingunit 703 may calculate values of a part or multiple parts of the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values, and values/such as motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on the whole frame from a part or multiple parts of the information of the motion compensation. That is to say, values of information of feature quantity of moving image in each block unit may be calculated by dividing a frame into several blocks, and theinformation 705 of the unique identification of the moving image may be extracted by using all of the above mentioned values of the information of the feature quantity of the moving image. And also, theinformation 705 of the unique identification of the moving image may be extracted by using some values of the information of the feature quantity of the moving image. As such embodiment, for example, values of the information of the feature quantity of the moving image, which are calculated from blocks that are located on the part except for the part of time display and the part of around frame (the part with unevenness), are used. - And more, the extracting
unit 703 may execute the weighting on the physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, then may calculate the above mentioned values from the weighted information of the feature quantity of the compressed moving image. On the other hand, the extractingunit 703 may execute the weighting on the information of the motion compensation and may calculate the above mentioned motion vectors from the weighted information of the motion compensation. - Moreover, in the extracting
unit 703, values provided by quantizing the above mentioned values of the physical information of the feature quantity of the compressed moving image and values provided by quantizing the above mentioned motion vectors, and the like may be applied. - As mentioned above, according to the fourth embodiment of the invention, the
information 705 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the compressed moving image provided by the standard compression encoding process for moving image such as MPEG can be extracted by the less information quantity with utilizing the time concept of the shot according to theinformation 702 of the cut point. And, identical moving image can be retrieved and identified from the moving image by using theinformation 705 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the fifth preferred embodiment according to the invention will be explained in
FIGS. 9, 10A and 10B. -
FIG. 9 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 9 withFIG. 1 andFIG. 3 . InFIG. 9 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M×N (M and N are natural numbers) pixels unit of moving image data 101, a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M′×N′ (M′ and N′ are natural numbers) pixels unit of the moving image data 101, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are the information 303 extracted by the extracting unit 302, or by the extracting unit 901 and the DCT operation unit 902, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. - FIGS. 10A and 10B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described in
FIG. 9 . Still, an identical step number is appended to the step having the identical process inFIGS. 10A and 10B withFIGS. 4A and 4B . InFIGS. 9, 10A and 10B, at first whether inputted moving image data is compressed is judged. (in the step 1001). - In case the inputted moving image data is the uncompressed moving
image data 101, the extractingunit 901 for extracting the information of the motion compensation and theDCT operation unit 902 fetch the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 1002). - The extracting
unit 901 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M×N pixels unit (in the step 1003). On the other hand, theDCT operation unit 902 divides each frame of the movingimage data 101 into the M′×N′ (M′ and N′ are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M′×N′ pixels unit (in the step 1004). - The information of the motion compensation in the M×N pixels unit provided by the extracting
unit 901 and the DCT coefficient in the M′×N′ pixels unit provided by theDCT operation unit 902 are sent to the extractingunit 304 for extracting the information of the unique identification of the compressed moving image as theinformation 303 of the feature quantity of the compressed moving image. - On the other hand, in case the inputted moving image data is the compressed moving
image data 301, the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 401). And the extractingunit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402). - Next, the extracting
unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303 extracted by extractingunit 302, or the extractingunit 901 and theDCT operation unit 902, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 404. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The M×N pixels unit and the M′×N′ pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- As mentioned above, according to the fifth embodiment of the invention, by executing the preprocessing (the
steps 1003 and 1004) on the uncompressed movingimage data 101, theinformation 305 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the movingimage 101 and the compressed movingimage 301 can be extracted by the equal processing (the step 403). And identical moving image can be retrieved and identified from the moving image by using theinformation 305 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the sixth preferred embodiment according to the invention will be explained in
FIGS. 11, 12A and 12B. -
FIG. 11 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 11 withFIG. 5 ,FIG. 7 andFIG. 9 . InFIG. 11 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M x N (M and N are natural numbers) pixels unit of moving image data 101, a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M′×N′ (M′ and N′ are natural numbers) pixels unit of the moving image data 101, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are included in the information 303 extracted by the extracting unit 901 and the DCT operation unit 902, or the extracting unit 302, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 12A and 12B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 11 . Still, an identical step number is appended to the step having the identical process inFIGS. 12A and 12B withFIGS. 6A, 6B , 8A, 8B, 10A and 10B. InFIGS. 11, 12A and 12B, at first whether inputted moving image data is compressed is judged (in the step 1001). - In case the inputted moving image data is the uncompressed moving
image data 101, the extractingunit 901 for extracting the information of the motion compensation, theDCT operation unit 902 and the detectingunit 501 for detecting the cut point fetch the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 1201). - The extracting
unit 901 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M×N pixels unit (in the step 1003). On the other hand, theDCT operation unit 902 divides each frame of the movingimage data 101 into the M′×N′ (M′ and N′ are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M′×N′ pixels unit (in the step 1004). - And, the detecting
unit 501 detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). - The information of the motion compensation in the M×N pixels unit provided by the extracting
unit 901 and the DCT coefficient in the M′×N′ pixels unit provided by theDCT operation unit 902 are sent to the extractingunit 703 for extracting the information of the unique identification of the compressed moving image with theinformation 502 of the cut point detected by the detectingunit 501 as theinformation 303 of the feature quantity of the compressed moving image. - On the other hand, in case the inputted moving image data is the compressed moving
image data 301, the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point fetch the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 801). Next, the extractingunit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step, 402). And, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as theinformation 702 of the cut point (in the step 802). - Next, the extracting
unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 703 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 703 extracts the picked out values in each shot unit of the movingimage data 101 divided by using theinformation 502 or in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 803). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 804. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The M×N pixels unit and the M′×N′ pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- As mentioned above, according to the sixth embodiment of the invention, by executing the preprocessing (the
steps 1003 and 1004) on the uncompressed movingimage data 101, theinformation 705 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the movingimage 101 and the compressed movingimage 301 can be extracted by the equal processing (the step 803). And, theinformation 705 of the unique identification of the compressed moving image for retrieving moving image can be extracted by the less information quantity with utilizing the time concept of the shot according to theinformation 502 and theinformation 702. Identical moving image can be retrieved and identified from the moving image by using theinformation 705 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the seventh preferred embodiment according to the invention will be explained in FIGS. 13 to 15B.
-
FIG. 13 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 13 withFIG. 3 andFIG. 9 . InFIG. 13 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M×N (M and N are natural numbers) pixels unit of moving image data 101, an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, Cr), in the M′×N′ (M′ and N′ are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 1304 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame or the average pixel value, which are included in the information 1303 extracted by the extracting unit 901 and the extracting unit 1301, or the extracting unit 1302, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. -
FIG. 14 is a conceptual scheme showing the relationship between the DCT (Discrete Cosine Transform) operation and the IDCT (Inverse Discrete Cosine Transform) operation, which are used in the international standard encoding system such as MPEG, and frame image. - The expression of the DCT operation is shown in the
following EXPRESSION 1. - And, the expression of the IDCT operation is shown in the
following EXPRESSION 2. - However, in the above mentioned
EXPRESSIONs - “N” is showing a block size of intra-code;
- “x” and “y” are showing a coordinate in a block;
- “f (x,y)” is showing a pixel value of the coordinate (x, y) in a block;
- “u” and “v” are showing a coordinate after the DCT conversion;
- “F (u, v)” is showing a DCT coefficient value of the coordinate (u, v) after the DCT conversion;
- “C (u)” is made 1/√ 2, if u=0;
-
- 1, if u≠0; and
- “C (v)” is made 1/√ 2, if v=0;
-
- 1, if v≠0.
- In
FIG. 14 , for example, in case the block size N of one frame is 8 and the coordinate (u, v) after the DCT conversion is (0, 0), F (u, v) in theEXPRESSION 1 is shown in the following EXPRESSION 3. - And more, an average pixel value in the block is shown in the following EXPRESSION 4.
- As shown in the above mentioned EXPRESSION 3 and 4, the DC component F (u, v) of the DCT coefficient becomes a value provided by multiplying the average pixel value in the block by the quantization number (N) of the block. That is to say, the DC components F (u, v) of the DCT coefficients extracted by the extracting
unit 1302 for extracting the information of the feature quantity of the compressed moving image and the average pixel value provided by the extractingunit 1301 for extracting the average value of the block unit may be handled as identical information data considering the quantization number of the block. Specifically, theinformation 1303 extracted by the extractingunit 1302, and theinformation 1303 depends on the information of the motion compensation extracted by the extractingunit 901 and the average pixel value extracted by the extractingunit 1301 may be considered information of data having an identical content composition. - On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following.
-
FIGS. 15A and 15B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 13 . Still, an identical step number is appended to the step having the identical process inFIGS. 15A and 15B withFIGS. 4A, 4B , 10A and 10B. InFIGS. 13, 15A and 15B, at first whether inputted moving image data is compressed is judged. (in the step 1001). - In case the inputted moving image data is the uncompressed moving
image data 101, the extractingunit 901 for extracting the information of the motion compensation and the extractingunit 1301 for extracting the average value of the block unit fetch the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 1501). - The extracting
unit 901 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M×N pixels unit (in the step 1003). On the other hand, the extractingunit 1301 divides each frame of the movingimage data 101 into the M′×N′ (M′ and N′ are natural numbers) pixels units, then calculates the average pixel value in the M′×N′ pixels unit (in the step 1502). - The information of the motion compensation in the M×N pixels unit provided by the extracting
unit 901 and the average pixel value in the M′×N′ pixels unit provided by the extractingunit 1301 are sent to the extractingunit 1304 for extracting the information of the unique identification of the compressed moving image as theinformation 1303 of the feature quantity of the compressed moving image. - On the other hand, in case the inputted moving image data is the compressed moving
image data 301, the extractingunit 1302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 401). And the extractingunit 1302 extracts theinformation 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 1503). - Next, the extracting
unit 1304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 1304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 1304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 1304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1504). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 404. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The above mentioned extracting
unit 1304 calculates values of physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/or difference value of the values. And the extractingunit 1304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. However, the extractingunit 1304 may calculate the above mentioned values by using a part or multiple parts of the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or the average pixel value of each frame, which are included in theinformation 1303, and the above mentioned vectors by using a part or multiple parts of the information of the motion compensation. - And more, the extracting
unit 1304 may execute the weighting on the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or on the average pixel value of each frame, then may calculate the above mentioned values from the weighted information of the feature quantity of the compressed moving image and/or the weighted average pixel value. On the other hand, the extractingunit 1304 may execute the weighting on the information of the motion compensation and may calculate the above mentioned vectors from the weighted information of the motion compensation. - Moreover, in the extracting
unit 1304, values provided by quantizing the above mentioned values of the physical information or of the average pixel value or the like, and values provided by quantizing the above mentioned motion vectors and the like may be applied. - Further more, the M×N pixels unit and the M′×N′ pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- As mentioned above, according to the seventh embodiment of the invention, by executing the preprocessing on the moving image, the
information 305 of the unique identification of the compressed moving image for retrieving the moving image depends on the content of the movingimage data 101 and the compressed movingimage data 301 can be extracted by the less information quantity and by executing the equal process for the movingimage data 101 and the compressed movingimage data 301. And, identical moving image can be retrieved and identified from the moving image by using theinformation 305 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the eighth preferred embodiment according to the invention will be explained in
FIGS. 16, 17A and 17B. -
FIG. 16 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 16 withFIG. 5 ,FIG. 7 andFIG. 13 . InFIG. 16 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M x N (M and N are natural numbers) pixels unit of moving image data 101, an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, Cr), in the M′×N′ (M′ and N′ are natural numbers) pixels unit, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 1603 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame or the average pixel value, which are included in the information 1303 extracted by the extracting unit 901 and the extracting unit 1301, or the extracting unit 1302, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 17A and 17B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 16 . Still, an identical step number is appended to the step having the identical process inFIGS. 17A and 17B withFIGS. 6A, 6B , 8A, 8B, 15A and 15B. InFIGS. 16, 17A and 17B, at first whether inputted moving image data is compressed is judged (in the step 1001). - In case the inputted moving image data is the uncompressed moving
image data 101, the extractingunit 901 for extracting the information of the motion compensation, the extractingunit 1301 for extracting the average value of the block unit and the detectingunit 501 for detecting the cut point fetch the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 1701). - The extracting
unit 901 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M×N pixels unit (in the step 1003). And, the extractingunit 1301 divides each frame of the movingimage data 101 into the M′×N′ (M′ and N′ are natural numbers) pixels units, then calculates the average pixel value in the M′×N′ pixels unit (in the step 1502). And more, the detectingunit 501 detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). - The information of the motion compensation in the M×N pixels unit provided by the extracting
unit 901 and the average pixel value in the M′×N′ pixels unit provided by the extractingunit 1301 are sent to the extractingunit 1603 for extracting the information of the unique identification of the compressed moving image with theinformation 502 of the cut point detected by the detectingunit 501 as theinformation 1303 of the feature quantity of the compressed moving image. - On the other hand, in case the inputted moving image data is the compressed moving
image data 301, the extractingunit 1302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point fetch the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 801). And the extractingunit 1302 extracts theinformation 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 1503). And then, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as the information of the cut point 702 (in the step 802). - Next, the extracting
unit 1603 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 1603 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 1603 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 1603 extracts the picked out values in each shot unit of the movingimage data 101 divided by using theinformation 502 of the cut point or in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 of the cut point as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 804. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The M×N pixels unit and the M′×N′ pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- As mentioned above, according to the eighth embodiment of the invention, by utilizing the time concept of the shot according to the information of the cut point and by executing the preprocessing on the moving image,
information 1303 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the movingimage data 101 and the compressed movingimage data 301 can be extracted by the less information quantity and by executing the equal process for the movingimage data 101 and the compressed movingimage data 301. And, identical moving image can be retrieved and identified from the moving image by using theinformation 1303 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the ninth preferred embodiment according to the invention will be explained in
FIGS. 18, 19A and 19B. -
FIG. 18 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 18 withFIG. 3 andFIG. 9 . InFIG. 18 , the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M×N (M and N are natural numbers) pixels unit of moving image data 101 as information 303 of the feature quantity of compressed moving image, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes the DCT coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or by the extracting unit 302, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 19A and 19B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 18 . Still, an identical step number is appended to the step having the identical process inFIGS. 19A and 19B withFIGS. 10A and 10B . InFIGS. 18, 19A and 19B, at first whether inputted moving image data is compressed is judged. (in the step 1001). - In case the inputted moving image data is the uncompressed moving
image data 101, theDCT operation unit 902 fetches the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 1901). - The
DCT operation unit 902 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M×N pixels unit as theinformation 303 of the feature quantity of the compressed moving image (in the step 1902). - On the other hand, in case the inputted moving image data is the compressed moving
image data 301, the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 401). And the extractingunit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed movingimage data 301 as (in the step 1903). - Next, the extracting
unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303 extracted by the extractingunit 902, mean value of the values, total value of the values and/or difference value of the values. And the extractingunit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1904). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 404. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The M×N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- As mentioned above, according to the ninth embodiment of the invention, by executing the preprocessing on the moving
image 101, theinformation 305 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the movingimage data 101 and the compressed movingimage data 301 can be extracted by executing the equal process. And, identical moving image can be retrieved and identified from the moving image by using theinformation 305 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the tenth preferred embodiment according to the invention will be explained in
FIGS. 20, 21A and 21B. -
FIG. 20 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 20 withFIG. 11 . InFIG. 20 , the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient as information 303 of the feature quantity of compressed moving image by executing a DCT operation in the M×N (M and N are natural numbers) pixels unit of moving image data 101, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coefficients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or the extracting unit 302, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the compressed moving image data 301 divided by using the information 502 of the cut point and/or the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 21A and 21B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 20 . Still, an identical step number is appended to the step having the identical process inFIGS. 21A and 21B withFIGS. 12A, 12B , 19A and 19B. InFIGS. 20, 21A and 21B, at first whether inputted moving image data is compressed is judged. (in the step 1001). - In case the inputted moving image data is the uncompressed moving
image data 101, theDCT operation unit 902 and the detectingunit 501 for detecting the cut point fetch the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 2101). - The
DCT operation unit 902 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M×N pixels unit as theinformation 303 of the feature quantity of the compressed moving image (in the step 1902). On the other hand, the detectingunit 501 for detecting the cut point detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). - The DCT coefficient in the M×N pixels unit provides by the
DCT operation unit 902 is sent to the extractingunit 703 for extracting the information of the unique identification of the compressed moving image with theinformation 502 detected by the detectingunit 501 as theinformation 303 of the feature quantity of the compressed moving image. - On the other hand, in case the inputted moving image data is the compressed moving
image data 301, the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point fetch the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 801). Next, the extractingunit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1903). And, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as theinformation 702 of the cut point (in the step 802). - Next, the extracting
unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values. And the extractingunit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 703 extracts the picked out values in each shot unit of the movingimage data 101 divided by using theinformation 502 or in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 2102). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 804. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The M×N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- As mentioned above, according to the tenth embodiment of the invention, by executing the preprocessing on the uncompressed moving
image data 101, theinformation 705 of the unique identification of the compressed moving image for retrieving moving image depends on the content of the movingimage 101 and the compressed movingimage 301 can be extracted by the equal processing. And, theinformation 705 for retrieving moving image can be extracted by the less information quantity with utilizing the time concept of the shot according to theinformation 502 and theinformation 702. Identical moving image can be retrieved and identified from the moving image by using theinformation 705 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the eleventh preferred embodiment according to the invention will be explained in
FIGS. 22, 23A and 23B. -
FIG. 22 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 22 withFIG. 13 . InFIG. 22 , the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, Cr), in the M×N (M and N are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 1304 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame or the average pixel value, which is included in the information 1303 extracted by the extracting unit 1301 and/or the extracting unit 1302, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify compressed moving image from the arrangement of the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 23A and 23B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 22 . Still, an identical step number is appended to the step having the identical process inFIGS. 23A and 23B withFIGS. 15A and 15B . InFIGS. 22, 23A and 23B, at first whether inputted moving image data is compressed is judged. (in the step 1001). - In case the inputted moving image data is the uncompressed moving
image data 101, the extractingunit 1301 for extracting the average value of the block unit fetches the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 2301). - The extracting
unit 1301 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the average pixel value in the M×N pixels unit as theinformation 1303 of the feature quantity of the compressed moving image (in the step 1502). - On the other hand, in case the inputted moving image data is the compressed moving
image data 301, the extractingunit 1302 for extracting the information of the feature quantity of the compressed moving image fetches the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 401). Then, the extractingunit 1302 extracts theinformation 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1503). - Next, the extracting
unit 1304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and i or difference value of the values. And the extractingunit 1304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 1304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1504). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 404. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or sprinting the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The above mentioned extracting
unit 1304 calculates values of the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame or values of the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/ or difference value of the values. However, the extractingunit 1304 may calculate the above mentioned values by using a part or multiple parts of the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame or the average pixel value of each frame, which are included in theinformation 1303. - And more, the extracting
unit 1304 may execute the weighting on the physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame or the average pixel value of each frame, which are included in theinformation 1303, then may calculate the above mentioned values by the weighted information of the feature quantity of the compressed moving image or the weighted average pixel value. - Moreover, in the extracting
unit 1304, values provided by quantizing the above mentioned physical information or average pixel value or the like may be applied. - Further more, the M×N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- As mentioned above, according to the eleventh embodiment of the invention, by executing the preprocessing on the moving image, the
information 305 of the unique identification of the compressed moving image for retrieving the moving image depends on the content of the movingimage data 101 and the compressed movingimage data 301 can be extracted by the less information quantity and by executing the equal process for the movingimage data 101 and the compressed movingimage data 301. And, identical moving image can be retrieved and identified from the moving image by using theinformation 305 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the twelfth preferred embodiment according to the invention will be explained in
FIGS. 24, 25A and 25B. -
FIG. 24 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 24 withFIG. 16 . InFIG. 24 , the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B) or information of average luminance and average color difference (Y, Cb, Cr), in the M×N (M and N are natural numbers) pixels unit of moving image data 101 as information 1303 of the feature quantity of compressed moving image, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 1603 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame or the average pixel value, which is included in the information 1303 extracted by the extracting unit 1301 and/or the extracting unit 1302, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. On the apparatus for retrieving the moving image according to the present invention constituted like the above, an operation will be explained in the following. -
FIGS. 25A and 25B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 24 . Still, an identical step number is appended to the step having the identical process inFIGS. 25A and 25B withFIGS. 17A and 17B . InFIGS. 24, 25A and 25B, at first whether inputted moving image data is compressed is judged (in the step 1001). - In case the inputted moving image data is the uncompressed moving
image data 101, the extractingunit 1301 for extracting the average value of the block unit and the detectingunit 501 for detecting the cut point fetch the movingimage data 101 to be a retrieval key for identifying the moving image (in the step 2501). - The extracting
unit 1301 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the average pixel value in the M×N pixels unit (in the step 1502). And, the detectingunit 501 detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). - On the other hand, in case the inputted moving image data is the compressed moving
image data 301, the extractingunit 1302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point fetch the compressed movingimage data 301 to be a retrieval key for identifying the compressed moving image (in the step 801). Then, the extractingunit 1302 extracts the information 1303.of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1503). And, the detectingunit 701 detests the cut point of the compressed movingimage data 301 as theinformation 702 of the cut point (in the step 802). - Next, the extracting
unit 1603 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/or difference value of the values. And the extractingunit 1603 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 1603 extracts the picked out values in each shot unit of the movingimage data 101 divided by using theinformation 502 of the cut point or in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 of the cut point as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 804. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above. The M×N pixels unit may be made to be a value equal to the pixel unit of the standard compression encoding system such as MPEG.
- As mentioned above, according to the twelfth embodiment of the invention, by utilizing the time concept of the shot according to the information of the cut point and executing the simple preprocessing on the moving image, the
information 1303 of the unique identification of the compressed moving image for retrieving the moving image depends on the content of the movingimage data 101 and the compressed movingimage data 301 can be extracted by the less information quantity and by executing the equal process for the movingimage data 101 and the compressed movingimage data 301. And, identical moving image can be retrieved and identified from the moving image by using theinformation 1303 of the unique identification of the compressed moving image at the higher speed. - An apparatus and a method for retrieving moving image in the thirteenth preferred embodiment according to the invention will be explained in
FIGS. 26, 27A and 27B. -
FIG. 26 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 26 withFIG. 1 andFIG. 3 . InFIG. 26 , the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of the feature quantity of moving image, which includes physical information of the feature quantity of moving image such as information of luminance and color difference (Y, Cb, Cr) of each frame and/or information of color (R, G, B) of each frame, from moving image data 101, an extracting unit 104 for arranging at least one physical information included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 105 of the unique identification of moving image to identify the moving image from the arrangement of the physical information on the time axis, a storing unit 106 for storing the information 105 of the unique identification of the moving image extracted by the extracting unit 104, a retrieving unit 109 for retrieving identical information with the information 105 of the unique identification of the moving image stored in the storing unit 106 from information 108 of the unique identification of moving image beforehand extracted from other moving image data, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are included in the information 303, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the extracting unit 102 and the compressed moving image data 301 to the extracting unit 302, a storing unit 110 for storing a retrieving result provided by the retrieving unit 109 and/or by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. -
FIGS. 27A and 27B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 26 . Still, an identical step number is appended to the step having the identical process inFIGS. 27A and 27B withFIGS. 2A, 2B , 4A and 4B. InFIGS. 26, 27A and 27B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the following process (thesteps 202 to 210) is executed. - The extracting
unit 102 extracts theinformation 103 of the feature quantity of the moving image, which includes the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame, from the moving image data 101 (in the step 202). - Next, the extracting
unit 104 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on theinformation 103, mean value of the values, total value of the values and/or difference value of the values on the time axis. Then, the extractingunit 104 picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 104 extracts the picked out values as theinformation 105 of the unique identification of the moving image to be used for identifying the moving image (in the step 203). - The storing
unit 106 for storing the information of the unique identification of the moving image stores theinformation 105 of the unique identification of the moving image extracted by the extracting unit 104 (in the step 204). - On the other hand, the
information 108 of the unique identification of the moving image is extracted from other moving image data (the object moving image data for retrieving) (in the step 205). This extracting process can be executed by the processing equal to the foregoingsteps 201 to 204 (inFIGS. 2A and 2B ). - Then, the retrieving
unit 109 for retrieving the information of the unique identification of the moving image compares theinformation 105 of the unique identification of the moving image stored in thestoring unit 106 with theinformation 108 of the unique identification of the moving image extracted from the other moving image data (in the step 206). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 206 to 209 is executed for all of theinformation 108 of the unique identification of the moving image extracted from the other moving image data (in the step 210). - In case the digital moving
image data 2601 is the compressed movingimage data 301, the following process (thesteps 402 to 410) is executed. - The extracting
unit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402). - Next, the extracting
unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 401 to 404 (inFIGS. 4A and 4B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209.is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the thirteenth embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of image data to be retrieved. - An apparatus and a method for retrieving moving image in the fourteenth preferred embodiment according to the invention will be explained in
FIGS. 28, 29A and 29B. -
FIG. 28 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 28 withFIG. 5 andFIG. 7 . InFIG. 28 , the apparatus for retrieving moving image comprises an extracting unit 102 for extracting information 103 of the feature quantity of moving image, which includes physical information of the feature quantity of moving image such as information of luminance and color difference (Y, Cb, Cr) of each frame and/or information of color (R, G, B) of each frame, from moving image data 101, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 503 for arranging at least one physical information included in the information 103 (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) on the time axis and for extracting information 505 of the unique identification of moving image to identify the moving image in each shot unit of the moving image data 101 divided by using the information 502 of the cut point from the arrangement of the physical information on the time axis, a storing unit 106 for storing the information 505 of the unique identification of the moving image extracted by the extracting unit 503, a retrieving unit 109 for retrieving identical information with the information 505 of the unique identification of the moving image stored in the storing unit 106 from information 508 of the unique identification of moving image beforehand extracted from other moving image data, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are included in the information 303, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the extracting unit 102 and the compressed moving image data 301 to the extracting unit 302, a storing unit 110 for storing the retrieving result provided by the retrieving unit 109 and/or the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. -
FIGS. 29A and 29B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 28 . Still, an identical step number is appended to the step having the identical process inFIGS. 29A and 29B withFIGS. 6A, 6B , 8A and 8B. InFIGS. 28, 29A and 29B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the following process (thesteps 202 to 210) is executed. - The extracting
unit 102 extracts theinformation 103 of the feature quantity of the moving image, which includes the physical information of the feature quantity of the moving image such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame, from the moving image data 101 (in the step 202). - And, the detecting
unit 501 detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). This cut point of the movingimage data 101 shows a point where the scene of the movingimage data 101 changes or a point where the camera for photographing the movingimage data 101 switches. - Next, the extracting
unit 503 for extracting the information of the unique identification of the moving image arranges at least one value from values of the physical information of the feature quantity of the moving image (such as the information of the luminance and the color difference (Y, Cb, Cr) of each frame and/or the information of the color (R, G, B) of each frame) based on theinformation 103, mean value of the values, total value of the values and/or difference value of the values on the time axis. Then, the extractingunit 503 picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 503 extracts the picked out values in each shot unit of the movingimage data 101 divided by theinformation 502 of the cut point as theinformation 505 of the unique identification of the moving image to be used for identifying the moving image (in the step 603). - The storing
unit 106 for storing the information of the unique identification of the moving image stores theinformation 505 of the unique identification of the moving image extracted by the extracting unit 503 (in the step 604). - On the other hand, the
information 508 of the unique identification of the moving image is extracted from other moving image data (the object moving image data for retrieving) (in the step 205). This extracting process can be executed by the processing equal to the foregoingsteps 601 to 604 (inFIGS. 6A and 6B ). - Then, the retrieving
unit 109 for retrieving the information of the unique identification of the moving image compares theinformation 505 of the unique identification of the moving image stored in thestoring unit 106 with theinformation 508 of the unique identification of the moving image extracted from the other moving image data (in the step 206). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or~printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 206 to 209 is executed for all of theinformation 508 of the unique identification of the moving image extracted from the other moving image data (in the step 210). - In case the digital moving
image data 2601 is the compressed movingimage data 301, the following process (thesteps 402 to 410) is executed. - The extracting
unit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402). And, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as theinformation 702 of the cut point (in the step 802). - Next, the extracting
unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 703 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 703 extracts the picked out values in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 of the cut point as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 803). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 801 to 804 (inFIGS. 8A and 8B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the fourteenth embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the fifteenth preferred embodiment according to the invention will be explained in
FIGS. 30, 31A and 31B. -
FIG. 30 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 30 withFIG. 9 . InFIG. 30 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M×N (M and N are natural numbers) pixels unit of moving image data 101, a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M′×N′ (M′ and N′ are natural numbers) pixels unit of the moving image data 101, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are the information 303 extracted by the extracting unit 302, or by the extracting unit 901 and the DCT operation unit 902, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the extracting unit 102 and the compressed moving image data 301 to the extracting unit. -
FIGS. 31A and 31B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 30 . Still, an identical step number is appended to the step having the identical process inFIGS. 31A and 31B withFIGS. 10A and 10B . InFIGS. 30, 31A and 31B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputs the movingimage data 101 to the extractingunit 901 for extracting the information of the motion compensation and theDCT operation unit 902. And, the following process (thesteps 1003 to 410) is executed. - The extracting
unit 901 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M×N pixels unit (in the step 1003). On the other hand, theDCT operation unit 902 divides each frame of the movingimage data 101 into the M′×N′ (M′ and N′ are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M′×N′ pixels unit (in the step 1004). - On the other hand, in case the digital moving
image data 2601 is the compressed movingimage data 301, the compressed/uncompressed judging unit 2602 outputs the compressed movingimage data 301 to the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image. And, the following process (thesteps 402 to 410) is executed. - The extracting
unit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402). - Next, the extracting
unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303 extracted by extractingunit 302, or the extractingunit 901 and theDCT operation unit 902, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 403). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 404 (inFIGS. 10A and 10B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the fifteenth embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the sixteenth preferred embodiment according to the invention will be explained in
FIGS. 32, 33A and 33B. -
FIG. 32 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 32 withFIG. 11 . InFIG. 32 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M×N (M and N are natural numbers) pixels unit of moving image data 101, a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M′×N′ (M′ and N′ are natural numbers) pixels unit of the moving image data 101, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coefficients of each frame and the information of the motion compensation of each frame, which are included in the information 303 extracted by the extracting unit 901 and the DCT operation unit 902, or the extracting unit 302, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the extracting unit, the DCT (Discrete Cosine Transform) operation unit 902 and the detecting unit, and the compressed moving image data 301 to the extracting unit 302 and the detecting unit 701. -
FIGS. 33A and 33B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 32 . Still, an identical step number is appended to the step having the identical process inFIGS. 33A and 33B withFIGS. 12A and 12B . InFIGS. 32, 33A and 33B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputs the movingimage data 101 to the extractingunit 901 for extracting the information of the motion compensation, theDCT operation unit 902 and the detectingunit 501 for detecting the cut point. And, the following process (thesteps 1003 to 410) is executed. - The extracting
unit 901 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M×N pixels unit (in the step 1003). On the other hand, theDCT operation unit 902 divides each frame of the movingimage data 101 into the M′×N′ (M′ and N′ are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M′×N′ pixels unit (in the step 1004). - And, the detecting
unit 501 detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). - On the other hand, in case the digital moving
image data 2601 is the compressed movingimage data 301, the compressed/uncompressed judging unit 2602 outputs the compressed movingimage data 301 to the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point. And, the following process (thesteps 402 to 410) is executed. - The extracting
unit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 402). And, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as theinformation 702 of the cut point (in the step 802). - Next, the extracting
unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 703 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 703 extracts the picked out values in each shot unit of the movingimage data 101 divided by using theinformation 502 or in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 803). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 804 (inFIGS. 12A and 12B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the sixteenth embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the seventeenth preferred embodiment according to the invention will be explained in
FIGS. 34, 35A and 35B. -
FIG. 34 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 34 withFIG. 13 . InFIG. 34 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M×N (M and N are natural numbers) pixels unit of moving image data 101, an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B), and/or information of average luminance and average color difference (Y, Cb, Cr), in the M′×N′ (M′ and N′ are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 1304 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame or the average pixel value, which are included in the information 1303 extracted by the extracting unit 901 and the extracting unit 1301, or the extracting unit 1302, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the extracting unit 901 and the extracting unit 1301, and the compressed moving image data 301 to the extracting unit 1302. -
FIGS. 35A and 35B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 34 . Still, an identical step number is appended to the step having the identical process inFIGS. 35A and 35B withFIGS. 15A and 15B . InFIGS. 34, 35A and 35B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputs the movingimage data 101 to the extractingunit 901 for extracting the information of the motion compensation and the extractingunit 1301 for extracting the average value of the block unit. And, the following process (thesteps 1003 to 410) is executed. - The extracting
unit 901 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M×N pixels unit (in the step 1003). On the other hand, the extractingunit 1301 divides each frame of the movingimage data 101 into the M′×N′ (M′ and N′ are natural numbers) pixels units, then calculates the average pixel value in the M′×N′ pixels unit (in the step 1502). - On the other hand, in case the digital moving
image data 2601 is the compressed movingimage data 301, the compressed/uncompressed judging unit 2602 outputs the compressed movingimage data 301 to the extractingunit 1302 for extracting the information of the feature quantity of the compressed moving image. And, the following process (thesteps 1503 to 410) is executed. - The extracting
unit 1302 extracts theinformation 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 1503). - Next, the extracting
unit 1304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 1304 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 1304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 1304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1504). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 404 (inFIGS. 15A and 15B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image *99 extracted from the other moving image data (in the step 410). - As mentioned above, according to the seventeenth embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the eighteenth preferred embodiment according to the invention will be explained in
FIGS. 36, 37A and 37B. -
FIG. 36 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 36 withFIG. 16 . InFIG. 36 , the apparatus for retrieving moving image comprises an extracting unit 901 for extracting information of motion compensation in the M×N (M and N are natural numbers) pixels unit of moving image data 101, an extracting unit 1301 for calculating an average pixel value such as information of average color (R, G, B) and/or information of average luminance and average color difference (Y, Cb, Cr) in the M′×N′ (M′ and N′ are natural numbers) pixels unit, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame and information of motion compensation of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 1603 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame or the average pixel values, which are included in the information 1303 extracted by the extracting unit 901 and the extracting unit 1301, or the extracting unit 1302, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the extracting unit 901, the extracting unit 1301 and the detecting unit 501, and the compressed moving image data 301 to the extracting unit 1302 and the detecting unit 701. -
FIGS. 37A and 37B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 36 . Still, an identical step number is appended to the step having the identical process inFIGS. 37A and 37B withFIGS. 17A and 17B . InFIGS. 36, 37A and 37B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputs the movingimage data 101 to the extractingunit 901 for extracting the information of the motion compensation, the extractingunit 1301 for extracting the average value of the block unit and the detectingunit 501 for detecting the cut point. And, the following process (thesteps 1003 to 410) is executed. - The extracting
unit 901 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the information of the motion compensation in the M×N pixels unit (in the step 1003). And, the extractingunit 1301 divides each frame of the movingimage data 101 into the M′×N′ (M′ and N′ are natural numbers) pixels units, then calculates the average pixel value in the M′×N′ pixels unit (in the step 1502). And more, the detectingunit 501 detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). - On the other hand, in case the digital moving
image data 2601 is the compressed movingimage data 301, the compressed/uncompressed judging unit 2602 outputs the compressed movingimage data 301 to the extractingunit 1302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point. And, the following process (thesteps 1503 to 410) is executed. - The extracting
unit 1302 extracts theinformation 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame and the information of the motion compensation of each frame, from the compressed moving image data 301 (in the step 1503). And then, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as the information of the cut point 702 (in the step 802). - Next, the extracting
unit 1603 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/or difference value of the values. On the other hand, the extractingunit 1603 calculates values such as a motion vector, a mean motion vector between frames before and behind, a total motion vector, a difference vector and/or a motion vector on a whole frame from the information of the motion compensation. And the extractingunit 1603 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 1603 extracts the picked out values in each shot unit of the movingimage data 101 divided by using theinformation 502 of the cut point or in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 of the cut point as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 804 (inFIGS. 17A and 17B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the eighteenth embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the nineteenth preferred embodiment according to the invention will be explained in
FIGS. 38, 39A and 39B. -
FIG. 38 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 38 withFIG. 18 . InFIG. 38 , the apparatus for retrieving moving image comprises a DCT (.Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient by executing a DCT operation in the M×N (M and N are natural numbers) pixels unit of moving image data 101 as information 303 of the feature quantity of compressed moving image, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 304 for arranging at least one information of feature quantity from the DCT coefficients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or by the extracting unit 302, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the DCT (Discrete Cosine Transform) operation unit 902 and the compressed moving image data 301 to the extracting unit 302. -
FIGS. 39A and 39B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 38 . Still, an identical step number is appended to the step having the identical process inFIGS. 39A and 39B withFIGS. 19A and 19B . InFIGS. 38, 39A and 39B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputs the movingimage data 101 to theDCT operation unit 902. The following process (thesteps 1902 to 410) is executed. - The
DCT operation unit 902 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M×N pixels unit as theinformation 303 of the feature quantity of the compressed moving image (in the step 1902). - On the other hand, in case the digital moving
image data 2601 is the compressed movingimage data 301, the compressed/uncompressed judging unit 2602 outputs the compressed movingimage data 301 to the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image. The following process (thesteps 1903 to 410) is executed. - The extracting
unit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed movingimage data 301 as (in the step 1903). - Next, the extracting
unit 304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303 extracted by the extractingunit 902, mean value of the values, total value of the values and/or difference value of the values. And the extractingunit 304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1994). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 404 (inFIGS. 19A and 19B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the nineteenth embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the twentieth preferred embodiment according to the invention will be explained in
FIGS. 40, 41A and 41B. -
FIG. 40 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 40 withFIG. 20 . InFIG. 40 , the apparatus for retrieving moving image comprises a DCT (Discrete Cosine Transform) operation unit 902 for obtaining a DCT coefficient as information 303 of the feature quantity of compressed moving image by executing a DCT operation in the M×N (M and N are natural numbers) pixels unit of moving image data 101, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 302 for extracting information 303 of the feature quantity of compressed moving image, which includes DCT coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 301 as information 702 of a cut point, an extracting unit 703 for arranging at least one information of feature quantity from the DCT coefficients of each frame, which is included in the information 303 extracted by the DCT operation unit 902 or the extracting unit 302, on the time axis and for extracting information 705 of the unique identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the compressed moving image data 301 divided by using the information 502 of the cut point and/or the information 702 of the cut point, a storing unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 703, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the DCT (Discrete Cosine Transform) operation unit 902 and the detecting unit 501, and the compressed moving image data 301 to the extracting unit 302 and the detecting unit 701. -
FIGS. 41A and 41B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 40 . Still, an identical step number is appended to the step having the identical process inFIGS. 41A and 41B withFIGS. 21A and 21B . InFIGS. 40, 41A and 41B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputs the movingimage data 101 to theDCT operation unit 902 and the detectingunit 501 for detecting the cut point. And, the following process (thesteps 1902 to 410) is executed. - The
DCT operation unit 902 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the DCT coefficient by executing the DCT operation in the M×N pixels unit as theinformation 303 of the feature quantity of the compressed moving image (in the step 1902). On the other hand, the detectingunit 501 for detecting the cut point detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). - On the other hand, in case the digital moving
image data 2601 is the compressed movingimage data 301, the compressed/uncompressed judging unit 2602 outputs the compressed movingimage data 301 to the extractingunit 302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point. The following process (thesteps 1903 to 410) is executed. - The extracting
unit 302 extracts theinformation 303 of the feature quantity of the compressed moving image, which includes the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1903). And, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as theinformation 702 of the cut point (in the step 802). - Next, the extracting
unit 703 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of the compressed moving image from the DCT coefficients of each frame, which is included in theinformation 303, mean value of the values, total value of the values and/or difference value of the values. And the extractingunit 703 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 703 extracts the picked out values in each shot unit of the movingimage data 101 divided by using theinformation 502 or in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 2102). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 703 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 804 (inFIGS. 21A and 21B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the twentieth embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the twenty-first preferred embodiment according to the invention will be explained in
FIGS. 42, 43A and 43B. -
FIG. 42 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 42 withFIG. 22 . InFIG. 42 , the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value, such as information of average color (R, G, B), and/or information of average luminance and average color difference (Y, Cb, Cr), in the M×N (M and N are natural numbers) pixels unit, an extracting unit 1302 for extracting information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, an extracting unit 1304 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame or the average pixel value, which is included in the information 1303 extracted by the extracting unit 1301 and/or the extracting unit 1302, on the time axis and for extracting information 305 of the unique identification of compressed moving image to identify compressed moving image from the arrangement of the information of the feature quantity on the time axis, a storing unit 306 for storing the information 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304, a retrieving unit 309 for retrieving identical information with the information 305 of the unique identification of the compressed moving image stored in the storing unit 306 from information 308 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the extracting unit 1301 and the compressed moving image data 301 to the extracting unit 1302. -
FIGS. 43A and 43B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 42 . Still, an identical step number is appended to the step having the identical process inFIGS. 43A and 43B withFIGS. 23A and 23B . InFIGS. 42, 43A and 43B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputs the movingimage data 101 to the extractingunit 1301 for extracting the average value of the block unit. The following process (thesteps 1502 to 410) is executed. - The extracting
unit 1301 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the average pixel value in the M×N pixels unit as theinformation 1303 of the feature quantity of the compressed moving image (in the step 1502). - On the other hand, in case the digital moving
image data 2601 is the compressed movingimage data 301, the compressed/uncompressed judging unit 2602 outputs the compressed movingimage data 301 to the extractingunit 1302 for extracting the information of the feature quantity of the compressed moving image. The following process (thesteps 1503 to 410) is executed. - The extracting
unit 1302 extracts theinformation 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1503). - Next, the extracting
unit 1304 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or values of the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/or difference value of the values. And the extractingunit 1304 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 1304 extracts the picked out values as theinformation 305 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1504). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305 of the unique identification of the compressed moving image extracted by the extracting unit 1304 (in the step 404). - On the other hand, the
information 308 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 404 (inFIGS. 23A and 23B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the twenty-first embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the twenty-second preferred embodiment according to the invention will be explained in
FIGS. 44, 45A and 45B. -
FIG. 44 is a block diagram showing an apparatus for retrieving moving image according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 44 withFIG. 24 . InFIG. 44 , the apparatus for retrieving moving image comprises an extracting unit 1301 for calculating an average pixel value such as information of average color (R, G, B) 5 and/or information of average luminance and average color difference (Y, Cb, Cr) in the M×N (M and N are natural numbers) pixels unit of moving image data 101 as information 1303 of the feature quantity of compressed moving image, a detecting unit 501 for detecting a cut point of the moving image data 101 as information 502 of a cut point, an extracting unit 1302 for extracting 10 information 1303 of the feature quantity of compressed moving image, which includes DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from compressed moving image data 301 provided by the standard compression encoding process for moving image such as MPEG, a detecting unit 701 for detecting a cut point of the compressed moving image data 15 301 as information 702 of a cut point, an extracting unit 1603 for arranging at least one information of feature quantity from the DC components of the DCT coefficients of each frame or the average pixel value, which is included in the information 1303 extracted by the extracting unit 1301 and/or the extracting unit 1302, on the time axis and for extracting information 705 of the unique 20 identification of compressed moving image to identify the compressed moving image from the arrangement of the information of the feature quantity on the time axis in each shot unit of the moving image data 101 divided by using the information 502 of the cut point or in each shot unit of the compressed moving image data 301 divided by using the information 702 of the cut point, a storing 25 unit 306 for storing the information 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603, a retrieving unit 309 for retrieving identical information with the information 705 of the unique identification of the compressed moving image stored in the storing unit 306 from information 708 of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110, and a judging unit 2602 for judging whether inputted digital moving image data 2601 is the moving image data 101 or the compressed moving image data 301 and for outputting the moving image data 101 to the extracting unit and the detecting unit 501, and the compressed moving image data 301 to the extracting unit 1302 and the detecting unit 701. -
FIGS. 45A and 45B are flowcharts of the process in the apparatus for retrieving the moving image according to the present invention described inFIG. 44 . Still, an identical step number is appended to the step having the identical process inFIGS. 45A and 45B withFIGS. 25A and 25B . InFIGS. 44, 45A and 45B, at first the compressed/uncompressed judging unit 2602 acquires the digital moving image data 2601 (in the step 2701). Then, the compressed/uncompressed judging unit 2602 judges whether the acquired digital movingimage data 2601 is the uncompressed movingimage data 101 or the compressed moving image data 301 (in the step 2702). - In case the digital moving
image data 2601 is the uncompressed movingimage data 101, the compressed/uncompressed judging unit 2602 outputs the movingimage data 101 to the extractingunit 1301 for extracting the average value of the block unit and the detectingunit 501 for detecting the cut point. And, the following process (thesteps 1502 to 410) is executed. - The extracting
unit 1301 divides each frame of the movingimage data 101 into the M×N (M and N are natural numbers) pixels units, then calculates the average pixel value in the M×N pixels unit (in the step 1502). And, the detectingunit 501 detects the cut point of the movingimage data 101 as theinformation 502 of the cut point (in the step 602). - On the other hand, in case the digital moving
image data 2601 is the compressed movingimage data 301, the compressed/uncompressed judging unit 2602 outputs the compressed movingimage data 301 to the extractingunit 1302 for extracting the information of the feature quantity of the compressed moving image and the detectingunit 701 for detecting the cut point. And, the following process (thesteps 1503 to 410) is executed. - The extracting
unit 1302 extracts theinformation 1303 of the feature quantity of the compressed moving image, which includes the DC components of the DCT coefficients of each frame, from the compressed moving image data 301 (in the step 1503). And, the detectingunit 701 detects the cut point of the compressed movingimage data 301 as theinformation 702 of the cut point (in the step 802). - Next, the extracting
unit 1603 for extracting the information of the unique identification of the compressed moving image calculates values of physical information of the feature quantity of compressed moving image from the DC components of the DCT coefficients of each frame and/or the average pixel value of each frame, which are included in theinformation 1303, mean value of the values, total value of the values and/or difference value of the values. And the extractingunit 1603 arranges at least one value from the above mentioned values on the time axis and then picks out all values arranged on the time axis, or some values from the values arranged on the time axis at uniform intervals or irregular intervals. And then, the extractingunit 1603 extracts the picked out values in each shot unit of the movingimage data 101 divided by using theinformation 502 of the cut point or in each shot unit of the compressed movingimage data 301 divided by using theinformation 702 of the cut point as theinformation 705 of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in the step 1702). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 705 of the unique identification of the compressed moving image extracted by the extracting unit 1603 (in the step 804). - On the other hand, the
information 708 of the unique identification of the compressed moving image is extracted from other compressed moving image data (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoingsteps 1001 to 804 (inFIGS. 25A and 25B ). - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 705 of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 708 of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - As mentioned above, according to the twenty-second embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed movingimage data 101 or the compressed movingimage data 301 can be judged automatically. Consequently, identical moving image can be retrieved and identified from the moving image at the higher speed regardless of the type (compressed/uncompressed) of retrieving image data. - An apparatus and a method for retrieving moving image in the twenty-third preferred embodiment according to the invention will be explained in
FIGS. 46, 47A and 47B. -
FIG. 46 is a block diagram showing an apparatus for monitoring television commercials, which is a specific example of the second embodiment according to the present invention. Still, an identical code is appended to the thing having the identical composition inFIG. 46 withFIG. 3 . - In
FIG. 46 , the apparatus for monitoring television commercials comprises an extracting unit 302A for extracting information (a DC component) 303A of the feature quantity of compressed moving image, which includes the only DC (Direct Current) components of DCT (Discrete Cosine Transform) coefficients of each frame, from MPEG-2 stream (television broadcasting) 301A compressed by MPEG-2 process, an extracting unit (a calculating unit for calculating average color information) 304A for arranging an average value of the DC components of each frame, which is included in the information 303A extracted by the extracting unit 302A, on the time axis and for extracting the arrangement of the average values on the time axis as information (time series information of average color information) 305A of the unique identification of compressed moving image to identify the compressed moving image, a storing unit 306 for storing the information 305A of the unique identification of the compressed moving image extracted by the extracting unit 304A, a retrieving unit 309 for retrieving identical information with the information 305A of the unique identification of the compressed moving image stored in the storing unit 306 from information (time series information of average color information of a television commercial) 308A of the unique identification of compressed moving image beforehand extracted from other moving image data, a storing unit 110 for storing the retrieving result provided by the retrieving unit 309, and an output unit 111 for displaying and/or printing the retrieving result stored in the storing unit 110. - The MPEG-2 stream (the television broadcasting) 301A is provided by encoding the moving image of the current television broadcasting.
- On the apparatus for monitoring television commercials according to the present invention constituted like the above, an operation will be explained in the following.
-
FIGS. 47A and 47B are flowcharts of the process in the apparatus for monitoring television commercials to the present invention described inFIG. 46 . Still, an identical step number is appended to the step having the identical process inFIGS. 47A and 47B withFIGS. 4A and 4B . InFIGS. 46, 47A and 47B, at first the extractingunit 302A for extracting the DC component fetches the MPEG-2 stream (the television broadcasting) 301A to be a retrieval key for identifying the compressed moving image (in thestep 401A). Then, the extractingunit 302A extracts the information (the DC component) 303A of the feature quantity of the compressed moving image, which includes the only DC components of the DCT coefficients of each frame, from the MPEG-2stream 301A (in thestep 402A). - Next, the extracting unit (the calculating unit for calculating the average color information) 304A for extracting the information of the unique identification of the compressed moving image calculates an average value by using the DC components of each frame, which is included in the
information 303A. Then, the extractingunit 304A arranges the average values on the time axis and extracts the arrangement of the average values on the time axis as the information (the time series information of the average color information) 305A of the unique identification of the compressed moving image to be used for identifying the compressed moving image (in thestep 403A). - The storing
unit 306 for storing the information of the unique identification of the compressed moving image stores theinformation 305A extracted by the extractingunit 304A (in the step 404). - In the twenty-third embodiment, the information (the time series information of the average color information) 305A of the unique identification of the compressed moving image consists of three kinds of time series information, which are information of luminance and color difference (Y, Cb, Cr). By making a graph of each time series information, which it makes the horizontal line to be time and makes the vertical line to be a value of the average color information, the
information 305A is shown by three graphs. - On the other hand, the information (the time series information of the average color information of the television commercial) 308A of the unique identification of the compressed moving image is extracted from the MPEG-2 stream of the television commercial (the object compressed moving image data for retrieving) (in the step 405). This extracting process can be executed by the processing equal to the foregoing steps 401A to 404.
- The MPEG-2 stream of the television commercial is provided by encoding the current television broadcasting equal to the above mentioned MPEG-2
stream 301A. - Then, the retrieving
unit 309 for retrieving the information of the unique identification of the compressed moving image compares theinformation 305A of the unique identification of the compressed moving image stored in thestoring unit 306 with theinformation 308A of the unique identification of the compressed moving image extracted from the other moving image data (in the step 406). - The apparatus for retrieving the moving image of the twenty-third embodiment is considered that is realized using the simple facility as PC (Personal Computer). Consequently, in the present television broadcasting, ghost phenomenon, in which image is multiply reflected by reflection and multipath of the television radio wave, and color slippage and the like state arise. A little effect is also generated in the
information 305A extracted using the television broadcasting. - As mentioned above, the
information 305A is shown by 3 graphs, and effect is also generated in the graph. However, since there is sufficient information of unique identification of moving image to be used for identifying the moving image in the graph, it is possible to be difficult to receive the effect by expanding and reducing the graph. - This comparing result is stored in the
storing unit 110 for storing the retrieving result (in the step 207). In the case of displaying and/or printing the comparing result (in the step 208), it is outputted by theoutput unit 111 for displaying and/or printing the retrieving result (in the step 209). - The process of the above mentioned
steps 406 to 209 is executed for all of theinformation 308A of the unique identification of the compressed moving image extracted from the other moving image data (in the step 410). - The apparatus and the method for retrieving the moving image according to the invention are explained the above.
- The experimental system shown in
FIG. 46 was mounted, and the experiment testing the effectiveness of this invention was executed. In the following, the result of the experiment is shown. - In the experiment for retrieving the television commercials that are 152 in number and 37 in type from the television broadcasting of 26 hours 15 minutes which is ground wave television broadcasting, the number of omission was 0 and the number of over-detection was 0.
- In the above mentioned methods and apparatuses for retrieving the moving image, which are referred from in the first embodiment to in the twenty-third embodiment, a process for dividing an image into a plurality of frames is executed. It is also possible to execute a process for dividing each of the frames into “n” (“n” is a
natural number 2 or over) regions in addition to the above mentioned process. In this case, the information of the unique identification of the moving image and/or the information of the unique identification of the compressed moving image for identifying the moving image are extracted respectively from the divided regions, and then, the extracted information is applied to theinformation 105 of the unique identification of the moving image and/or theinformation unit 109 for retrieving the information of the unique identification of the moving image and/or a retrievingunit 309 for retrieving the information of the unique identification of the compressed moving image retrieve the identical information with theinformation 105 and or theinformation information 108 of the unique identification of the moving image and/or theinformation information 105 and/or theinformation information 105 and/or theinformation - And more, by dividing each of the frames into “n” regions and extracting the
information 105 and/or theinformation information 105 and/or theinformation - Moreover, when the
information 105 and/or theinformation information 105 and/or theinformation - Until now, the methods and the apparatuses for retrieving the moving image, which are referred from in the first embodiment to in the twenty-third embodiment, are explained. A program for executing processes of the above mentioned methods for retrieving the moving image may be executed by a computer system.
- Still, in the above mentioned embodiments according to the invention, although either the uncompressed moving image data or the compressed moving image data was applied as the object moving image data for retrieving, both the uncompressed moving image data and the compressed moving image data may be applied as the object moving image data for retrieving.
- And, in the above mentioned embodiments according to the invention, the
information information - In addition, when the
information unit information - Although the invention has been described in its preferred form with a certain degree of particularity, it is understood that the present disclosure of the preferred form has been changed in the details of construction and the combination and arrangement of parts may be resorted to without departing from the spirit and the scope of the invention as hereinafter claimed.
Claims (49)
1-57. (canceled)
58. An apparatus for retrieving a moving image to retrieve a pre-determined moving image from an object moving image, comprising:
first extracting means for extracting information of the feature quantity of each frame from the pre-determined moving image;
second extracting means for extracting information of a unique identification created by arranging one or more physical quantities of the information of the feature quantity on a time axis; and
retrieving means for retrieving a position of the pre-determined moving image on the time axis from the object moving image by comparing the arrangement of the information of the unique identification of the pre-determined moving image, with that of the information of the unique identification prepared in accordance with the object moving image.
59. The apparatus for retrieving the moving image of claim 58 , further comprising:
detecting means for detecting cut points of the moving image from the pre-determined moving image and dividing the pre-determined moving image into a plurality of shots at each of the cut points, wherein:
the second extracting means extracts the information of the unique identification created by arranging one or more physical quantities of the information of the feature quantity at each of the shots divided by the detecting means; and
the retrieving means retrieves the pre-determined moving image from the object moving image by comparing the arrangement of the information of the unique identification of the pre-determined moving image, with that of the information of the unique identification prepared in accordance with the object moving image.
60. The apparatus for retrieving the moving image of claim 58 , wherein
the second extracting means extracts the information of the unique identification created by arranging one or more expanded or reduced physical quantities of the information of the physical quantity on a time axis.
61. The apparatus for retrieving the moving image of claim 58 , wherein
the first extracting means extracts the information as a block unit of M×N pixels, where M and N are natural numbers, from the pre-determined moving image data, as the information of the feature quantity.
62. The apparatus for retrieving the moving image of claim 58 , wherein
one or more physical quantities of the information of the feature quantity are arranged on the time axis making a horizontal line of a graph represent time.
63. The apparatus for retrieving the moving image of claim 58 , wherein
the information of the unique identification prepared in accordance with the object moving image is a certain time duration.
64. The apparatus for retrieving the moving image of claim 58 , wherein
the retrieving means compares an expanded or reduced value of the information of the unique identification which is extracted from the pre-determined moving image, with an expanded or reduced value of the information of the unique identification which is prepared in accordance with the object moving image, on the time axis.
65. The apparatus for retrieving the moving image of claim 58 , wherein
the second extracting means extracts a value as the information of the unique identification of pre-determined moving image by arranging all information of the feature quantity, or some information of the feature quantity extracted by uniform or irregular intervals, on the time axis, from the pre-determined moving image.
66. The apparatus for retrieving the moving image of claim 58 , wherein
the second extracting means extracts a value as the information of the unique identification of object moving image prepared in accordance with the object image by arranging all information of the feature quantity, or some information of the feature quantity extracted by uniform or irregular intervals, on the time axis, from the object moving image.
67. The apparatus for retrieving the moving image of claim 58 , wherein
the information of the feature quantity of each frame extracted by the first extracting means is physical information of the feature quantity obtained from information of luminance and color difference (Y, Cb, Cr) or information of color (R, G, B) of each frame.
68. The apparatus for retrieving the moving image of claim 67 , wherein
the information of the feature quantity of each frame extracted by the first extracting means is one or more transformed values obtained from the information of luminance and color difference (Y, Cb, Cr) or the information of color (R, G, B) of each frame.
69. The apparatus for retrieving the moving image of claim 68 , wherein
the transformed value obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame is one or more of a value, a mean value, a total value, and a difference value of the physical information of the feature quantity obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame; and
the transformed value obtained from the information of color (R, G, B) of each frame is one or more of a value, a mean value, a total value, and a difference value of the physical information of the feature quantity obtained from the information of color (R, G, B) of each frame.
70. The apparatus for retrieving the moving image of claim 68 , wherein
the transformed value obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame is one or more of a value, a mean value, a total value, and a difference value of weighted physical information of the feature quantity obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame; and
the transformed value obtained from the information of color (R, G, B) of each frame is one or more of the value, the mean value, the total value, and the difference value of the weighted physical information of the feature quantity obtained from the information of color (R, G, B) of each frame.
71. The apparatus for retrieving the moving image of claim 69 , wherein
the transformed value obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame is a quantified value of the physical information of the feature quantity obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame, or a quantified value obtained by transforming the physical information of the feature quantity from the information of luminance and color difference (Y, Cb, Cr) of each frame; and
the transformed value obtained from the information of the information of color (R, G, B) of each frame is a quantified value of the physical information of the feature quantity obtained from the information of the information of color (R, G, B) of each frame, or a quantified value obtained by transforming the physical information of the feature quantity from the information of the information of color (R, G, B) of each frame.
72. The apparatus for retrieving the moving image of claim 58 , wherein
the first extracting means extracts information of a feature quantity of a compressed moving image of each frame by executing a Discrete Cosine Transform operation or from a coded stream.
73. The apparatus for retrieving the moving image of claim 72 , wherein
the first extracting means extracts Discrete Cosine Transform coefficients of each frame or information of motion compensation of each frame, as the information of the feature quantity of the compressed moving image.
74. The apparatus for retrieving the moving image of claim 73 , wherein
the information of the feature quantity of the compressed moving image of each frame extracted by the first extracting means is one of a value, a mean value, a total value, and a difference value of physical information of the feature quantity of the compressed moving image obtained from the Discrete Cosine Transform coefficients of each frame, and a vector or a value of the vector which is at least one of a motion vector, a mean motion vector between frames before and after, a total motion vector, a difference motion vector, and a motion vector on a whole frame obtained from the information of the motion compensation.
75. The apparatus for retrieving the moving image of claim 73 , wherein
the information of the feature quantity of the compressed moving image of each frame extracted by the first extracting means is one of a value, a mean value, a total value, and a difference value of weighted physical information of the feature quantity of the compressed moving image obtained from the Discrete Cosine Transform coefficients of each frame, and a vector or a value of the vector which is at least one of a motion vector, a mean motion vector between frames before and after, a total motion vector, a difference motion vector, and a motion vector on a whole frame obtained from weighted information of the motion compensation.
76. The apparatus for retrieving the moving image of claim 72 , wherein
the first extracting means extracts only a direct current element of the Discrete Cosine Transform coefficients of each frame, as the information of the feature quantity of the compressed moving image.
77. The apparatus for retrieving the moving image of claim 73 , wherein
the information of the feature quantity of the compressed moving image of each frame extracted by the first extracting means is a quantified value of one of the Discrete Cosine Transform coefficients, a value of the motion compensation, and a value obtained from the value of the motion compensation.
78. The apparatus for retrieving the moving image of claim 58 , wherein
the information of the feature quantity of the compressed moving image of each frame extracted by the first extracting means is the combination of physical information of the feature quantity of each frame which is obtained from information of luminance and color difference (Y, Cb, Cr) or information of color (R, G, B) of each frame, and information of a feature quantity of a compressed moving image which is obtained by executing a Discrete Cosine Transform operation or by extracting from a coded stream.
79. An apparatus for monitoring television commercials to retrieve a pre-determined moving image such as a television commercial from an object moving image such as the television broadcast and a stored moving image, comprising:
first extracting means for extracting information of a feature quantity of each frame from the pre-determined moving image;
second extracting means for extracting information of a unique identification created by arranging one or more physical quantities of the information of the feature quantity on a time axis;
retrieving means for retrieving a position of the pre-determined moving image on the time axis from the object moving image by comparing the arrangement of the information of the unique identification of the pre-determined moving image, with that of the information of the unique identification prepared in accordance with the object moving image; and
displaying means for displaying a result of retrieving by the retrieving means.
80. The apparatus for monitoring television commercials of claim 79 , further comprising storing means for storing the result of retrieving by the retrieving means, wherein
the displaying means displays the stored result stored by the storing means.
81. A method for retrieving a moving image to retrieve a pre-determined moving image from an object moving image, comprising:
(A) extracting information of a feature quantity of each frame from the pre-determined moving image;
(B) extracting information of a unique identification created by arranging one or more physical quantities of the information of the feature quantity on a time axis; and
(C) retrieving a position of the pre-determined moving image on the time axis from the object moving image by comparing the arrangement of the information of the unique identification of the pre-determined moving image, with that of the information of the unique identification prepared in accordance with the object moving image.
82. The method for retrieving the moving image of claim 81 , further comprising:
(D) detecting cut points of the moving image from the pre-determined moving image and dividing the pre-determined moving image into a plurality of shots at each of the cut points, wherein:
the step (B) extracts the information of the unique identification created by arranging one or more physical quantities of the information of the feature quantity at each of the shots divided by the step (D); and
the step (C) retrieves the pre-determined moving image from the object moving image by comparing the arrangement of the information of the unique identification of the pre-determined moving image, with that of the information of the unique identification prepared in accordance with the object moving image.
83. The method for retrieving the moving image of claim 81 , wherein
the step (B) extracts the information of the unique identification created by arranging one or more expanded or reduced physical quantities of the information of the physical quantity on a time axis.
84. The method for retrieving the moving image of claim 81 , wherein
the step (A) extracts the information as a block unit of M×N pixels, where M and N are natural numbers, from the pre-determined moving image data, as the information of the feature quantity.
85. The method for retrieving the moving image of claim 81 , wherein
one or more physical quantities of the information of the feature quantity are arranged on the time axis making a horizontal line of a graph represent time.
86. The method for retrieving the moving image of claim 81 , wherein
the information of the unique identification prepared in accordance with the object moving image is a certain time duration.
87. The method for retrieving the moving image of claim 81 , wherein
the step (C) compares an expanded or reduced value of the information of the unique identification which is extracted from the pre-determined moving image, with an expanded or reduced value of the information of the unique identification which is prepared in accordance with the object moving image, on the time axis.
88. The method for retrieving the moving image of claim 81 , wherein
the step (B) extracts a value as the information of the unique identification of pre-determined moving image by arranging all information of the feature quantity, or some information of the feature quantity extracted by uniform or irregular intervals, on the time axis, from the pre-determined moving image.
89. The method for retrieving the moving image of claim 81 , wherein
the step (B) extracts a value as the information of the unique identification of the object moving image prepared in accordance with the object image by arranging all information of the feature quantity, or some information of the feature quantity extracted by uniform or irregular intervals, on the time axis, from the object moving image.
90. The method for retrieving the moving image of claim 81 , wherein
the information of the feature quantity of each frame extracted by the step (A) is physical information of the feature quantity obtained from information of luminance and color difference (Y, Cb, Cr) or the information of color (R, G, B) of each frame.
91. The method for retrieving the moving image of claim 90 , wherein
the information of the feature quantity of each frame extracted by the step (A) is one or more transformed values obtained from the information of luminance and color difference (Y, Cb, Cr) or the information of color (R, G, B) of each frame.
92. The method for retrieving the moving image of claim 91 , wherein
the transformed value obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame is one or more of a value, a mean value, a total value, and a difference value of the physical information of the feature quantity obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame; and
the transformed value obtained from the information of color (R, G, B) of each frame is one or more of a value, a mean value, a total value, and a difference value of the physical information of the feature quantity obtained from the information of color (R, G, B) of each frame.
93. The method for retrieving the moving image of claim 91 , wherein
the transformed value obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame is one or more of a value, a mean value, a total value, and a difference value of weighted physical information of the feature quantity obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame; and
the transformed value obtained from the information of color (R, G, B) of each frame is one or more of a value, a mean value, a total value, and a difference value of weighted physical information of the feature quantity obtained from the information of color (R, G, B) of each frame.
94. The method for retrieving the moving image of claim 92 , wherein
the transformed value obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame is a quantified value of the physical information of the feature quantity obtained from the information of luminance and color difference (Y, Cb, Cr) of each frame, or a quantified value obtained by transforming the physical information of the feature quantity from the information of luminance and color difference (Y, Cb, Cr) of each frame; and
the transformed value obtained from the information of the information of color (R, G, B) of each frame is a quantified value of the physical information of the feature quantity obtained from the information of color (R, G, B) of each frame, or a quantified value obtained by transforming the physical information of the feature quantity from the information of the information of color (R, G, B) of each frame.
95. The method for retrieving the moving image of claim 81 , wherein
step (A) extracts the information of the feature quantity of the compressed moving image of each frame by executing a discrete Cosine Transform operation or extracting from a coded stream.
96. The method for retrieving the moving image of claim 95 , wherein
step (A) extracts the Discrete Cosine Transform coefficients of each frame or the information of motion compensation of each frame, as the information of the feature quantity of compressed moving image.
97. The method for retrieving the moving image of claim 96 , wherein
the information of the feature quantity of compressed moving image of each frame extracted by the step (A) is one of a value, a mean value, a total value, and a difference value of the physical information of the feature quantity of the compressed moving image obtained from the Discrete Cosine Transform coefficients of each frame, and a vector or a value of the vector which is at least one of a motion vector, a mean motion vector between frames before and after, a total motion vector, a difference motion vector, and a motion vector on a whole frame obtained from the information of the motion compensation.
98. The method for retrieving the moving image of claim 96 , wherein
the information of the feature quantity of compressed moving image of each frame extracted by step (A) is one of a value, a mean value, a total value, and a difference value of weighted physical information of the feature quantity of the compressed moving image obtained from Discrete Cosine Transform coefficients of each frame, and a vector or a value of the vector which is at least one of a motion vector, a mean motion vector between frames before and after, a total motion vector, a difference motion vector, and a motion vector on a whole frame obtained from the weighted information of the motion compensation.
99. The method for retrieving the moving image of claim 95 , wherein
step (A) extracts only a Direct Current element of the Discrete Cosine Transform coefficients of each frame, as the information of the feature quantity of the compressed moving image.
100. The method for retrieving the moving image of claim 96 , wherein
the information of the feature quantity of the compressed moving image of each frame extracted by step (A) is a quantified value of one of the Discrete Cosine Transform coefficients, a value of the motion compensation, and a value obtained from the value of the motion compensation.
101. The method for retrieving the moving image of claim 81 , wherein
the information of the feature quantity of the compressed moving image of each frame extracted by the step (A) is the combination of the physical information of the feature quantity of each frame which is obtained from information of luminance and color difference (Y, Cb, Cr) or information of color (R, G, B) of each frame, and information of a feature quantity of a compressed moving image which is obtained by executing a Discrete Cosine Transform operation or by extracting from a coded stream.
102. A method for monitoring television commercials to retrieve a pre-determined moving image such as a television commercial from an object moving image such as a television broadcast and stored moving image, comprising:
(A) extracting information of a feature quantity of each frame from the pre-determined moving image;
(B) extracting information of a unique identification created by arranging one or more physical quantities of the information of the feature quantity on a time axis;
(C) retrieving a position of the pre-determined moving image on the time axis from the object moving image by comparing the arrangement of the information of the unique identification of the pre-determined moving image, with that of the information of the unique identification prepared in accordance with the object moving image; and
(E) displaying the result of retrieving by step (C).
103. The method for monitoring television commercials of claim 102 , further comprising a step (F) of storing the result of retrieving by step (C), wherein step(E) displays the stored result stored by step (F).
104. Program products for retrieving a moving image to retrieve a pre-determined moving image from an object moving image, which is executed by a computer system, comprising:
(A) extracting information of a feature quantity of each frame from the pre-determined moving image;
(B) extracting information of a unique identification created by arranging one or more physical quantities of the information of the feature quantity on a time axis; and
(C) retrieving a position of the pre-determined moving image on the time axis from the object moving image by comparing the arrangement of the information of the unique identification of the pre-determined moving image with that of the information of the unique identification prepared in accordance with the object moving image.
105. Program products for monitoring television commercials to retrieve a pre-determined moving image such as a television commercial from an object moving image such as the television broadcast and a stored moving image, which is executed by computer system, comprising:
(A) extracting information of a feature quantity of each frame from the pre-determined moving image;
(B) extracting information of a unique identification created by arranging one or more physical quantities of the information of the feature quantity on a time axis;
(C) retrieving a position of the pre-determined moving image on the time axis from the object moving image by comparing the arrangement of the information of the unique identification of the pre-determined moving image, with that of the information of the unique identification prepared in accordance with the object moving image; and
(E) displaying the result of retrieving by step (C).
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/866,164 US20060002463A1 (en) | 1998-10-23 | 2004-08-06 | Method, apparatus and program products for retrieving moving image |
US12/249,682 US20090049103A1 (en) | 1998-10-23 | 2008-10-10 | Method, apparatus and program products for retrieving moving image |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JPHEI.10-319890 | 1998-10-23 | ||
JP31989098 | 1998-10-23 | ||
US09/421,264 US6792043B1 (en) | 1998-10-23 | 1999-10-20 | Method, apparatus and program products for retrieving moving image |
US10/866,164 US20060002463A1 (en) | 1998-10-23 | 2004-08-06 | Method, apparatus and program products for retrieving moving image |
Related Parent Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/421,264 Continuation US6792043B1 (en) | 1998-10-23 | 1999-10-20 | Method, apparatus and program products for retrieving moving image |
US09/421,264 Division US6792043B1 (en) | 1998-10-23 | 1999-10-20 | Method, apparatus and program products for retrieving moving image |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/249,682 Division US20090049103A1 (en) | 1998-10-23 | 2008-10-10 | Method, apparatus and program products for retrieving moving image |
Publications (1)
Publication Number | Publication Date |
---|---|
US20060002463A1 true US20060002463A1 (en) | 2006-01-05 |
Family
ID=18115384
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/421,264 Expired - Lifetime US6792043B1 (en) | 1998-10-23 | 1999-10-20 | Method, apparatus and program products for retrieving moving image |
US10/866,164 Abandoned US20060002463A1 (en) | 1998-10-23 | 2004-08-06 | Method, apparatus and program products for retrieving moving image |
US12/249,682 Abandoned US20090049103A1 (en) | 1998-10-23 | 2008-10-10 | Method, apparatus and program products for retrieving moving image |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/421,264 Expired - Lifetime US6792043B1 (en) | 1998-10-23 | 1999-10-20 | Method, apparatus and program products for retrieving moving image |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/249,682 Abandoned US20090049103A1 (en) | 1998-10-23 | 2008-10-10 | Method, apparatus and program products for retrieving moving image |
Country Status (3)
Country | Link |
---|---|
US (3) | US6792043B1 (en) |
EP (1) | EP0996289A3 (en) |
JP (1) | JP5116788B2 (en) |
Families Citing this family (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6792043B1 (en) * | 1998-10-23 | 2004-09-14 | Telecommunications Advancement Organization Of Japan | Method, apparatus and program products for retrieving moving image |
JP3738631B2 (en) * | 1999-09-27 | 2006-01-25 | 三菱電機株式会社 | Image search system and image search method |
US7106906B2 (en) * | 2000-03-06 | 2006-09-12 | Canon Kabushiki Kaisha | Moving image generation apparatus, moving image playback apparatus, their control method, and storage medium |
DE60206478T3 (en) | 2001-01-03 | 2018-11-29 | Nice Systems Ltd. | CONTENT BASED STORAGE MANAGEMENT |
JP4783985B2 (en) * | 2001-02-28 | 2011-09-28 | 日本電気株式会社 | Video processing apparatus, video display apparatus, video processing method used therefor, and program thereof |
US20060242568A1 (en) * | 2005-04-26 | 2006-10-26 | Xerox Corporation | Document image signature identification systems and methods |
CN101185341B (en) * | 2005-05-30 | 2010-07-14 | 松下电器产业株式会社 | Recording/reproducing apparatus and integrated circuit |
US7435627B2 (en) * | 2005-08-11 | 2008-10-14 | International Business Machines Corporation | Techniques for providing decoupling capacitance |
US8594429B2 (en) * | 2005-10-11 | 2013-11-26 | Hewlett-Packard Development Company, L.P. | Method and apparatus for processing a video stream |
US20080002225A1 (en) * | 2006-06-27 | 2008-01-03 | Masajiro Iwasaki | Printing control method, printing control device, printing sytem, terminal device, program, and recording medium |
US8315466B2 (en) * | 2006-12-22 | 2012-11-20 | Qualcomm Incorporated | Decoder-side region of interest video processing |
US8068693B2 (en) | 2007-07-18 | 2011-11-29 | Samsung Electronics Co., Ltd. | Method for constructing a composite image |
US8335251B2 (en) | 2009-01-23 | 2012-12-18 | Nec Corporation | Video signature extraction device |
WO2010084738A1 (en) | 2009-01-23 | 2010-07-29 | 日本電気株式会社 | Collation weighting information extracting device |
US8224157B2 (en) * | 2009-03-30 | 2012-07-17 | Electronics And Telecommunications Research Institute | Method and apparatus for extracting spatio-temporal feature and detecting video copy based on the same in broadcasting communication system |
KR20120030813A (en) * | 2010-09-20 | 2012-03-29 | 삼성전자주식회사 | Method of processing data and display apparatus performing the same |
JP5406813B2 (en) * | 2010-10-05 | 2014-02-05 | 株式会社ソニー・コンピュータエンタテインメント | Panorama image display device and panorama image display method |
EP2781057B1 (en) * | 2011-11-18 | 2015-10-21 | Telefonaktiebolaget L M Ericsson (PUBL) | Data distribution platform |
JP2017117089A (en) * | 2015-12-22 | 2017-06-29 | ローム株式会社 | Sensor node, sensor network system, and monitoring method |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5485611A (en) * | 1994-12-30 | 1996-01-16 | Intel Corporation | Video database indexing and method of presenting video database index to a user |
US5631697A (en) * | 1991-11-27 | 1997-05-20 | Hitachi, Ltd. | Video camera capable of automatic target tracking |
US5805746A (en) * | 1993-10-20 | 1998-09-08 | Hitachi, Ltd. | Video retrieval method and apparatus |
US6208693B1 (en) * | 1997-02-14 | 2001-03-27 | At&T Corp | Chroma-key for efficient and low complexity shape representation of coded arbitrary video objects |
US6400890B1 (en) * | 1997-05-16 | 2002-06-04 | Hitachi, Ltd. | Image retrieving method and apparatuses therefor |
US6400980B1 (en) * | 1996-11-05 | 2002-06-04 | Jerome Lemelson | System and method for treating select tissue in a living being |
US6516090B1 (en) * | 1998-05-07 | 2003-02-04 | Canon Kabushiki Kaisha | Automated video interpretation system |
US6792043B1 (en) * | 1998-10-23 | 2004-09-14 | Telecommunications Advancement Organization Of Japan | Method, apparatus and program products for retrieving moving image |
US7143434B1 (en) * | 1998-11-06 | 2006-11-28 | Seungyup Paek | Video description system and method |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4843562A (en) * | 1987-06-24 | 1989-06-27 | Broadcast Data Systems Limited Partnership | Broadcast information classification system and method |
JP3200784B2 (en) | 1993-07-29 | 2001-08-20 | キヤノン株式会社 | Moving image search method and apparatus |
JP2971724B2 (en) | 1994-02-22 | 1999-11-08 | ケイディディ株式会社 | Video cut point detection device |
JPH07152779A (en) | 1993-11-29 | 1995-06-16 | Toshiba Corp | Processing method for detecting moving picture index and moving picture processor having moving picture index detection processing function |
US5835163A (en) | 1995-12-21 | 1998-11-10 | Siemens Corporate Research, Inc. | Apparatus for detecting a cut in a video |
US5870754A (en) | 1996-04-25 | 1999-02-09 | Philips Electronics North America Corporation | Video retrieval of MPEG compressed sequences using DC and motion signatures |
JP3198980B2 (en) * | 1996-10-22 | 2001-08-13 | 松下電器産業株式会社 | Image display device and moving image search system |
-
1999
- 1999-10-20 US US09/421,264 patent/US6792043B1/en not_active Expired - Lifetime
- 1999-10-22 EP EP99120814A patent/EP0996289A3/en not_active Ceased
-
2004
- 2004-08-06 US US10/866,164 patent/US20060002463A1/en not_active Abandoned
-
2008
- 2008-10-10 US US12/249,682 patent/US20090049103A1/en not_active Abandoned
-
2010
- 2010-03-05 JP JP2010049058A patent/JP5116788B2/en not_active Expired - Fee Related
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5631697A (en) * | 1991-11-27 | 1997-05-20 | Hitachi, Ltd. | Video camera capable of automatic target tracking |
US5805746A (en) * | 1993-10-20 | 1998-09-08 | Hitachi, Ltd. | Video retrieval method and apparatus |
US6021231A (en) * | 1993-10-20 | 2000-02-01 | Hitachi, Ltd. | Video retrieval method and apparatus |
US5485611A (en) * | 1994-12-30 | 1996-01-16 | Intel Corporation | Video database indexing and method of presenting video database index to a user |
US6400980B1 (en) * | 1996-11-05 | 2002-06-04 | Jerome Lemelson | System and method for treating select tissue in a living being |
US6208693B1 (en) * | 1997-02-14 | 2001-03-27 | At&T Corp | Chroma-key for efficient and low complexity shape representation of coded arbitrary video objects |
US6400890B1 (en) * | 1997-05-16 | 2002-06-04 | Hitachi, Ltd. | Image retrieving method and apparatuses therefor |
US6516090B1 (en) * | 1998-05-07 | 2003-02-04 | Canon Kabushiki Kaisha | Automated video interpretation system |
US6792043B1 (en) * | 1998-10-23 | 2004-09-14 | Telecommunications Advancement Organization Of Japan | Method, apparatus and program products for retrieving moving image |
US7143434B1 (en) * | 1998-11-06 | 2006-11-28 | Seungyup Paek | Video description system and method |
Also Published As
Publication number | Publication date |
---|---|
EP0996289A2 (en) | 2000-04-26 |
JP5116788B2 (en) | 2013-01-09 |
EP0996289A3 (en) | 2002-07-17 |
US6792043B1 (en) | 2004-09-14 |
US20090049103A1 (en) | 2009-02-19 |
JP2010148154A (en) | 2010-07-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20090049103A1 (en) | Method, apparatus and program products for retrieving moving image | |
KR100452064B1 (en) | Apparatus for processing digital video | |
EP1211644B1 (en) | Method for describing motion activity in video | |
US5774593A (en) | Automatic scene decomposition and optimization of MPEG compressed video | |
US7003038B2 (en) | Activity descriptor for video sequences | |
US6778708B1 (en) | Compressed bit-stream segment identification and descriptor | |
US6823011B2 (en) | Unusual event detection using motion activity descriptors | |
US7352877B2 (en) | Digital-watermark-embedding and picture compression unit | |
US20050002569A1 (en) | Method and apparatus for processing images | |
KR101163774B1 (en) | Device and process for video compression | |
Fernando et al. | Fade-in and fade-out detection in video sequences using histograms | |
US20060147186A1 (en) | Image searching device | |
JP2000194727A (en) | Device and method for retrieving moving image and recording medium recording moving image retrieval program | |
US7656951B2 (en) | Digital video processing method and apparatus thereof | |
JP4167245B2 (en) | Digital video processing method and apparatus | |
EP1065877A1 (en) | Segment identification for compressed video bit-streams | |
WO2004097744A1 (en) | Image processing method and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: NATIONAL INSTITUTE OF INFORMATION AND COMMUNICATIO Free format text: CHANGE OF NAME;ASSIGNOR:TELECOMMUNICATIONS ADVANCEMENT ORGANIZATION OF JAPAN;REEL/FRAME:015452/0177 Effective date: 20040401 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |