WO1994009593A1 - Apparatus and method for encoding/recovering multimedia control signals in an audio/video program - Google Patents

Apparatus and method for encoding/recovering multimedia control signals in an audio/video program Download PDF

Info

Publication number
WO1994009593A1
WO1994009593A1 PCT/GB1993/002113 GB9302113W WO9409593A1 WO 1994009593 A1 WO1994009593 A1 WO 1994009593A1 GB 9302113 W GB9302113 W GB 9302113W WO 9409593 A1 WO9409593 A1 WO 9409593A1
Authority
WO
WIPO (PCT)
Prior art keywords
signals
program
signal
video
audio
Prior art date
Application number
PCT/GB1993/002113
Other languages
French (fr)
Inventor
Joseph Daniel Long
Original Assignee
Gilbarco Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to AU51540/93A priority Critical patent/AU5154093A/en
Application filed by Gilbarco Inc. filed Critical Gilbarco Inc.
Publication of WO1994009593A1 publication Critical patent/WO1994009593A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/08—Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band, e.g. by time division
    • H04N7/087—Systems for the simultaneous or sequential transmission of more than one television signal, e.g. additional information signals, the signals occupying wholly or partially the same frequency band, e.g. by time division with signal insertion during the vertical blanking interval only
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/06—Electricity, gas or water supply
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/91—Television signal processing therefor
    • H04N5/92—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N5/9201—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving the multiplexing of an additional signal and the video signal
    • H04N5/9206—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving the multiplexing of an additional signal and the video signal the additional signal being a character code signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/91—Television signal processing therefor
    • H04N5/92—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N5/926—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback by pulse code modulation
    • H04N5/9265—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback by pulse code modulation with processing of the sound signal
    • H04N5/9267—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback by pulse code modulation with processing of the sound signal using time division multiplex of the PCM audio and PCM video signals

Definitions

  • a first aspect of the present invention relates to apparatus for encoding control
  • a second aspect relates to apparatus for recovering signals embedded in a program.
  • Multi-media systems are known as devices able to present information from a variety of information sources, such as computer displays, video displays, synthesized
  • an apparatus for encoding multi-media control signals into program signals in an audio/video program including detector means for detecting predetermined portions of the program signals and substitution means for substituting multi-media control signals into the program signals in place of the predetermined portions of the program signals.
  • This invention can be used in any audio/video reproduction system in which it
  • control information to the reproduction mechanism along with the recorded information.
  • the control information may contain the information required to synchronize multiple,
  • the information is used by a multi-media controller,
  • control typically multi-media control signals.
  • the audio component's volume or other characteristics could be varied, or the visual image could be faded out, windowed, or otherwise manipulated. Many other manipulations can also be included, as will be apparent to those of ordinary skill in the art.
  • This invention is independent of the audio/video format because the control information can be encoded either in the audio or video portion of the program. Further, the audio encoding method allows the use of commercial, standard recording and playback equipment, easing reproduction problems.
  • the detector means may be a synchronization
  • the means may be a switching means for switching the control signals into the video
  • the synchronization separator may
  • switching means may stop switching the control signals into the video signals when
  • a horizontal synchronization signal is detected.
  • the substitution may take place immediately upon detection of the start-of-frame signal, or after a predetermined delay, and may continue until either the first or a subsequent horizontal synchronization signal, as desired.
  • the switching means includes a latch having the start-of-frame input from the synchronization separator as one of a set and reset input and the horizontal synchronization signal as the other of a set and reset input and a switch having a signal routing controlled by the latch, whereby the switching means stops switching the control signals into the video signals when a horizontal synchronization signal is detected.
  • the apparatus preferably includes a control signal synchronizer to synchronize the control signals with the video signal upon detection of the start-of-frame signal.
  • This aspect of the invention is particularly useful if the video signal is a composite video signal.
  • the apparatus may include a video cassette recorder (or other recording device) coupled to the substitution means for recording the combined multi-media control signals and video signals.
  • a video cassette recorder or other recording device
  • the invention also provides a method of encoding multi-media control signals into program signals in an audio-video program including detecting predetermined portions of the program signals and substituting multi-media control signals into the program signals in place of the predetermined control signals.
  • the invention provides an apparatus for recovering multi-media control signals embedded in video signals in a video program.
  • This apparatus includes rerouting means for rerouting the control signals in the video signals to an output adapted for attachment to a multi-media controller to act on the video signals as directed to a media display.
  • the apparatus further comprises a detector means for detecting
  • the detector means is a synchronization separator to separate start-of-frame signals from the video signals and the rerouting means may be a gate for routing the control signals to the output when a start-of-frame signal is detected.
  • the synchronization separator to separate start-of-frame signals from the video signals and the rerouting means may be a gate for routing the control signals to the output when a start-of-frame signal is detected.
  • the gate also separates a horizontal synchronization signal from the video signal and the gate stops routing the control signals to the output when a horizontal synchronization signal is detected.
  • the rerouting may take place immediately upon detection of the start-of- frame signal, or after a predetermined delay, and may continue until either the first or subsequent horizontal synchronization signal, as desired.
  • This technique is particularly useful if the video signal is a composite video signal.
  • the apparatus preferably further includes a multi-media controller having the output as a first input and the video signals as a second input and control means for controlling the output of video signals from the multi-media controller in response to the control signals.
  • the invention provides a method for recovering multi-media control signals embedded in video signals in a video program.
  • the method includes detecting portions of the video signals having embedded control signals and rerouting the control signals in the video signals to an output adapted for attachment to a multi-media controller to act on the video signals as directed to a
  • the invention provides an apparatus for encoding multi-media control signals into audio signals in a program.
  • This apparatus includes a control signal modulator for modulating the control signals to a bandwidth of an audio signal, a filter for the audio signals to delete portions of the audio signals in the bandwidth, and a summing circuit to sum the modulated control signal and the filtered audio signal to form a combined control and audio signal.
  • the audio signal has a bandwidth and the bandwidth in which the control signals are modulated is an upper part of the audio bandwidth.
  • the control signal modulator may modulate the control signals in a modulation mode selected from the group consisting of frequency modulation, amplitude modulation and pulse modulation.
  • the apparatus may include a recorder to record the combined control and audio signals.
  • the invention also provides an apparatus for recovering multi-media control signals embedded in audio signals in a video program wherein the audio signals with embedded control signals are arranged to pass along two paths, a low pass filter in one path and a high pass filter in the other path, the filters having outputs.
  • One output directs an audio program having filtered-out control signals to an audio channel, and the other output directs a control signal having a filtered-out audio program to a multi ⁇ media controller channel.
  • the output of the high pass filter directs control signals to the multi ⁇
  • Figure 1 is a functional block diagram of components for encoding audio/video signals with multi-media controller signals according to a first embodiment of the invention
  • Figure 2 is a functional block diagram of components for decoding multi-media
  • Figure 3 is a functional block diagram of components for encoding audio/video signals with multi-media controller signals according to a second embodiment
  • Figure 4 is a functional block diagram of components for decoding multi-media controller signals from audio/video signals according to the second embodiment.
  • Figure 5 is an elevation view, partially broken away of a fuel dispenser according to the elevation.
  • a fuel dispenser housing 12 is provided with conventional fuel supply line 28, metering device 30, outlet hose 34 and nozzle 36.
  • the metering device 30 communicates data about the volume of fuel dispensed along line 32 to a transaction computer 40, which displays transaction information on displays 38.
  • Dispenser 12 also includes a video display screen 125, preferably an active LCD display, with an adjacent keypad 130, which permits a customer to respond to
  • the apparatus also includes keypads 42, 44, card reader 22, cash acceptor 24 and receipt discharge 26.
  • the present invention enhances the displays available to be shown on the screen 125, by providing the ability to embed multi-media control signals on a recording to be
  • the apparatus permits the control signals to be recovered to enable a multi-media controller to enhance the display to be recovered to enable
  • control information can be embedded in an unused horizontal line of a video frame, which contains no displayed information. Additional circuitry detects the control information and passes the information on to the multi-media controller of the reproduction mechanism, which then acts upon that information to switch various signal sources in and out, or otherwise perform as directed in the control information.
  • FIG. 1 A block diagram of a circuit to encode the control information in the video signal is shown in Figure 1.
  • the control information is embedded in the first horizontal line of an interlaced composite video signal.
  • the original video program 101 is directed into a sync separator circuit 107 designed to detect the horizontal sync pulse 102, and a start of first frame signal 103.
  • the start of first frame signal causes the normal video signal to be switched out by RS latch 105 and switch 108 and is replaced with the control information 104.
  • the start-of-frame signal inputs a signal to set the latch to output a Q value from the latch.
  • the horizontal sync signal resets the latch to halt outputting the Q value.
  • the Q value is applied to the switch 108, which acts to route the multi-media control signals to a common output when the latch outputs the Q value.
  • the switch routes the video signal to the common output when the latch does not output the Q value.
  • the reverse latch arrangement in which the start-of-frame signal is a reset and the horizontal sync signal is the set can readily be implemented by those of ordinary skill in the art.
  • the embodiment shown in Figure 1 causes the multi-media control signal to be recorded in the first horizontal line after the start-of-frame signal, but the invention is not so limited. There are typically 20 or so unused lines, and the circuit may include delays or counters to choose a different line or lines in which to embed the multi ⁇ media control signals. Also, the horizontal sync signal used to redirect the program
  • the first frame signal is also used to trigger the control signal's synchronizing
  • control signal circuit 106 which may be required depending on the type of control signal format selected.
  • the control signal format may be as simple as a single pulse, a series of pulses, or a digital signal in a frequency modulated or amplitude " modulated format.
  • the RS latch is reset and normal video resumes.
  • the combined video and control signals are recorded on a recorder 109, such as a video cassette recorder, a video disk, or any other suitable recording medium.
  • a recorder 109 such as a video cassette recorder, a video disk, or any other suitable recording medium.
  • a video signal player 219 is provided playing a recorded video signal with embedded control signals.
  • the signals need not be taken from a recording, but could be communicated in any suitable video transmission fashion.
  • the embedded video program 201 is directed into a sync separator circuit 207 designed to detect the horizontal sync pulse 202 and the start of first frame 203.
  • the start of first frame signal causes RS latch 205 to output a Control Signal Prompt 204
  • Control Signal Decoder 206 The output of the Control Signal Decoder is routed to the multi-media controller 208, allowing the controller to synchronize with the
  • the multi-media controller 208 may also receive inputs from other program sources, such as a computer data source 212
  • the controller acts under the direction of the control signal output of the control signal decoder to direct selected (or combined) ones of the program signals to the media display 210, which may be a television or the like.
  • control information may be
  • control signal format could be as simple as a single tone, a series of pulsed tones, or a digital signal in a frequency modulated or amplitude modulated format.
  • FIG. 3 A block diagram of a circuit to embed the control information into the audio frequency band is shown in Figure 3.
  • the control information is embedded in the upper portion of the audio frequency band.
  • the audio program 301 is routed into low pass filter 302 in order to eliminate its high frequency content.
  • the control signal input 303 is modulated into the upper portion of the band by modulator 304, using one of the formats discussed in the previous paragraph.
  • the control information is then added to the low frequency audio by summing the signals in a conventional summing circuit 305.
  • the summed audio program 306 is then passed on to a standard recording mechanism 307.
  • FIG. 4 A block diagram of a circuit for detecting control information embedded in the upper portion of the audio frequency band is shown in Figure 4. For this example, it is assumed that the audio and video information have already been separated, if required.
  • the embedded audio program 401 is routed from a standard playback mechanism 400 or other video signal source into a signal separator or T 402.
  • Low pass filter 403 then separates the audio program 404 which is routed to a conventional
  • High pass filter 406 separates the control information 407, which is routed to the control signal detector 408, which detects, demodulates and/or decodes the control information.
  • the decoded information is sent to the multi-media controller 409, allowing the controller to synchronize with the prerecorded audio program.
  • the multi-media controller 409 may also receive inputs from other program sources, such as a computer data source 412 or other signals 414.
  • the controller acts under the direction of the control signal output of the control signal detector 408 to direct selected (or combined) ones of the program signals to the media display 410, which may be a television or the like.
  • This audio encoding method would be required if the normally unused horizontal lines be in use so that the video encoding method is not available, or the particular format not employ raster scanning, such as transparent film projection, or audio only formats.
  • control information resides in an existing subset of the audio band, the many potential problems are avoided in the merging of the additional information with the program, and later, with the reproduction of the combined program.
  • the control information can be encoded in almost any current art form, such as Frequency Modulation, Amplitude Modulation, and pulse modulation.
  • a particularly preferred implementation of this invention is in the Graphic Multiline Display System under development by Gilbarco, Inc. of Greensboro, NC, assignee of this application. That product combines computer generated graphics with prerecordeaS r audio and video information. This invention enhances the quality of the display by adding control information to the audio portion of the recording, thus allowing for the smooth integration of computer graphics with prerecorded audio and video.

Abstract

Apparatus is provided for encoding multi-media control signals into program signals in an audio/video program by detecting and substituting predetermined portions of the program signals with control signals. Apparatus is further provided for separating the control signals from the combined signal. The invention is particularly applicable to a fuel dispenser.

Description

APPARATUS AND METHOD FOR ENCODING/RECOVERING MULTIMEDIA CONTROL SIGNALS IN AN AUDIO/VIDEO PROGRAM
A first aspect of the present invention relates to apparatus for encoding control
signals into a program, and a second aspect relates to apparatus for recovering signals embedded in a program.
Multi-media systems are known as devices able to present information from a variety of information sources, such as computer displays, video displays, synthesized
music, prerecorded audio, etc., in a synchronized manner. One of the general decisions to be made in multi-media development is how to coordinate the various information sources in order to form a coherent, and aesthetically pleasing format. Typical prior art multi-media systems control and coordinate their various inputs using a computer program. The computer, then, is in charge of "making decisions" as to what is to be included in a display. But, when the visual or audio programming to be played is recorded on a tape or video disk, the computer does not have data concerning the recording unless it is separately supplied to the computer as another database or the like. This adds to the cost and complexity of the system, since the separate database must be created and manipulated. Accordingly, there is a need in c the art for a multi-media control apparatus and methodology for controlling various inputs other than by a computer control.
According to a first aspect of the present invention there is provided an apparatus for encoding multi-media control signals into program signals in an audio/video program including detector means for detecting predetermined portions of the program signals and substitution means for substituting multi-media control signals into the program signals in place of the predetermined portions of the program signals.
This invention can be used in any audio/video reproduction system in which it
is desirably to allow a prerecorded audio/visual program to simultaneously convey
control information to the reproduction mechanism along with the recorded information. The control information may contain the information required to synchronize multiple,
independent media sources. The information is used by a multi-media controller,
which manipulates audio and/or video outputs based upon prerecorded or interactive
controls, typically multi-media control signals. Thus, the audio component's volume or other characteristics could be varied, or the visual image could be faded out, windowed, or otherwise manipulated. Many other manipulations can also be included, as will be apparent to those of ordinary skill in the art. This invention is independent of the audio/video format because the control information can be encoded either in the audio or video portion of the program. Further, the audio encoding method allows the use of commercial, standard recording and playback equipment, easing reproduction problems.
According to one aspect, the detector means may be a synchronization
separator to separate start-of-frame signals from the video signals and the substitution
means may be a switching means for switching the control signals into the video
signals after a start-of-frame signals is detected. The synchronization separator may
also separate horizontal synchronization signals from the video signal and the
switching means may stop switching the control signals into the video signals when
a horizontal synchronization signal is detected. The substitution may take place immediately upon detection of the start-of-frame signal, or after a predetermined delay, and may continue until either the first or a subsequent horizontal synchronization signal, as desired.
According to a preferred embodiment, the switching means includes a latch having the start-of-frame input from the synchronization separator as one of a set and reset input and the horizontal synchronization signal as the other of a set and reset input and a switch having a signal routing controlled by the latch, whereby the switching means stops switching the control signals into the video signals when a horizontal synchronization signal is detected.
The apparatus preferably includes a control signal synchronizer to synchronize the control signals with the video signal upon detection of the start-of-frame signal.
This aspect of the invention is particularly useful if the video signal is a composite video signal.
The apparatus may include a video cassette recorder (or other recording device) coupled to the substitution means for recording the combined multi-media control signals and video signals.
The invention also provides a method of encoding multi-media control signals into program signals in an audio-video program including detecting predetermined portions of the program signals and substituting multi-media control signals into the program signals in place of the predetermined control signals.
According to another aspect, the invention provides an apparatus for recovering multi-media control signals embedded in video signals in a video program. This apparatus includes rerouting means for rerouting the control signals in the video signals to an output adapted for attachment to a multi-media controller to act on the video signals as directed to a media display.
Preferably the apparatus further comprises a detector means for detecting
portions of the video signals having embedded control signals and the detector means is a synchronization separator to separate start-of-frame signals from the video signals and the rerouting means may be a gate for routing the control signals to the output when a start-of-frame signal is detected. Preferably, the synchronization separator
also separates a horizontal synchronization signal from the video signal and the gate stops routing the control signals to the output when a horizontal synchronization signal is detected. The rerouting may take place immediately upon detection of the start-of- frame signal, or after a predetermined delay, and may continue until either the first or subsequent horizontal synchronization signal, as desired.
This technique is particularly useful if the video signal is a composite video signal.
The apparatus preferably further includes a multi-media controller having the output as a first input and the video signals as a second input and control means for controlling the output of video signals from the multi-media controller in response to the control signals.
According to another aspect, the invention provides a method for recovering multi-media control signals embedded in video signals in a video program. The method includes detecting portions of the video signals having embedded control signals and rerouting the control signals in the video signals to an output adapted for attachment to a multi-media controller to act on the video signals as directed to a
media display.
According to another aspect, the invention provides an apparatus for encoding multi-media control signals into audio signals in a program. This apparatus includes a control signal modulator for modulating the control signals to a bandwidth of an audio signal, a filter for the audio signals to delete portions of the audio signals in the bandwidth, and a summing circuit to sum the modulated control signal and the filtered audio signal to form a combined control and audio signal. Typically, the audio signal has a bandwidth and the bandwidth in which the control signals are modulated is an upper part of the audio bandwidth.
The control signal modulator may modulate the control signals in a modulation mode selected from the group consisting of frequency modulation, amplitude modulation and pulse modulation.
The apparatus may include a recorder to record the combined control and audio signals.
The invention also provides an apparatus for recovering multi-media control signals embedded in audio signals in a video program wherein the audio signals with embedded control signals are arranged to pass along two paths, a low pass filter in one path and a high pass filter in the other path, the filters having outputs. One output directs an audio program having filtered-out control signals to an audio channel, and the other output directs a control signal having a filtered-out audio program to a multi¬ media controller channel.
Typically, the output of the high pass filter directs control signals to the multi¬
media controller channel.
One embodiment of the invention is hereinbelow described by way of example only, with reference to the accompanying figures of which:
Figure 1 is a functional block diagram of components for encoding audio/video signals with multi-media controller signals according to a first embodiment of the invention;
Figure 2 is a functional block diagram of components for decoding multi-media
controller signals from audio/video signals according to the first embodiment;
Figure 3 is a functional block diagram of components for encoding audio/video signals with multi-media controller signals according to a second embodiment;
Figure 4 is a functional block diagram of components for decoding multi-media controller signals from audio/video signals according to the second embodiment; and
Figure 5 is an elevation view, partially broken away of a fuel dispenser according to the elevation.
As seen in Figure 5, a fuel dispenser housing 12 is provided with conventional fuel supply line 28, metering device 30, outlet hose 34 and nozzle 36. The metering device 30 communicates data about the volume of fuel dispensed along line 32 to a transaction computer 40, which displays transaction information on displays 38. Dispenser 12 also includes a video display screen 125, preferably an active LCD display, with an adjacent keypad 130, which permits a customer to respond to
inquiries from the transaction computer 40 by depressing a key. The apparatus also includes keypads 42, 44, card reader 22, cash acceptor 24 and receipt discharge 26.
The present invention enhances the displays available to be shown on the screen 125, by providing the ability to embed multi-media control signals on a recording to be
played on the screen 125. The apparatus permits the control signals to be recovered to enable a multi-media controller to enhance the display to be recovered to enable
a multi-media controller to enhance the display to the customer. Two specific examples of ways to embed and recover the control signals will now be described: Video Encoding
Should the format consist of composite video (which does not include an audio signal), the control information can be embedded in an unused horizontal line of a video frame, which contains no displayed information. Additional circuitry detects the control information and passes the information on to the multi-media controller of the reproduction mechanism, which then acts upon that information to switch various signal sources in and out, or otherwise perform as directed in the control information.
A block diagram of a circuit to encode the control information in the video signal is shown in Figure 1. For this example, the control information is embedded in the first horizontal line of an interlaced composite video signal. The original video program 101 is directed into a sync separator circuit 107 designed to detect the horizontal sync pulse 102, and a start of first frame signal 103. The start of first frame signal causes the normal video signal to be switched out by RS latch 105 and switch 108 and is replaced with the control information 104. The start-of-frame signal inputs a signal to set the latch to output a Q value from the latch. The horizontal sync signal resets the latch to halt outputting the Q value. The Q value is applied to the switch 108, which acts to route the multi-media control signals to a common output when the latch outputs the Q value. The switch routes the video signal to the common output when the latch does not output the Q value. Of course, the reverse latch arrangement in which the start-of-frame signal is a reset and the horizontal sync signal is the set can readily be implemented by those of ordinary skill in the art.
The embodiment shown in Figure 1 causes the multi-media control signal to be recorded in the first horizontal line after the start-of-frame signal, but the invention is not so limited. There are typically 20 or so unused lines, and the circuit may include delays or counters to choose a different line or lines in which to embed the multi¬ media control signals. Also, the horizontal sync signal used to redirect the program
signal to the output need not be the first one encountered.
The first frame signal is also used to trigger the control signal's synchronizing
circuit 106 which may be required depending on the type of control signal format selected. The control signal format may be as simple as a single pulse, a series of pulses, or a digital signal in a frequency modulated or amplitude "modulated format. At the first horizontal sync, the RS latch is reset and normal video resumes.
In the embodiment depicted in Figure 1 , the combined video and control signals are recorded on a recorder 109, such as a video cassette recorder, a video disk, or any other suitable recording medium.
An almost identical circuit, shown in Figure 2, may be used to recover the control information. A video signal player 219 is provided playing a recorded video signal with embedded control signals. Of course, the signals need not be taken from a recording, but could be communicated in any suitable video transmission fashion. The embedded video program 201 is directed into a sync separator circuit 207 designed to detect the horizontal sync pulse 202 and the start of first frame 203. The start of first frame signal causes RS latch 205 to output a Control Signal Prompt 204
to the Control Signal Decoder 206. The output of the Control Signal Decoder is routed to the multi-media controller 208, allowing the controller to synchronize with the
pre-recorded video program supplied directly to it. The multi-media controller 208 may also receive inputs from other program sources, such as a computer data source 212
or other signals 214. The controller acts under the direction of the control signal output of the control signal decoder to direct selected (or combined) ones of the program signals to the media display 210, which may be a television or the like.
Audio Encoding
Should the media format include audio, the control information may be
contained in an existing subset of the audio frequency band allotted for that media. This requires audio signal filtering to ensure that control signals and standard audio signals do not cross outside their allotted frequency bands. A" circuit within the reproduction mechanism is tuned to the control signal band and detects the control signals and passes them on to the reproduction mechanism. The control signal format could be as simple as a single tone, a series of pulsed tones, or a digital signal in a frequency modulated or amplitude modulated format.
A block diagram of a circuit to embed the control information into the audio frequency band is shown in Figure 3. For this example, the control information is embedded in the upper portion of the audio frequency band. The audio program 301 is routed into low pass filter 302 in order to eliminate its high frequency content. The control signal input 303 is modulated into the upper portion of the band by modulator 304, using one of the formats discussed in the previous paragraph. The control information is then added to the low frequency audio by summing the signals in a conventional summing circuit 305. The summed audio program 306 is then passed on to a standard recording mechanism 307.
A block diagram of a circuit for detecting control information embedded in the upper portion of the audio frequency band is shown in Figure 4. For this example, it is assumed that the audio and video information have already been separated, if required. The embedded audio program 401 is routed from a standard playback mechanism 400 or other video signal source into a signal separator or T 402. Low pass filter 403 then separates the audio program 404 which is routed to a conventional
audio amplifier 405, if required.
High pass filter 406 separates the control information 407, which is routed to the control signal detector 408, which detects, demodulates and/or decodes the control information. The decoded information is sent to the multi-media controller 409, allowing the controller to synchronize with the prerecorded audio program.
The multi-media controller 409 may also receive inputs from other program sources, such as a computer data source 412 or other signals 414. The controller acts under the direction of the control signal output of the control signal detector 408 to direct selected (or combined) ones of the program signals to the media display 410, which may be a television or the like.
This audio encoding method would be required if the normally unused horizontal lines be in use so that the video encoding method is not available, or the particular format not employ raster scanning, such as transparent film projection, or audio only formats.
Because the control information resides in an existing subset of the audio band, the many potential problems are avoided in the merging of the additional information with the program, and later, with the reproduction of the combined program. The control information can be encoded in almost any current art form, such as Frequency Modulation, Amplitude Modulation, and pulse modulation.
A particularly preferred implementation of this invention is in the Graphic Multiline Display System under development by Gilbarco, Inc. of Greensboro, NC, assignee of this application. That product combines computer generated graphics with prerecordeaSraudio and video information. This invention enhances the quality of the display by adding control information to the audio portion of the recording, thus allowing for the smooth integration of computer graphics with prerecorded audio and video.
" This invention is specifically designed to be used in conjunction with the inventions described in copending applications entitled "An Apparatus and Method for Displaying Video Information" of Hans Atchley and "Transaction Apparatus" of Russel Dean Leatherman and Walter Baker, both filed on even date herewith. The
disclosures of those two applications are hereby incorporated herein by reference. Those of ordinary skill in the art will recognize that the invention as disclosed may be implemented in numerous embodiments which differ from the specific disclosure hereof. All such embodiments which fall within the scope of the appended claims are deemed to be within the scope of the patent.

Claims

1. An apparatus for encoding multi-media control signals into program signals in
an audio/video program comprising
detector means for detecting predetermined portions of the program signals and substitution means for substituting multi-media control signals into the program
signals in place of the predetermined portions of the program signals.
2. An apparatus as claimed in claim 1 wherein said program signals are video signals and said detector means is a synchronization separator to separate start-of- frame signals from the video signals and said substitution means is a switching means for switching the control signals into the video signals after detection of a start-of- frame signal.
3. An apparatus as claimed in claim 2 wherein said synchronization separator also separates a horizontal synchronization signal from the video signal and said switching
means stops switching the control signals into the video signals when a horizontal synchronization signal is detected.
4. An apparatus as claimed in claim 3 wherein said switching means stops
switching the control signals into the video signals when a first horizontal synchronization signal is detected after the start-of-frame signal.
5. An apparatus as claimed in claim 2, 3 or 4 further comprising a control signal synchronizer to synchronize the control signals with the video signal upon detection of the start-of-frame signal.
6. An apparatus as claimed in claim 2, 3, 4 or 5 wherein said synchronization separator also separates a horizontal synchronization signal from the video signal and said switching means includes a latch having the start-of-frame input from the synchronization separator as one of a set and reset input and the horizontal synchronization signal as the other of a set and reset input a switch having a signal routing controlled by said latch, whereby said switching means stops switching the control signals into the video signals when a horizontal synchronization signal is detected.
7. An apparatus as claimed in any one of claims 2 to 6 wherein the video signal is a composite video signal.
8. An apparatus as claimed in any one of claims 2 to 7 further comprising a video cassette recorder coupled to said substitution means for recording the combined multi¬ media control signals and video signals.
9. An apparatus for recovering multi-media control signals embedded in program signals in an audio/video program comprising rerouting means for rerouting the control signals in the program signals to an output adapted for attachment to a multi-media controller to act on the program signals as directed to a media display.
10. An apparatus as claimed in claim 9 comprising detector means for detecting predetermined portions of the program signals having embedded control signals
wherein the program signals are video signals and said detector means is a
synchronization separator to separate start-of-frame signals from the video signals and
said rerouting means is a gate for routing the control signals to said output after a start-of-frame signal is detected.
11. An apparatus as claimed in claim 10 wherein said synchronization separator
also separates a horizontal synchronization signal from the video signal and said gate stops routing the control signals to said output when a horizontal synchronization signal is detected.
12. An apparatus as claimed in claim 11 wherein said gate stops routing the control signals to said output when a first horizontal synchronization signal is detected after the start-of-frame signal.
13. An apparatus as claimed in claims 9 to 12 wherein the video signal is a composite video signal.
14. An apparatus as claimed in claim 1 wherein the predetermined portion of the
program signal is a portion of the bandwidth of audio signals, the apparatus further comprising
a control signal modulator for modulating the control signals to a portion of the bandwidth of the audio signals, a filter for the audio signals of the program to delete portions of the audio signals of the program in the portion of the bandwidth, and a summing circuit to sum the modulated control signal and the filtered audio signal to form a combined control and audio signal.
15. An apparatus as claimed in claim 14 wherein the bandwidth in which the control signals are modulated is an upper part of the program's audio bandwidth.
16. An apparatus as claimed in claim 14 or 15 wherein said control signal modulator modulates the control signals in a modulation mode selected from the group consisting of frequency modulation, amplitude modulation and pulse modulation.
17. An apparatus as claimed in claim 14, 15 or 16 further comprising a recorder to record the combined control and audio signals.
18. An apparatus as claimed in claim 9 for recovering multi-media control signals embedded in audio signals in a program wherein the audio signals with embedded control signals are arranged to pass along two paths, the rerouting means comprising a low pass filter in one path and a high pass filter in the other path, said filters having outputs, one output directing an audio program having filtered-out control signals to an audio channel, and the other output directing a control signal having a filtered-out audio program to a multi-media controller channel.
19. An apparatus as claimed in claim 18 wherein the output of said high pass filter directs control signals to the multi-media controller channel.
20. An apparatus as claimed in any one of claims 9 to 13, 18 or 19 further
comprising a multi-media controller having the output with the control signal as a first
input and a video signal as a second input and having control means for controlling the output of the video signal from said multi-media controller in response to the
control signals.
21. A multi-media apparatus comprising apparatus for encoding control signals as claimed in any one of claims 1 to 8, 14 to 17 or 20 and apparatus for recovering control signals as claimed in any one of claims 9 to 13, 18 to 20.
22. A fuel dispenser system comprising apparatus as claimed in any preceding claim.
23. A fuel dispenser comprising apparatus as claimed in any one of claims 9 to 13, or 18 to 20.
24. A multi-media control apparatus substantially as illustrated in, and hereinbefore described with reference to, the accompanying drawings.
25. A method of encoding multi-media control signals into program signals in an
audio/video program comprising detecting predetermined portions of the program signals and substituting multi-media control signals into the program signals in place of the predetermined program signals.
26. A method for recovering multi-media control signals embedded in signals in an audio/video program comprising detecting portions of the signals having embedded control signals and rerouting the control signals in the program signals to an output adapted for attachment to a multi-media controller to act on the program signals as directed to a media display.
PCT/GB1993/002113 1992-10-13 1993-10-13 Apparatus and method for encoding/recovering multimedia control signals in an audio/video program WO1994009593A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU51540/93A AU5154093A (en) 1992-10-13 1992-10-13 Apparatus and method for encoding/recovering multimedia control signals in an audio/video program

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US07/959,844 1992-10-13
US07/959,844 US5543849A (en) 1992-10-13 1992-10-13 Synchronization of prerecorded audio/video signals with multi-media controllers

Publications (1)

Publication Number Publication Date
WO1994009593A1 true WO1994009593A1 (en) 1994-04-28

Family

ID=25502485

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB1993/002113 WO1994009593A1 (en) 1992-10-13 1993-10-13 Apparatus and method for encoding/recovering multimedia control signals in an audio/video program

Country Status (3)

Country Link
US (3) US5543849A (en)
AU (1) AU5154093A (en)
WO (1) WO1994009593A1 (en)

Families Citing this family (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0728689A (en) * 1993-07-09 1995-01-31 Hitachi Ltd Information processor
US5751824A (en) * 1994-09-23 1998-05-12 Lucent Technologies Inc. Method of stabilizing the level of an audio signal in television broadcasting
US5905865A (en) 1995-10-30 1999-05-18 Web Pager, Inc. Apparatus and method of automatically accessing on-line services in response to broadcast of on-line addresses
US20030212996A1 (en) * 1996-02-08 2003-11-13 Wolzien Thomas R. System for interconnection of audio program data transmitted by radio to remote vehicle or individual with GPS location
US5761606A (en) 1996-02-08 1998-06-02 Wolzien; Thomas R. Media online services access via address embedded in video or audio program
US20020038383A1 (en) 1999-12-23 2002-03-28 Craig Ullman Enhanced video programming system and method for incorporating and displaying retrieved integrated internet information segments
US20020049832A1 (en) 1996-03-08 2002-04-25 Craig Ullman Enhanced video programming system and method for incorporating and displaying retrieved integrated internet information segments
US6018768A (en) * 1996-03-08 2000-01-25 Actv, Inc. Enhanced video programming system and method for incorporating and displaying retrieved integrated internet information segments
US6513069B1 (en) * 1996-03-08 2003-01-28 Actv, Inc. Enhanced video programming system and method for providing a distributed community network
USD409207S (en) * 1996-04-24 1999-05-04 Alvern-Norway A/S Multipurpose utility station for use at e.g. a petrol station, providing e.g. supplies of air, oil, screen washer fluid, coolant as well as washing facilities
US5945975A (en) * 1996-04-30 1999-08-31 Dresser Ind Graphics display advertising system for a fuel dispenser
US6052629A (en) * 1997-07-18 2000-04-18 Gilbarco Inc. Internet capable browser dispenser architecture
US5890520A (en) * 1997-09-26 1999-04-06 Gilbarco Inc. Transponder distinction in a fueling environment
US6078896A (en) * 1997-11-05 2000-06-20 Marconi Commerce Systems Inc. Video identification for forecourt advertising
US6175382B1 (en) 1997-11-24 2001-01-16 Shell Oil Company Unmanned fueling facility
US5980090A (en) * 1998-02-10 1999-11-09 Gilbarco., Inc. Internet asset management system for a fuel dispensing environment
US5969691A (en) * 1998-02-10 1999-10-19 Gilbarco Inc. Fuel dispenser transponder antenna arrangement
US6571151B1 (en) * 1998-03-06 2003-05-27 Russel Dean Leatherman Wireless nozzle interface for a fuel dispenser
US8315909B1 (en) 1998-03-11 2012-11-20 West Corporation Methods and apparatus for intelligent selection of goods and services in point-of-sale commerce
US7729945B1 (en) 1998-03-11 2010-06-01 West Corporation Systems and methods that use geographic data to intelligently select goods and services to offer in telephonic and electronic commerce
US7386485B1 (en) 2004-06-25 2008-06-10 West Corporation Method and system for providing offers in real time to prospective customers
US7437313B1 (en) 1998-03-11 2008-10-14 West Direct, Llc Methods, computer-readable media, and apparatus for offering users a plurality of scenarios under which to conduct at least one primary transaction
US20020055906A1 (en) * 1998-03-11 2002-05-09 Katz Ronald A. Methods and apparatus for intelligent selection of goods and services in telephonic and electronic commerce
US7364068B1 (en) 1998-03-11 2008-04-29 West Corporation Methods and apparatus for intelligent selection of goods and services offered to conferees
US6055513A (en) 1998-03-11 2000-04-25 Telebuyer, Llc Methods and apparatus for intelligent selection of goods and services in telephonic and electronic commerce
AU726243B2 (en) * 1998-05-01 2000-11-02 Global Vending Pty Ltd Vending method and apparatus
US6697632B1 (en) 1998-05-07 2004-02-24 Sharp Laboratories Of America, Inc. Multi-media coordinated delivery system and method
US6442448B1 (en) 1999-06-04 2002-08-27 Radiant Systems, Inc. Fuel dispensing home phone network alliance (home PNA) based system
US6470288B1 (en) * 1999-06-18 2002-10-22 Tokheim Corporation Dispenser with updatable diagnostic system
US7120871B1 (en) 1999-09-15 2006-10-10 Actv, Inc. Enhanced video programming system and method utilizing a web page staging area
US7949722B1 (en) 1999-09-29 2011-05-24 Actv Inc. Enhanced video programming system and method utilizing user-profile information
US6421616B1 (en) * 2000-01-31 2002-07-16 Gilbarco Inc. Fraud detection through inference
JP4961575B2 (en) 2000-03-31 2012-06-27 オープンティービー、インコーポレイテッド System and method for regional metadata insertion
US6360138B1 (en) 2000-04-06 2002-03-19 Dresser, Inc. Pump and customer access terminal interface computer converter to convert traditional pump and customer access terminal protocols to high speed ethernet protocols
US6642966B1 (en) * 2000-11-06 2003-11-04 Tektronix, Inc. Subliminally embedded keys in video for synchronization
US20020120500A1 (en) * 2001-02-28 2002-08-29 Johnson Duwayne Retail outdoors communication kiosk system
US6708797B2 (en) * 2001-04-23 2004-03-23 Gilbarco Inc. Display enclosure having thin speaker
US7739162B1 (en) 2001-05-04 2010-06-15 West Corporation System, method, and business method for setting micropayment transaction to a pre-paid instrument
US20020178060A1 (en) * 2001-05-25 2002-11-28 Sheehan Patrick M. System and method for providing and redeeming electronic paperless coupons
US20030037340A1 (en) * 2001-08-17 2003-02-20 Koninklijke Philips Electronics N.V. Method and apparatus for managing TV broadcast content that has synchronized web applications
US20030075600A1 (en) * 2001-09-28 2003-04-24 Environ Products, Inc. Fuel dispenser having a guided customer transaction interface and the method therefor
GB0214401D0 (en) * 2002-06-21 2002-07-31 Newell Ltd Shelving system
US20040045038A1 (en) * 2002-08-29 2004-03-04 Duff John F. System and method for the synchronized activation of external devices in association with video programs
US8306908B1 (en) 2002-12-31 2012-11-06 West Corporation Methods and apparatus for intelligent selection of goods and services in telephonic and electronic commerce
US7930716B2 (en) 2002-12-31 2011-04-19 Actv Inc. Techniques for reinsertion of local market advertising in digital video from a bypass source
US8712857B1 (en) 2003-03-31 2014-04-29 Tuxis Technologies Llc Methods and apparatus for intelligent selection of goods and services in mobile commerce
US7945141B2 (en) * 2003-10-06 2011-05-17 Samsung Electronics Co., Ltd. Information storage medium including event occurrence information, and apparatus and method for reproducing the information storage medium
US7178720B1 (en) 2004-09-30 2007-02-20 West Corporation Methods, computer-readable media, and computer program product for intelligent selection of items encoded onto portable machine-playable entertainment media
US8035665B2 (en) * 2005-02-25 2011-10-11 Hewlett-Packard Development Company, L.P. Methods and systems for using control data to control a display of an image by a display device
US8713827B2 (en) * 2006-07-10 2014-05-06 David R. Gustafson Advertising device and method
US20090307730A1 (en) * 2008-05-29 2009-12-10 Mark Donaldson Media enhancement module
US10838378B2 (en) * 2014-06-02 2020-11-17 Rovio Entertainment Ltd Control of a computer program using media content

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4086005A (en) * 1976-10-26 1978-04-25 Audio-Sine, Inc. Audio-visual programmer
US4216492A (en) * 1977-04-18 1980-08-05 Axel Schmalz Method of treating television picture signals and wide-band data signals by modulating the color carrier signal
GB2060312A (en) * 1979-10-12 1981-04-29 Rca Corp Improved digital on video recording and playback system
EP0493881A2 (en) * 1990-12-11 1992-07-08 International Business Machines Corporation Bus architecture for a multimedia system
WO1992015175A2 (en) * 1991-02-26 1992-09-03 Sasktel Video store and forward on demand apparatus and method
EP0532277A2 (en) * 1991-09-09 1993-03-17 Pioneer Electronic Corporation Method of recording information on video disk
US5216500A (en) * 1991-07-15 1993-06-01 Rj Lee Group, Inc. Simultaneously recording of video image and microscope stage position data

Family Cites Families (27)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3732365A (en) * 1968-09-03 1973-05-08 Ultronic Systems Corp Selective blanking of video display
US4862268A (en) * 1980-03-31 1989-08-29 General Instrument Corporation Addressable cable television control system with video format data transmission
US4534054A (en) * 1980-11-28 1985-08-06 Maisel Douglas A Signaling system for FM transmission systems
US4426728A (en) * 1981-08-31 1984-01-17 Kahn Leonard R Multiple system AM stereo receiver and pilot signal detector
JPS602491A (en) * 1983-06-17 1985-01-08 トキコ株式会社 Lubrication system
US4750130A (en) * 1985-03-20 1988-06-07 Tokyo Tatsuno Co., Ltd. Fuel delivery display and control system
JPS6232494A (en) * 1985-08-06 1987-02-12 三洋電機株式会社 Graphic decoder circuit
JPH0670876B2 (en) * 1987-02-10 1994-09-07 ソニー株式会社 Optical disc and optical disc reproducing apparatus
US4855827A (en) * 1987-07-21 1989-08-08 Worlds Of Wonder, Inc. Method of providing identification, other digital data and multiple audio tracks in video systems
CA1301929C (en) * 1987-08-06 1992-05-26 Peter William Hollidge Interactive pump system
US4837638A (en) * 1987-08-06 1989-06-06 Fullwood John W Video tape editing system with vertical interval time code
US5019905A (en) * 1987-09-18 1991-05-28 Vicon Industries, Inc. Encoding and decoding of multiple video sources
US4814883A (en) * 1988-01-04 1989-03-21 Beam Laser Systems, Inc. Multiple input/output video switch for commerical insertion system
JPH01221070A (en) * 1988-02-29 1989-09-04 Casio Comput Co Ltd Image information processor
US4956709A (en) * 1988-03-11 1990-09-11 Pbs Enterprises, Inc. Forward error correction of data transmitted via television signals
JP2664944B2 (en) * 1988-08-12 1997-10-22 株式会社日立製作所 Video signal processing circuit
AU617588B2 (en) * 1989-02-16 1991-11-28 Caltex Oil (Australia) Pty. Limited Point of sale audio-visual advertising system
US5008749A (en) * 1989-03-25 1991-04-16 Bts Broadcast Television Systems Gmbh Method and apparatus for generating an auxiliary timing signal in the horizontal blanking interval of a video signal
US4974085A (en) * 1989-05-02 1990-11-27 Bases Burke Institute, Inc. Television signal substitution
US5027426A (en) * 1989-07-07 1991-06-25 Chiocca Jr Joseph J Signal coupling device and system
US5140420A (en) * 1990-10-05 1992-08-18 General Electric Company Information in vertical blanking interval of video sync signal
US5119503A (en) * 1991-02-19 1992-06-02 Mankovitz Roy J Apparatus and methods for broadcasting auxiliary data in an FM stereo broadcast system
US5200822A (en) * 1991-04-23 1993-04-06 National Broadcasting Company, Inc. Arrangement for and method of processing data, especially for identifying and verifying airing of television broadcast programs
JPH0561491A (en) * 1991-09-02 1993-03-12 Sanyo Electric Co Ltd Karaoke device and its recording medium
US5343239A (en) * 1991-11-20 1994-08-30 Zing Systems, L.P. Transaction based interactive television system
US5289272A (en) * 1992-02-18 1994-02-22 Hughes Aircraft Company Combined data, audio and video distribution system in passenger aircraft
KR960004813B1 (en) * 1992-10-06 1996-04-13 엘지전자주식회사 Scent occurring television receiver

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4086005A (en) * 1976-10-26 1978-04-25 Audio-Sine, Inc. Audio-visual programmer
US4216492A (en) * 1977-04-18 1980-08-05 Axel Schmalz Method of treating television picture signals and wide-band data signals by modulating the color carrier signal
GB2060312A (en) * 1979-10-12 1981-04-29 Rca Corp Improved digital on video recording and playback system
EP0493881A2 (en) * 1990-12-11 1992-07-08 International Business Machines Corporation Bus architecture for a multimedia system
WO1992015175A2 (en) * 1991-02-26 1992-09-03 Sasktel Video store and forward on demand apparatus and method
US5216500A (en) * 1991-07-15 1993-06-01 Rj Lee Group, Inc. Simultaneously recording of video image and microscope stage position data
EP0532277A2 (en) * 1991-09-09 1993-03-17 Pioneer Electronic Corporation Method of recording information on video disk

Also Published As

Publication number Publication date
US5543849A (en) 1996-08-06
AU5154093A (en) 1994-05-09
US5561715A (en) 1996-10-01
US5535130A (en) 1996-07-09

Similar Documents

Publication Publication Date Title
US5543849A (en) Synchronization of prerecorded audio/video signals with multi-media controllers
US5289288A (en) Method and apparatus for encoding a video signal having multi-language capabilities
JP4443578B2 (en) Information carrier
JPS62191A (en) Video information control device
SG168417A1 (en) Method and device for seamless-reproducing a bitstream containing noncontinuous system time information
ES8500492A1 (en) Interactive video disc systems.
KR960038900A (en) A recording medium in which attribute information of reproduction data and reproduction data is recorded together
EP0195639A2 (en) Improvements relating to television audience research systems
ZA200505617B (en) Fast play DVD
NL192967C (en) Video theater installation.
KR930007060B1 (en) Recoding and displaying system in vtr
US20020001448A1 (en) Selective recording and reproduction of video programs according to video programs according to video program grading
JP3106682B2 (en) Video information editing service system
GB2228593A (en) Synchronising audio and video tape players
JPS6116685A (en) Recording medium disk for recorder information
KR100403584B1 (en) Recording medium containing main contents and additional contents, recording apparatus, recording method, reproducing apparatus, and reproducing method therefor
JPH0877680A (en) Method and apparatus for controlling reproduction of still image of photo cd apparatus
JPH0317884A (en) State display system
JPH05336461A (en) Video signal outputting device
JPH0625102Y2 (en) Video output detector
JPH089322A (en) Image monitor recording and reproducing device
JPH08317329A (en) Index system
JPH0946594A (en) Superimposition display method for karaoke device
JPS5972471A (en) Reproducer
JPH08140038A (en) Magnetic recording and reproducing device

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AU CA NO NZ

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: CA