WO2008079505B1 - Method and apparatus for hybrid audio-visual communication - Google Patents

Method and apparatus for hybrid audio-visual communication

Info

Publication number
WO2008079505B1
WO2008079505B1 PCT/US2007/082598 US2007082598W WO2008079505B1 WO 2008079505 B1 WO2008079505 B1 WO 2008079505B1 US 2007082598 W US2007082598 W US 2007082598W WO 2008079505 B1 WO2008079505 B1 WO 2008079505B1
Authority
WO
WIPO (PCT)
Prior art keywords
stream
video stream
avatar control
media content
accordance
Prior art date
Application number
PCT/US2007/082598
Other languages
French (fr)
Other versions
WO2008079505A3 (en
WO2008079505A2 (en
Inventor
Renxiang Li
Carl M Danielsen
Faisal Ishtiaq
Jay J Williams
Original Assignee
Motorola Inc
Renxiang Li
Carl M Danielsen
Faisal Ishtiaq
Jay J Williams
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Motorola Inc, Renxiang Li, Carl M Danielsen, Faisal Ishtiaq, Jay J Williams filed Critical Motorola Inc
Publication of WO2008079505A2 publication Critical patent/WO2008079505A2/en
Publication of WO2008079505A3 publication Critical patent/WO2008079505A3/en
Publication of WO2008079505B1 publication Critical patent/WO2008079505B1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M3/00—Automatic or semi-automatic exchanges
    • H04M3/42—Systems providing special services or facilities to subscribers
    • H04M3/56—Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities
    • H04M3/567—Multimedia conference systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/70—Media network packetisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M3/00—Automatic or semi-automatic exchanges
    • H04M3/22—Arrangements for supervision, monitoring or testing
    • H04M3/2227—Quality of service monitoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/142—Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/57—Arrangements for indicating or recording the number of the calling subscriber at the called subscriber's set
    • H04M1/575—Means for retrieving and displaying personal data about calling party
    • H04M1/576—Means for retrieving and displaying personal data about calling party associated with a pictorial or graphical representation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72427—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality for supporting games or graphical animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M3/00—Automatic or semi-automatic exchanges
    • H04M3/42—Systems providing special services or facilities to subscribers
    • H04M3/56—Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities
    • H04M3/563—User guidance or feature selection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/16—Central resource management; Negotiation of resources or communication parameters, e.g. negotiating bandwidth or QoS [Quality of Service]
    • H04W28/18—Negotiating wireless communication parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W84/00—Network topologies
    • H04W84/02—Hierarchically pre-organised networks, e.g. paging networks, cellular networks, WLAN [Wireless Local Area Network] or WLL [Wireless Local Loop]
    • H04W84/04—Large scale networks; Deep hierarchical networks
    • H04W84/042—Public Land Mobile systems, e.g. cellular systems

Abstract

A method and apparatus for providing communication between a sending terminal and one or more receiving terminals in a communication network. The media content of a signal transmitted by the sending terminal is detected and one or more of a voice stream, an avatar control parameter stream and a video stream are generated from the media content. At least one of the voice stream, the avatar control parameter stream and the video stream are selected as an output to be transmitted to the receiving terminal. The network server may be operable to generate synthetic video from the voice input, a natural video input and/or incoming avatar control parameters. Figure 7 is a flow chart of a method for providing hybrid audio visual communication consistent with some embodiments of the invention.

Claims

AMENDED CLAIMS received by the International Bureau on 31 July 2008 (31.07.2008)We claim:
1. A method for providing communication between a sending terminal and at least one receiving terminal in a communication network, the method comprising: detecting media content of a signal transmitted by the sending terminal; generating, from the media content, a voice stream, an avatar control parameter stream and a video stream; selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream; and transmitting the selected output to the at least one receiving terminal.
2. A method in accordance with claim 1, wherein the media content comprises a voice stream and wherein generating an avatar control parameter stream from the media content comprises detecting features in the voice stream that correspond to visemes and generating avatar control parameters representative of the visemes.
3. A method in accordance with claim 2, wherein generating a video stream from the media content comprises: rendering images using the avatar control parameters; and encoding the rendered images as the video stream.
4. A method in accordance with claim 1, wherein the media content comprises a video stream and wherein generating an avatar control parameter stream from the media content comprises: detecting facial expressions in video images contained in the video stream; and encoding the facial expressions as avatar control parameters.
5. A method in accordance with claim I5 wherein the media content comprises a video stream and wherein generating an avatar control parameter stream from the media content comprises: detecting gestures in video images of the video stream; and encoding the gestures as avatar control parameters.
6. A method in accordance with claim 1 wherein the media content comprises a natural video stream, the method further comprising detecting facial expressions in video images of the natural video stream; encoding the facial expressions as avatar control parameters; rendering images using the avatar control parameters; encoding the rendered images as a synthetic video stream; and selecting, as output, at least one of the voice stream, the avatar control parameter stream, the natural video stream, and the synthetic video stream.
7. A method in accordance with claim 1 wherein the media content comprises a natural video stream, the method further comprising detecting gestures in video images of the natural video stream; encoding the gestures as avatar control parameters; rendering images using the avatar control parameters; encoding the rendered images as a synthetic video stream; and selecting, as output, at least one of the voice stream, the avatar control parameter stream, the natural video stream, and the synthetic video stream.
8. A method in accordance with claim 1, wherein the media content comprises an avatar parameter stream, and wherein generating a video stream from the media content comprises: rendering images using the avatar control parameter stream; and encoding the rendered images as a synthetic video stream.
9. A method in accordance with claim 1 , wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon a preference of the user of the sending terminal.
10. A method in accordance with claim 1, wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon a preference of a user of the at least one receiving terminal.
11. A method in accordance with claim 1, wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon capabilities of the at least one receiving terminal.
12. A method in accordance with claim 1, wherein the capabilities of the at least one receiving terminal are determined by a data exchange between the at least one receiving terminal and a network server performing the method.
13. A method in accordance with claim 1, wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon a load status of a network server performing the method.
14. A method in accordance with claim 1, wherein selecting, as output, at least one of the voice stream, the avatar control parameter stream and the video stream is dependent upon the available capacity of a communication channel between the at least one receiving terminal and a network server performing the method.
PCT/US2007/082598 2006-12-21 2007-10-26 Method and apparatus for hybrid audio-visual communication WO2008079505A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/614,560 2006-12-21
US11/614,560 US20080151786A1 (en) 2006-12-21 2006-12-21 Method and apparatus for hybrid audio-visual communication

Publications (3)

Publication Number Publication Date
WO2008079505A2 WO2008079505A2 (en) 2008-07-03
WO2008079505A3 WO2008079505A3 (en) 2008-10-09
WO2008079505B1 true WO2008079505B1 (en) 2008-12-04

Family

ID=39542639

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2007/082598 WO2008079505A2 (en) 2006-12-21 2007-10-26 Method and apparatus for hybrid audio-visual communication

Country Status (2)

Country Link
US (1) US20080151786A1 (en)
WO (1) WO2008079505A2 (en)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080256452A1 (en) * 2007-04-14 2008-10-16 Philipp Christian Berndt Control of an object in a virtual representation by an audio-only device
US8180029B2 (en) * 2007-06-28 2012-05-15 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US11095583B2 (en) 2007-06-28 2021-08-17 Voxer Ip Llc Real-time messaging method and apparatus
US8346206B1 (en) * 2007-07-23 2013-01-01 At&T Mobility Ii Llc Customizable media feedback software package and methods of generating and installing the package
US8063905B2 (en) * 2007-10-11 2011-11-22 International Business Machines Corporation Animating speech of an avatar representing a participant in a mobile communication
KR101597286B1 (en) * 2009-05-07 2016-02-25 삼성전자주식회사 Apparatus for generating avatar image message and method thereof
US8878773B1 (en) 2010-05-24 2014-11-04 Amazon Technologies, Inc. Determining relative motion as input
JP6392497B2 (en) * 2012-05-22 2018-09-19 コモンウェルス サイエンティフィック アンド インダストリアル リサーチ オーガニゼーション System and method for generating video
US8970656B2 (en) * 2012-12-20 2015-03-03 Verizon Patent And Licensing Inc. Static and dynamic video calling avatars
GB2509323B (en) 2012-12-28 2015-01-07 Glide Talk Ltd Reduced latency server-mediated audio-video communication
US20140258419A1 (en) * 2013-03-05 2014-09-11 Motorola Mobility Llc Sharing content across modalities
US9094576B1 (en) * 2013-03-12 2015-07-28 Amazon Technologies, Inc. Rendered audiovisual communication
KR102169523B1 (en) * 2013-05-31 2020-10-23 삼성전자 주식회사 Display apparatus and control method thereof
GB201315142D0 (en) * 2013-08-23 2013-10-09 Ucl Business Plc Audio-Visual Dialogue System and Method
US9152377B2 (en) * 2013-08-29 2015-10-06 Thomson Licensing Dynamic event sounds
US9307191B2 (en) 2013-11-19 2016-04-05 Microsoft Technology Licensing, Llc Video transmission
KR20150068609A (en) * 2013-12-12 2015-06-22 삼성전자주식회사 Method and apparatus for displaying image information
US9614969B2 (en) * 2014-05-27 2017-04-04 Microsoft Technology Licensing, Llc In-call translation
JP6946724B2 (en) 2017-05-09 2021-10-06 ソニーグループ株式会社 Client device, client device processing method, server and server processing method
JP7173249B2 (en) * 2017-05-09 2022-11-16 ソニーグループ株式会社 CLIENT DEVICE, DISPLAY SYSTEM, CLIENT DEVICE PROCESSING METHOD AND PROGRAM
US10924710B1 (en) * 2020-03-24 2021-02-16 Htc Corporation Method for managing avatars in virtual meeting, head-mounted display, and non-transitory computer readable storage medium
US11218666B1 (en) * 2020-12-11 2022-01-04 Amazon Technologies, Inc. Enhanced audio and video capture and presentation
US11429835B1 (en) * 2021-02-12 2022-08-30 Microsoft Technology Licensing, Llc Holodouble: systems and methods for low-bandwidth and high quality remote visual communication
GB2606131A (en) * 2021-03-12 2022-11-02 Palringo Ltd Communication platform
US20230199147A1 (en) * 2021-12-21 2023-06-22 Snap Inc. Avatar call platform
US11831696B2 (en) 2022-02-02 2023-11-28 Microsoft Technology Licensing, Llc Optimizing richness in a remote meeting

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6483513B1 (en) * 1998-03-27 2002-11-19 At&T Corp. Method for defining MPEP 4 animation parameters for an animation definition interface
US6307576B1 (en) * 1997-10-02 2001-10-23 Maury Rosenfeld Method for automatically animating lip synchronization and facial expression of animated characters
US6272231B1 (en) * 1998-11-06 2001-08-07 Eyematic Interfaces, Inc. Wavelet-based facial motion capture for avatar animation
US6081278A (en) * 1998-06-11 2000-06-27 Chen; Shenchang Eric Animation object having multiple resolution format
US7039676B1 (en) * 2000-10-31 2006-05-02 International Business Machines Corporation Using video image analysis to automatically transmit gestures over a network in a chat or instant messaging session
AUPR212600A0 (en) * 2000-12-18 2001-01-25 Canon Kabushiki Kaisha Efficient video coding
JP3385320B2 (en) * 2001-03-06 2003-03-10 シャープ株式会社 Animation playback terminal, animation playback method, and program therefor
US7663628B2 (en) * 2002-01-22 2010-02-16 Gizmoz Israel 2002 Ltd. Apparatus and method for efficient animation of believable speaking 3D characters in real time
US6873854B2 (en) * 2002-02-14 2005-03-29 Qualcomm Inc. Method and an apparatus for adding a new member to an active group call in a group communication network
US7640293B2 (en) * 2002-07-17 2009-12-29 Research In Motion Limited Method, system and apparatus for messaging between wireless mobile terminals and networked computers
US7130282B2 (en) * 2002-09-20 2006-10-31 Qualcomm Inc Communication device for providing multimedia in a group communication network
US8411594B2 (en) * 2002-09-20 2013-04-02 Qualcomm Incorporated Communication manager for providing multimedia in a group communication network
US6925438B2 (en) * 2002-10-08 2005-08-02 Motorola, Inc. Method and apparatus for providing an animated display with translated speech
KR100932483B1 (en) * 2002-11-20 2009-12-17 엘지전자 주식회사 Mobile communication terminal and avatar remote control method using the same
US7283489B2 (en) * 2003-03-31 2007-10-16 Lucent Technologies Inc. Multimedia half-duplex sessions with individual floor controls
US20050030905A1 (en) * 2003-08-07 2005-02-10 Chih-Wei Luo Wireless communication device with status display
US20050041625A1 (en) * 2003-08-22 2005-02-24 Brewer Beth Ann Method and apparatus for providing media communication setup strategy in a communication network
US7308649B2 (en) * 2003-09-30 2007-12-11 International Business Machines Corporation Providing scalable, alternative component-level views
JP2005117141A (en) * 2003-10-03 2005-04-28 Nec Corp Apparatus, system and method of half-duplex communication

Also Published As

Publication number Publication date
WO2008079505A3 (en) 2008-10-09
US20080151786A1 (en) 2008-06-26
WO2008079505A2 (en) 2008-07-03

Similar Documents

Publication Publication Date Title
WO2008079505B1 (en) Method and apparatus for hybrid audio-visual communication
US7508413B2 (en) Video conference data transmission device and data transmission method adapted for small display of mobile terminals
CN102981613B (en) terminal and terminal control method
CN101594528A (en) Information processing system, messaging device, information processing method and program
TW201021576A (en) System and method for dynamic video encoding in multimedia streaming
CN101123730A (en) Apparatus and method for transmitting moving picture stream using bluetooth
CN103109528A (en) System and method for the control and management of multipoint conferences
US20070147367A1 (en) VoIP communication remote control system and remote controller thereof
WO2017050067A1 (en) Video communication method, apparatus, and system
US11914922B2 (en) Audio mixing for teleconferencing
CN102984496A (en) Processing method, device and system of video and audio information in video conference
CN102025972A (en) Mute indication method and device applied for video conference
CN105247875A (en) Distribution control system and distribution system
WO2019165960A1 (en) Media data real time transmission control method, system and storage medium
US6928087B2 (en) Method and apparatus for automatic cross-media selection and scaling
CN105392032A (en) Method and apparatus for controlling multimedia playing
JP2009065696A (en) Device, method and program for synthesizing video image
JP2006033743A5 (en)
CN110730362A (en) Low-flow video communication transmission system and method
US20070195962A1 (en) Apparatus and method for outputting audio data using wireless terminal
WO2015117383A1 (en) Method for call, terminal and computer storage medium
CN114257771A (en) Video playback method and device for multi-channel audio and video, storage medium and electronic equipment
CN104378651A (en) Dynamic encoding device and method based on bandwidth detection
EP3860151A1 (en) Audio / video capturing using audio from remote device
CN113573004A (en) Video conference processing method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 07854435

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 07854435

Country of ref document: EP

Kind code of ref document: A2