US20110249667A1

US20110249667A1 - Apparatus and method for transmitting media using either network efficient protocol or a loss tolerant transmission protocol

Info

Publication number: US20110249667A1
Application number: US12/792,680
Authority: US
Inventors: Matthew J. Ranney
Original assignee: Rebelvox LLC
Current assignee: Voxer IP LLC
Priority date: 2010-04-13
Filing date: 2010-06-02
Publication date: 2011-10-13
Also published as: WO2011129978A1; WO2011130082A1; US8924593B2; US20110252083A1; US20110252161A1

Abstract

A method and apparatus for transmitting voice media over a network where the voice media may be consumed either in a real-time mode or a time-shifted mode. The method comprising transmitting the voice media over the network using a network efficient protocol when either (i) the media is not being consumed in the real-time mode or (ii) the condition on the network is good enough to support the real-time transmission and consumption of the voice media in the real-time mode. Alternatively, the voice media is transmitted using a loss tolerant transmission protocol when the media is being consumed in the real-time mode and the condition on the network is sufficiently poor to prevent the real-time consumption of the voice media in real-time using the network efficient protocol. The apparatus, which may be a communication device or a server, implements the above-described method.

Description

This application claims the benefit of U.S. Provisional Patent Application No. 61/323,609, filed Apr. 13, 2010, entitled “Communication Services Network and Client Enabled Communication Devices,” which is incorporated herein by reference for all purposes.

BACKGROUND

1. Field of the Invention
This invention relates to communications, and more particularly, to a communication apparatus and method for transmitting media between nodes using either a network efficient transmission protocol or a loss tolerant transmission protocol, depending on (i) the condition on the network and (ii) if either the transmitted media is being consumed in real-time or in a time-shifted mode.
2. Description of Related Art
The Transmission Control Protocol or “TCP” is an example of a network efficient protocol. TCP guarantees the delivery of transmitted data between a sender and a recipient at the expense of speed. For this reason, TCP is currently the most common delivery protocol used on the Internet. A feature called “flow control” is the main reason why TCP is able to guarantee the delivery of media. Flow control determines when data needs to be re-sent and stops the flow of data until previous packets are successfully transferred. For example, when a recipient receives a defective packet or a packet is not received (i.e., a missing packet), a request for retransmission of the defective and/or missing packet is made and flow of subsequent packets is stopped until the retransmission request is satisfied. The guaranteed delivery feature of TCP is beneficial for certain applications, such as the transfer of the content of web pages, files or database information. The possibility that the flow of data may be stopped, however, makes TCP less than ideal for delivery of time critical media, such as streaming voice or video.
The User Datagram Protocol or “UDP” is an example of a loss tolerant protocol, commonly used on the Internet for streaming audio, video and other time-based media (i.e., media that changes over time). UDP is mainly concerned with the delivery of the most recently available media, as opposed to quality. To achieve the necessary delivery rate for streaming audio or video, there is no form of flow control or error correction with UDP. Without any mechanism for guaranteeing delivery, packets may be received out of order, defective, or lost altogether, possibly resulting in reduced quality of the media delivered to the recipient. As a result when the condition on the network are poor, media may be delivered at a rate sufficient for real-time consumption using UDP when TCP would otherwise be inadequate.
Conventional communication systems are typically either real-time or time-shifted. Consequently, a protocol like UDP, which is optimized for “real-time” delivery, is typically used for real-time systems, while a protocol like TCP, which is optimized for reliable delivery, is used for time-shifted systems.

SUMMARY OF THE INVENTION

The invention relates to a method and apparatus for transmitting voice media over a network where the voice media may be consumed either in a real-time mode or a time-shifted mode. The method comprising transmitting the voice media over the network using a network efficient protocol when either (i) the media is not being consumed in the real-time mode or (ii) the condition on the network is good enough to support the real-time transmission and consumption of the voice media in the real-time mode. Alternatively, the voice media is transmitted using a loss tolerant transmission protocol when the media is being consumed in the real-time mode and the condition on the network is sufficiently poor to prevent the real-time consumption of the voice media in real-time using the network efficient protocol. The apparatus, which may be a communication device or a server, implements the above-described method.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may best be understood by reference to the following description taken in conjunction with the accompanying drawings, which illustrate specific embodiments of the invention.

FIG. 1 is an architecture diagram of the communication services network according to the invention.

FIG. 2 is a block diagram of a client application used in the communication services network of the present invention.

FIG. 3 is a media flow diagram on a client device running the client application of the present invention.

FIG. 4 is a diagram of a server architecture used in the communication system of the present invention.

FIG. 5 is a flow diagram for transmitting media according to the present invention.

FIG. 6 is a timing diagram illustrating the transmission of media between sending and receiving nodes according to the present invention.

It should be noted that like reference numbers refer to like elements in the figures.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

The invention will now be described in detail with reference to various embodiments thereof as illustrated in the accompanying drawings. In the following description, specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art, that the invention may be practiced without using some of the implementation details set forth herein. It should also be understood that well known operations have not been described in detail in order to not unnecessarily obscure the invention.
The term “media” as used herein is intended to broadly mean virtually any type of media, such as but not limited to, voice, video, text, still pictures, sensor data, GPS data, or just about any other type of media, data or information.
As used herein, the term “conversation” is also broadly construed. In one embodiment, a conversation is intended to mean a thread of messages, strung together by some common attribute, such as a subject matter or topic, by name, by participants, by a user group, or some other defined criteria. In another embodiment, the messages of a conversation do not necessarily have to be tied together by some common attribute. Rather one or more messages may be arbitrarily assembled into a conversation. Thus a conversation is intended to mean two or more messages, regardless if they are tied together by a common attribute or not.

A. System Architecture

Referring to FIG. 1, a block diagram of the communication system according to the invention is shown. The system 10 includes a communication services network 12, a circuit switched (CS) network 14 such as the Public Switched Telephone Network (PSTN), a cellular or mobile phone network, a Push to Talk (PTT) network, or a combination thereof, and a gateway 16 coupling the two networks 12, 14. The communication services network 12 includes one or more servers 18. The circuit switch network 14 includes, depending on the type of network or combination of networks, one or more circuit switches 22 and possibly one or more radio towers 24.
The communication services network 12 is IP based and layered over one or more communication networks (not illustrated), such as the Public Switched Telephone Network (PSTN), a cellular network based on CDMA or GSM for example, the Internet, an intranet or private communication network, a tactical radio network, a satellite radio network, a first responder network, or any other communication network, or a combination thereof. In various embodiments, the communication services network 12 is either heterogeneous or homogeneous.
One or more legacy communication devices 26 are connected to the circuit switched network 14 through either wired or wireless connection(s) 28, as is well known in the art. In various embodiments, the legacy device(s) may include conventional land-line telephones, mobile or cellular phones, PTT radios, satellite phones or radios, desktop or mobile computers, or any combination thereof.
One or more client application 30 enabled communication devices 32 are coupled to the communication services network 12 through an IP-based network connection 34. Depending on the type of communication device 32, the connection 34 may be wired (e.g., Ethernet) or wireless (e.g., Wi-Fi, PTT, radio, satellite, CDMA, GSM, etc.). In various embodiments, the client application 30 enabled communication devices 32 may be any type of telephone, including both land-line, cellular or mobile phones, a PTT radio, satellite based communication device, any type of computer, including but not limited to desktop, laptop, note pad computers, or any other type of wired or wireless communication device.

B. Client Application Architecture

The client application 30 is a messaging application that operates in a time-shifted mode, a real-time mode, and provides the ability to seamlessly transition between the two modes. With the client application 30, both inbound and outgoing media is simultaneously and progressively stored as it is either (i) received over a network connection 34 at the communication device 32 or (ii) created on the communication device 32 and transmitted over the network connection 34. The storage of the media allows the participants to converse in a time-shifted mode, providing a user experience similar to conventional messaging systems (e.g., email or voice Short Messaging Service (SMS)). The simultaneous and progressive nature of the application

- enables the participants to converse “live” in a real-time mode, providing a user experience similar to a full-duplex telephone conversation.

The simultaneous and progressive storage of both transmitted media as it is being created and received media as it is being received further enables a host of rendering options. Such rendering options include, but are not limited to: the real-time rendering of media as the media is received over the network connection 34, pause, replay, play faster, play slower, jump backward, jump forward, catch up to the most recently received media, Catch up to Live (CTL), or jump to the most recently received media. As described in more detail below, the storage of media and certain rendering options allow the participants of a conversation to seamlessly transition the conversation from the time-shifted mode to the real-time mode and vice versa. In addition, the client application 30 is capable of supporting multiple types of media, including but not limited to, voice, video, text, still pictures, sensor data, GPS data, or just about any other type of media, data or information.
Referring to FIG. 2, a logic block diagram of the client application 30 is shown. The application 30 includes a Multiple Conversation Management System (MCMS) module 40, a Store and Stream module 42, and an interface 44 provided between the two modules. In one embodiment, the MCMS module 40 and the Store and Stream module 42 communicate through interface 44 using an encapsulation format (e.g., JSON or XML) over a transport header protocol (e.g., Hypertext Transfer Protocol or HTTP). The application 30 functionally is similar to the client application described in U.S. application Ser. Nos. 12/028,400 (U.S. Publication No. 2009/0003558), 12/253,833 (U.S. Publication No. 2009/0168760), 12/253,820 (U.S. Publication No. 20090168759) and 12/253,833 (U.S. Publication No. 2009/0168760), all of which are incorporated by reference herein for all purposes. The individual modules are briefly described below. For more details, the above-listed, co-pending applications should be reviewed.
The MCMS module 40 includes a number of modules and services for creating, managing and conducting multiple conversations. The MCMS module 40 includes a user interface module 40A for enabling the user to interface and control the audio and video rendering and creating functions on the device 32, rendering/encoding module 40B for performing rendering and encoding tasks, a contacts service 40C for managing and maintaining information needed for creating and maintaining contact lists (e.g., telephone numbers and/or email addresses), a presence status service 40D for both sharing the online status of the user of the device 32 as well as the online status of the other users on the communication services network 12. The MCMS data base 40E stores and manages the meta data for messages and conversations conducted using the application 30 running on a device 32 as well as contact and presence status information. In alternative embodiments, the MCMS database 40E may include, but is not limited to, relational databases, file-based databases, object databases, document-oriented databases, or any other type of database and/or database management system that is capable of storing and retrieving data.
The Store and Stream module 42 includes a Permanent Infinite Memory Buffer or PIMB 46 for storing, in an indexed format, the media of received and sent messages. The store and stream module 42 also includes an encode-receive module 42A, net receive module 42B, transmit module 42C and a render module 42D. The encode-receive module 42A performs the function of receiving, encoding, indexing and storing in a time-indexed format in the PIMB 46 media created using the client application 30 on device 32. The net receive module 42B performs the function of indexing and storing in the time-indexed format in the PIMB 46 the media contained in messages received from other devices 32 or 26 through a gateway client 16. The transmit module 42C is responsible for both storing in the PIMB and transmitting to recipients the media of messages created using the device 32. The render module 42D enables the client application 30 to render on device 32 the media of messages, either in the near real-time mode as media is received over the network 12 or in the time-shifted mode by retrieving and rendering the media stored in the PIMB 46.
The MCMS module 40 and the Store and Stream module 42 also each communicate with various hardware components 48 provided on the device 32, including, but not limited to, encoder/decoder hardware 48A, network interface 48B for connecting the device 32 to network connection 34, and media drivers 48C. The encoder/decoder hardware 48A is provided for encoding the media, such as voice, text, video or sensor data, generated by a microphone, camera, keyboard, touch-sensitive display, GPS, sensor, etc. provided on or associated with the device 32 and decoding similar media before it is rendered on the device 32. The media drivers 48C are provided for driving the media generating components, such as speaker and/or a display (not illustrated) after the media has been decoded. The network interface 48B is provided for the connecting device 32 to a network connection 34, either through a wireless or wired connection. Although not illustrated, the client application 30 runs or is executed by an underlying processor embedded in device 32, such as a microprocessor or microcontroller.
The transmitted and received media stored in the PIMB 46 is persistently stored. The term persistent storage as used herein is intended to have broad meaning. In various embodiments, persistent storage is intended to mean the storage of media and meta data from just beyond transient storage needed to either transmit or render media in real-time to storage for an indefinite period of time. The term persistent storage therefore may have different meanings, depending specific implementations or embodiments.
In addition, “real-time” is intended to mean the consumption or rendering of time-based media (i.e., media that changes over time) as the media is being transmitted, regardless if the media is “live” or not. The real-time consumption of “live” media is intended to mean the rendering of time-based media as the media is being created and transmitted. The real-time consumption of non-live media is intended to mean the consumption of previously recorded time-based media that is being transmitted out of storage.
Referring to FIG. 3, a media flow diagram on a communication device 32 running the client application 30 is shown. The diagram illustrates the flow of media for both the transmission and receipt of media, in either the real-time mode or the time-shifted mode.
(i) The Receipt of Media: Media received from the communication services network 12 is simultaneously and progressively stored in the PIMB 46 by the net receive module 42B as the media is over the network connection 34, as designated by arrow 50, regardless if the media is to be rendered in real-time or in the time-shifted mode. When in the real-time mode, the media is also simultaneously and progressively provided by the net receive module 42B, as designated by arrow 52, to the render module 42D. In response, the render module 42D simultaneously and progressively renders the media as it is received over connection 34 on a media-rendering device (e.g., a speaker and/or display). In the time-shifted mode, the media is retrieved by the render module 42D, as designated by arrow 54, from the PIMB 46 at an arbitrary time after it was persistently stored. The retrieved media is then rendered on the media rendering devices, such as a speaker and/or display. In this manner, the recipient of the media may review persistently stored media at any time after storage in the time-shifted mode.
(ii) Transmitting Media: Media created on device 32 by a media creating device (e.g. a microphone, keyboard, video and/or still camera, a sensor such as a thermometer or GPS, or any combination thereof) is progressively stored in the PIMB 46 in a time-indexed format as it is created, as designated by arrow 58. In most situations, the media is also provided, as designated by arrow 56, to the transmit module 42C, which simultaneously and progressively transmits the media as it is created. In other situations, media may be transmitted by transmit module 42C out of the PIMB 46 at some arbitrary time after it was created, as designated by arrow 60. Transmissions out of the PIMB 46 typically occur when media is created when the device 32 is disconnected from the network 12. When the device 32 reconnects, the media is read from the PIMB 46 and transmitted by the transmit module 42C.
As a clarification, the media creating devices (e.g, microphone, camera, keyboard, etc.) and media rendering devices (e.g., speaker and display) as illustrated in FIG. 4 are intended to be symbolic of such devices. It should be understood such devices are typically embedded in communication devices 32, such as mobile or cellular phones, radios, mobile computers, etc. With other types of communication devices 32, such as desktop computers, the media rendering or generating devices may be either embedded in or plug-in accessories.

C. Communication Services Network

Referring to FIG. 4, a block diagram of the communication services network 12 is shown. As noted above, the network 12 includes one or more servers 18. In a non-exclusive embodiment, each server 18 includes one or more routers 20, one or more header stores 62, and one or more body stores 64.
The routers 20 communicate with other routers 20, to header stores 62 for read and/or write operations, to body stores 64 for read and/or write operations. Routers 20 are further responsible for updating routing tables and maintaining the presence status information of users on the network 12. Routers 20 also perform a number of security functions, including authentication, encryption, and authorization.
In a non-exclusive embodiment, the one or more servers 18 on the network 12 are highly configurable and scalable. For example, if a large number of users subscribe to the services provided by the network 12, then a large number of routers 20 may be needed. If a server 18 routes a high volume of traffic, but the messages tend to be relatively short in duration (e.g., contain minimal media), then the number of header stores 62 may be increased relative to the number of body stores 64. Alternatively, if the traffic handled by a server tends to have large amounts of media (i.e., the messages are long in duration), then more body stores 64 may be needed. Further, the number of servers 18 included on the network may be increased or reduced as needed.
On the network 12, each of the server(s) 18 subscribe to all of the header and body data for a given user and/or group of users. As a result, if a server 18 that holds the header and/or body information for a user becomes unavailable, a router 20 may be able to locate another server 18 to obtain the data. In other embodiments, one or more users, servers or any other entity may subscribe based on the domain of user(s), defined sets of users, media type, codec type used to encode the media, conversation name, conversation subject or topic, time range, or any other type of defined criteria.

D. Vox Messages and Media Payloads

Client application 30 enabled devices 32 communicate with one another using individual message units, referred to herein as “Vox messages”. By sending Vox messages back and forth over the communication services network 12, the users of the devices 32 may communicate with one another, either in the real-time mode or in a time-shifted messaging mode, and with the ability to seamlessly transition between the two modes.
There are two types of Vox messages, including (i) messages that do not contain media and (ii) messages that do contain media. Vox messages that do not contain media are generally used for message meta data, such as media headers and descriptors, contacts information (telephone numbers or email addresses), presence status information, etc. Message meta data includes such attributes as a message identifier or ID, the identification of the message originator, a recipient list, and a message subject. The identifier information may be used for a variety of reasons, including, but not limited to, building contact lists, associating media with messages, and/or associating messages with conversations. The Vox messages that contain media are used for the transport of media. In one embodiment, messages containing text media may include both meta data and the text media, whereas messages containing time-based media, such as voice or video, do not contain meta data.
In one embodiment, Vox messages are layered on top of the application layer of whatever transport protocol or protocols are used on the underlying network infrastructure below the network 12. As a result, a new transport protocol for Vox messages is not needed. Rather, Vox messages are transmitted and routed across the network 12 using current transport protocols running over the existing telecommunications infrastructure.
The presence status information contained in Vox messages may be used to identify the users that are currently authenticated by the system 10 and/or if a given user is reviewing a message in real-time or not. The presence data is therefore useful in determining, in certain embodiments, how messages are delivered across the network 12. In situations where the presence status indicates an authenticated user is reviewing a message in real-time for example, then a transport protocol optimized for timely (i.e., real-time) delivery, such as UDP, may be used, whereas a transport protocol optimized for efficient delivery of messages, such as TCP, may be used when the presence status indicates the authenticated user is not reviewing the message in real-time.

E. Transmission Protocols and Complete Copies of Media

Referring to FIG. 5, a flow diagram 80 for transmitting media between nodes on the communication services network 12 according to the present invention is shown. The sending and receiving node pair may be between a communication device 32 and a server 18, between server 18 hops on the network 12, or between any two nodes on the network. As described below, either a network efficient protocol or a loss tolerant protocol may be used, depending on (i) the condition on the network and (ii) if the media is being consumed in real-time by the recipient.
In the initial decision 82, a transmission loop is defined at the sending node and the sending node determines if there is any media available for transmission. If not, step 82 is continually repeated until media becomes available for transmission. When media is available, it is next determined (decision 84) if the media is or will be consumed in real-time based on the presence information of the recipient(s). If the presence information of all the recipient(s) indicates none are reviewing in real-time, then the media is transmitted using a network efficient protocol (step 86). If one or more of the recipients is reviewing the media in real-time, then the transmitting node determines if the condition on the network (decision 88) is sufficient for transmitting the media at a rate sufficient to support real-time consumption using the network efficient protocol.
If the condition on the network is sufficient for supporting real-time communication using the network efficient protocol, then the transmitting node continues transmitting using the network efficient protocol (step 86). If the condition is not sufficient, however, then the transmitting node transmits the media using a loss tolerant protocol (step 90).
With media that is transmitted using the loss tolerant protocol, it is determined in decision 92 if the condition on the network is sufficiently good enough (i.e., the rate of media loss is minimal) to support real-time communication. If the condition on the network is not sufficient, meaning real-time communication is not possible or practical using even the loss tolerant protocol, then the transmitting node stops using the loss tolerant protocol. Instead, the media is transmitted using the network efficient protocol (step 86) as the condition of the network permits. On the other hand if the condition on the network is sufficiently good enough to support real-time communication, then the media is transmitted using the loss tolerant protocol.
In decision 94, it is determined if the message has ended or if the recipient is no longer reviewing in the real-time mode. If neither condition is met, then another transmission loop is defined (decision 82) and the above process is repeated. When either condition is met, the media originally transmitted using the loss tolerant protocol is retransmitted when network conditions permit using the network efficient protocol, which guarantees the eventually delivery of a complete copy of the media as originally encoded. In most situations, the retransmission occurs when bandwidth on the network is available beyond what is needed to support real-time communication. In one embodiment, a complete copy of the media previous sent using the loss tolerant protocol is retransmitted. In an alternative embodiment, just the missing media is transmitted.
The aforementioned process is continually repeated while media is available for transmission. With each cycle, a transmission loop is defined and the above-described process is repeated.
The transmission protocol as described above with respect to FIG. 5 resides in layer above or on top of the underlying network efficient and loss tolerant protocols. In one embodiment, the network efficient protocol is used as the default protocol, meaning when a transmission begins, the network efficient protocol is initially used and the loss tolerant protocol is used only if the media is being consumed in real-time and the condition on the network is not good enough to support real-time consumption. In an alternative embodiment, the loss tolerant protocol may be the protocol that is initially used when a transmission begins and the network efficient protocol is used only when either (i) the media is not being consumed in real-time and/or (ii) the condition on the network is good enough to support real-time communication using the network efficient protocol. The second embodiment is typically used in situations when it is known that the transmitted media is or will likely be consumed in real-time and/or the condition on the network is typically inadequate to support real-time consumption.
In a specific embodiment using TCP and UDP, the transmission protocol as described above with respect to FIG. 5 monitors the rate at which the TCP protocol transmits media. So long as the transmit buffer used by the TCP protocol does not back-up beyond a predetermined threshold or overflow, TCP is used for transmissions, regardless if the media is being consumed in real-time or not. If the transmission buffer backs-up beyond the predetermined threshold level or overflows, indicating that the condition on the network is inadequate to support the transmission of media at a rate sufficient to support real-time consumption, then a switch occurs and UDP is used for media that is being consumed in real-time. The transmitting node will continue using UDP until (i) the transmitted media is no longer being consumed in real-time, (ii) the condition on the network has improved to the point where TCP may be used for the transmission and consumption of the media in real-time; and/or (iii) conditions on the network are so poor, it is not practical for the media to be consumed in real-time even using UDP.
The transmission protocol as described above with respect to FIG. 5 resides at both communication devices 32 and at servers 18 on the network 12. In a non-exclusive embodiment, the transmission protocol is embedded in the client application 30. In an alternative embodiment, the transmission protocol may be separate from but cooperate with the application 30. Regardless of how implemented, the transmission protocol as described herein allows both client 30 communication devices 32 and server(s) 18 to communicate with one another using either a loss tolerant or network efficient protocol, depending on if the media being transmitted is being consumed in real-time and the condition of the network.
Referring to FIG. 6, a timing diagram illustrating the transmission of media between sending and receiving nodes is illustrated. Initially at time T1, client 32A begins transmitting the media of a message (step 82) to recipient client 32B using the network efficient protocol (e.g., TCP). At time T2, the presence information of the recipient 32B indicates if the media of the message is being reviewed in real-time or not. Based on the presence information (decision 84), the sending node evaluates if the media is being consumed in real-time or not. In this timing diagram example, it is assumed that the media is in fact being consumed in real-time and the network is not good enough (decision 88) to support real-time communication using the network efficient protocol. As a result at time T3, the media of the message is transmitted using the loss tolerant protocol (step 90). Eventually the message ends and/or the recipient transitions from the real-time to the time-shifted mode. When either event occurs, as designated by the time T4, the transmitting client 32A retransmits, according to different embodiments, either just the missing media or the media previously sent using the loss tolerant protocol. In either case, the network efficient protocol is used, guaranteeing the delivery of a complete copy of the media of the message to the recipient 32B.
The timing diagram is intended to illustrate the asynchronous nature of the transmission flow diagram illustrated in FIG. 5. Many of the steps and/or decisions outlined in the flow diagram of FIG. 5 occur in parallel to the extent possible. Many of the events as described overlap with one another to some extent and do not necessarily occur in “lock-step”, meaning it is not necessary for one step to start only after the previous step is complete, as is typical with most synchronous “clock-driven” communication systems. Rather, each step is started as soon as the conditions necessary to perform that step are completed. This is particularly true with the transmission of media for example. Conventional communication systems typically transmit media using only a single transmission protocol. With the protocol as described herein, however, the transmission of media may be switched “on the fly” during the course of a message between either the network efficient or loss tolerant protocols, depending on the presence status of the recipient(s) and/or condition on the network.
The transmission of media between communication devices 32, as described above, occurs through one or more servers 18 on the network 12. In an alternative pier-to-pier embodiment however, it would be possible for two communication devices 32 to communicate directly with one another. With this embodiment, the flow and timing diagrams of FIGS. 5 and 6 would still be applicable, except the communication flow would occur directly over the network 12 between a sending communication device 32 and one or more recipient communication devices 32.
It also should be noted that for the sake of simplicity, the transmission of messages as described above has been “one-way”. It should be understood that the transmissions of media, using either the network efficient and/or the loss tolerant protocol, is often bi-directional or “two-way”. With two-way communication in the real-time mode, the user experience is similar to a conventional full-duplex conversation. In addition, bi-directional communication can also take place between multiple parties (i.e, more than two), similar to a multi-party conference call.
By using either the network efficient or loss tolerant protocol, depending on if media is being consumed in real-time and/or on the condition of the network, transmissions are optimized for real-time when needed or for efficient delivery when real-time delivery is not critical or network conditions are sufficiently good to support real-time communication using the network efficient protocol. On the other hand when the media is being reviewed in real-time and network conditions are poor, the loss tolerant protocol is typically used to extend or enhance the ability to conduct real-time communication. Since any missing or media transmitted using the loss tolerant protocol is eventually retransmitted using a network efficient protocol that guarantees the delivery, the recipient eventually receives a complete copy (i.e. a full bit rate representation as the media was originally encoded). The full bit rate representation of the media typically replaces any missing, defective or out of order representations of media previously received and stored in the PIMB 46 of the receiving device 32. In this manner, the recipient may review the complete, full quality, version of the media in the time-shifted mode at a later arbitrary time.
In one embodiment, TCP is used as the network efficient protocol, while UDP is used as the loss tolerant protocol. In other embodiments, TCP is used as the network efficient protocol, while the Cooperative Transmission Protocol (CTP) described in U.S. application Ser. No. 12/192,890, incorporated by reference herein, is used as the loss tolerant protocol. In other embodiments, any network efficient or loss tolerant protocol may be used.
Although many of the components and processes are described above in the singular for convenience, it will be appreciated by one of skill in the art that multiple components and repeated processes can also be used to practice the techniques of the system and method described herein. Further, while the invention has been particularly shown and described with reference to specific embodiments thereof, it will be understood by those skilled in the art that changes in the form and details of the disclosed embodiments may be made without departing from the spirit or scope of the invention. For example, embodiments of the invention may be employed with a variety of components and should not be restricted to the ones mentioned above. It is therefore intended that the invention be interpreted to include all variations and equivalents that fall within the true spirit and scope of the invention.

Claims

1-17. (canceled)

18. A method for transmitting media from a node to one or more recipients over a network using either a network efficient protocol or a loss tolerant protocol, the method comprising:

(a) ascertaining if media available for transmission is or will be consumed in real-time by any of the one or more recipients; and either:

(b) transmitting the media from the node over the network to the one or more recipients using the loss tolerant protocol when:

(i) at least one of the one or more recipients is or will be consuming the media in real-time; and

(ii) the available bandwidth on the network is not good enough to support the transmission of the media at a rate sufficient for real-time consumption using the network efficient protocol; or

(c) transmitting the media from the node over the network to the one or more recipients using the network efficient protocol when:

(iii) none of the one or more recipients is or will be consuming the media in real-time; or

(iv) one or more of the recipients is or will be reviewing the media in real-time and the available bandwidth on the network is good enough to support the transmission of the media at a rate sufficient for real-time consumption using the network efficient protocol.

19. The method of claim 18, wherein the network efficient protocol is TCP.

20. The method of claim 18, wherein the network efficient protocol guarantees the delivery of a complete copy of the media transmitted to the one or more recipients.

21. The method of claim 18, wherein the loss tolerant protocol is UDP.

22. The method of claim 18, wherein the loss tolerant protocol is tolerant of the loss of packets containing bits representative of the media during transmission.

23. The method of claim 18, further comprising:

identifying the media transmitted using the loss tolerant protocol; and

re-transmitting at least a portion of the identified media using the network efficient protocol.

24. The method of claim 23, wherein re-transmitting the at least a portion of the identified media comprises one of the following:

(i) re-transmitting the identified media in full; or

(ii) re-transmitting just the defective or media lost when transmitting using the loss tolerant protocol.

25. The method of claim 23, wherein the identified media is re-transmitted when bandwidth on the network in excess of what is needed for real-time communication is available on the network.

26. The method of claim 18, further comprising repeatedly defining transmission loops and performing (a) and either (b) or (c) for the media available for transmission during each transmission loop respectively.

27. The method of claim 18, further comprising determining from presence information associated with each of the one or more recipients if the media available for transmission is or will be consumed by any of the one or more recipients in real-time.

28. The method of claim 18, wherein the media is time-based media that changes over time.

29. The method of claim 18, wherein the consumption of the media in real-time further comprises one of the following:

(i) the rendering of the media “live” as the media is created and transmitted; or

(ii) the rendering of previously created and stored media streamed from storage.

30. The method of claim 18, further comprising:

defining the network efficient protocol as a default protocol when initially transmitting the media; and

switching the transmission of the media from the network efficient protocol to the loss tolerant protocol when:

the media is or will be consumed in real-time; and

the available bandwidth on the network is not good enough to support the transmission of the media at a rate suitable for consumption in real-time using the network efficient protocol.

31. The method of claim 18, further comprising:

defining the loss tolerant protocol as a default protocol when initially transmitting the media; and

switching the transmission of the media from the loss tolerant protocol to the network efficient protocol when either:

(i) the media is not being consumed in real-time; or

(ii) the media is being consumed in real-time and the available bandwidth on the network is good enough to support the transmission of the media at a rate suitable for consumption in real-time using the network efficient protocol

32. The method of claim 18, further comprising ascertaining the bandwidth condition on the network by determining when a transmission buffer used for buffering the media for transmission at the node backs-up beyond a predetermined threshold or overflows during the transmission of the media.

33. The method of claim 18, wherein the media comprises one of the following: voice, video, sensor data, GPS data, text, pictures, or any combination thereof.

34. A communication application embedded in a non-transient computer readable medium and intended to run on a communication device, the application comprising:

a transmission module, the transmission module configured to:

(a) ascertain if media available for transmission is or will be consumed in real-time by any of one or more recipients; and either:

(b) transmit the media over a network to the one or more recipients using the loss tolerant protocol when:

(c) transmit the media over the network to the one or more recipients using the network efficient protocol when:

35. The application of claim 34, wherein the network efficient protocol is TCP.

36. The application of claim 34, wherein the network efficient protocol guarantees the delivery of a complete copy of the media transmitted to the one or more recipients.

37. The application of claim 34, wherein the loss tolerant protocol is UDP.

38. The application of claim 34, wherein the loss tolerant protocol is tolerant of the loss of packets containing bits representative of the media during transmission.

39. The application of claim 34, wherein the transmission module is further configured to:

identify the media transmitted using the loss tolerant protocol; and

re-transmit at least a portion of the identified media using the network efficient protocol.

40. The application of claim 39, wherein the transmission module is further configured to re-transmit by performing one of the following:

(i) re-transmitting the identified media in full; or

(ii) re-transmitting just media lost when transmitting using the loss tolerant protocol.

41. The application of claim 39, wherein the transmission module is further configured to re-transmit when bandwidth on the network in excess of what is needed for real-time communication is available on the network.

42. The application of claim 34, wherein the transmission module is further configured to repeatedly define transmission loops and perform (a) and either (b) or (c) for the media available for transmission during each transmission loop respectively.

43. The application of claim 34, wherein the transmission module is further configured to determine from presence information associated with each of the one or more recipients if the media available for transmission is or will be consumed by any of the one or more recipients in real-time.

44. The application of claim 34, wherein the media is time-based media that changes over time.

45. The application of claim 34, wherein the consumption of the media in real-time comprises one of the following:

46. The application of claim 34, wherein the transmission module is further configured to:

define the network efficient protocol as a default protocol when initially transmitting the media; and

switch the transmission of the media from the network efficient protocol to the loss tolerant protocol when:

the media is or will be consumed in real-time; and

47. The application of claim 34, wherein the transmission module is further configured to:

define the loss tolerant protocol as a default protocol when initially transmitting the media; and

switch the transmission of the media from the loss tolerant protocol to the network efficient protocol when either:

(i) the media is not being consumed in real-time; or

48. The application of claim 34, wherein the transmission module is further configured to ascertain a bandwidth condition on the network by determining when a transmission buffer used for buffering the media for transmission backs-up beyond a predetermined threshold or overflows during the transmission of the media.

49. The application of claim 34, wherein the media comprises one of the following:

voice, video, sensor data, GPS data, text, pictures, or any combination thereof.

50. A communication device, comprising:

a transmission element, the transmission element configured to:

51. The device of claim 50, wherein the network efficient protocol is TCP.

52. The device of claim 50, wherein the network efficient protocol guarantees the delivery of a complete copy of the media transmitted to the one or more recipients.

53. The device of claim 50, wherein the loss tolerant protocol is UDP.

54. The device of claim 50, wherein the loss tolerant protocol is tolerant of the loss of packets containing bits representative of the media during transmission.

55. The device of claim 50, wherein the transmission element is further configured to:

identify the media transmitted using the loss tolerant protocol; and

56. The device of claim 55, wherein the transmission element is further configured to re-transmit by performing one of the following:

(i) re-transmitting the identified media in full; or

57. The device of claim 55, wherein the transmission element is further configured to re-transmit when bandwidth on the network in excess of what is needed for real-time communication is available on the network.

58. The device of claim 50, wherein the transmission element is further configured to repeatedly define transmission loops and perform (a) and either (b) or (c) for the media available for transmission during each transmission loop respectively.

59. The device of claim 50, wherein the transmission element is further configured to determine from presence information associated with each of the one or more recipients if the media available for transmission is or will be consumed by any of the one or more recipients in real-time.

60. The device of claim 50, wherein the media is time-based media that changes over time.

61. The device of claim 50, wherein the consumption of the media in real-time comprises one of the following:

62. The device of claim 50, wherein the transmission element is further configured to:

the media is or will be consumed in real-time; and

63. The device of claim 50, wherein the transmission element is further configured to:

(i) the media is not being consumed in real-time; or

64. The device of claim 50, wherein the transmission element is further configured to ascertain a bandwidth condition on the network by determining when a transmission buffer used for buffering the media for transmission backs-up beyond a predetermined threshold or overflows during the transmission of the media.

65. The device of claim 50, wherein the media comprises one of the following: voice, video, sensor data, GPS data, text, pictures, or any combination thereof.

66. The apparatus of claim 50, wherein the communication device comprises one of the following: a cellular phone, a mobile phone, a land-line phone, a PTT radio, a satellite radio, a desktop computer, a mobile computer, a wired communication device, a wireless communication device or a server.