WO1997028502A1 - Server-group messaging system for interactive applications - Google Patents

Server-group messaging system for interactive applications Download PDF

Info

Publication number
WO1997028502A1
WO1997028502A1 PCT/US1997/000567 US9700567W WO9728502A1 WO 1997028502 A1 WO1997028502 A1 WO 1997028502A1 US 9700567 W US9700567 W US 9700567W WO 9728502 A1 WO9728502 A1 WO 9728502A1
Authority
WO
WIPO (PCT)
Prior art keywords
message
group
host
ulp
address
Prior art date
Application number
PCT/US1997/000567
Other languages
French (fr)
Inventor
Daniel Joseph Samuel
Marc Peter Kwiatkowski
Jeffrey Jackiel Rothschild
Original Assignee
Mpath Interactive, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=24382776&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=WO1997028502(A1) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Mpath Interactive, Inc. filed Critical Mpath Interactive, Inc.
Priority to AU17471/97A priority Critical patent/AU1747197A/en
Priority to EP97904759A priority patent/EP1012724A1/en
Priority to JP9527663A priority patent/JP2000504133A/en
Publication of WO1997028502A1 publication Critical patent/WO1997028502A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/02Details
    • H04L12/16Arrangements for providing special services to substations
    • H04L12/18Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/185Arrangements for providing special services to substations for broadcast or conference, e.g. multicast with management of multicast group membership
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/02Details
    • H04L12/16Arrangements for providing special services to substations
    • H04L12/18Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1886Arrangements for providing special services to substations for broadcast or conference, e.g. multicast with traffic restrictions for efficiency improvement, e.g. involving subnets or subdomains
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08Configuration management of networks or network elements
    • H04L41/0893Assignment of logical groups to network elements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/131Protocols for games, networked simulations or virtual reality
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/327Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the session layer [OSI layer 5]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40Network security protocols
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/329Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]

Definitions

  • the present invention relates to computer network systems, and particularly to server group messaging systems and methods for reducing message rate and latency
  • a network is comprised of nodes and links
  • the nodes are connected in such a way that there exists a path from each node over the links and through the other nodes to each of the other nodes in the network
  • Each node may be connected to the network with one or more links
  • Nodes are further categorized into hosts, gateways and routers Hosts are computer systems that are connected to the network by one link They communicate with the other nodes on the network by sending messages and receiving messages
  • Gateways are computer systems connected to the network by more than one link. They not only communicate with the other nodes as do hosts, but they also forward messages on one of their network links to other nodes on their other network links. This processing of forwarding messages is called routing.
  • gateways may perform other functions in a network.
  • Routers are nodes that are connected to the network by more than one link and whose sole function is the forwarding of messages on one network link to the other network links to which it is connected.
  • a network consisting of many network links can be thought of as a network of sub-networks with gateways and/or routers connecting the sub-networks together into what is called an internet.
  • Internet Today the widely known example of a world wide internet is the so called "Internet" which in 1995 has over 10 million computers connected full time world-wide.
  • a game designed to be deployed over a network which is to be played by multiple players simultaneously The game could be implemented in software on a PC connected to a network.
  • a rate set by its internal time base it would sample the inputs of the local user, receive messages from the network from the PCs of the other players and send messages out to the PCs of the other players.
  • a typical rate will be ten time per second for a time period of 100ms.
  • the messages sent between the PCs would contain information that was needed to keep the game consistent between all of the PCs.
  • the packets could contain information about the new positions of the players as they moved.
  • PC games Today there are many commercial example of PC games that can be played between multiple players on Local Area Networks (LANs) or by two players over dial-up phone lines using modems.
  • the network messages sent by such games contain a wide variety of information specific to the game. This can include position and velocity information of the objects in the game along with special actions taken by a player that effect the other players in the game.
  • the case of a two player game played over a modem is particularly simple. If the message rate is 10 messages per second, each PC sends 10 messages per second to the other PC and receives 10 messages per second. The delay introduced by the modems and phone line is small and will not be noticed in most games. Unfortunately, the case of two players is uninteresting for networked interactive applications. With the same game played with 8 players on a LAN, the message rate increases. Each PC must send 7 messages, one to each of the other 7 players every time period and will receive 7 messages from the other players in the same time period. If the messaging time period is 100ms, the total message rate will be 70 messages sent per second and 70 messages received per second. As can be seen the message rate increases linearly with the number of players in the game. The message rates and data rates supported by popular LANs are high enough to support a large number of players at reasonable message sizes. Unfortunately, LANs are only deployed in commercial applications and cannot be considered for deploying a networked interactive application to consumer users.
  • the Internet is a widely known example of a wide area network.
  • the Internet is based on a protocol appropriately called the Internet
  • IP IP
  • IP IP
  • the addressing model provides a 32 bit address for all nodes in the network and all packets carry source and destination addresses.
  • IP also defines the routing of packets between network links in an inter-network. Gateways and routers maintain tables that are used to lookup routing information based on the destination addresses of the packets they receive. The routing information tells the gateway/router whether the destination of the packet is directly reachable on a local network link connected to the gateway/router or if not, the address of another gateway/router on one of the local network links to which the packet should be forwarded.
  • IP On top of IP are the layer 4 transport protocols TCP and UDP.
  • UDP provides datagram delivery services to applications that does not guarantee reliable or in-order delivery of the datagrams.
  • TCP is a connection oriented service to applications that does provide reliable delivery of a data stream. It handles division of the stream into packets and ensures reliable, in- order delivery. See the Internet Society RFCs: RFC-791 "Internet Protocol”, RFC-793 "Transmission Control Protocol” and RFC- 1180 "A TCP/IP tutorial”.
  • IP, TCP and UDP are unicast protocols: packets, streams or datagrams are transmitted from a source to a single destination.
  • Figure 1 shows a conventional unicast network with hosts 1, 2, 3 and 4 and network links 11, 12, 13, 14, 15,16,17, 18 and 19 and routers 5, 6, 7, 8, 9 and 10.
  • each host wants to send a data payload to each of the other hosts.
  • Host 1 has network address A
  • host 2 has network address C
  • host 3 has network address B
  • host 4 has network address D.
  • Existing network protocols are typically based on packet formats that contain a source address, destination address and a payload. This is representative of commonly used wide area network protocols such as IP.
  • FIG. 2 shows the example packets that are sent by the hosts to one another using a conventional unicast network protocol such as IP.
  • Host 1 send packets 20, to host 3, packet 21 to host 2 and packet 22 to host 4.
  • Host 1 wants to send the same data Pl to each of the other three hosts, therefore the payload in all three packets is the same.
  • Packet 20 travels over network links 11, 12, 15 and 18 and through routers 5, 6, and 8 to reach host 3.
  • host 3 sends packets 23 to host 1, packet 24 to host 2 and packet 25 to host 4.
  • Host 2 and host 4 send packets 26, 27, 28 and 29, 30, 31 respectively to the other three hosts. All of these packets are carried by the unicast network individually from the source host to the destination host. So in this example each host must send three packets and receive three packets in order for each host to send its payload to the other three hosts.
  • each host must send a packet to every other host that it wishes to communicate with in an interactive application. Further, it receives a packet from every other host that wishes to communicate with it. In an interactive application, this will happen at a regular and high rate. All of the hosts that wish to communicate with one another will need to send packets to each other eight to ten times per second. With four hosts communicating with one another as in this example, each host will send three messages and receive three messages eight to ten times per second. As the number of hosts in the application that need to communicate with one another grows, the message rate will reach a rate that cannot be supported by conventional dial-up lines. This makes unicast transport protocols unsuitable for delivering interactive applications for multiple participants since their use will result in the problem of high packet rates that grow with the number of participants.
  • IP multicasting supports the transmission of a IP datagram to a host group by addressing the datagram to a single destination address.
  • Multicast addresses are a subset of the IP address space and identified by class D IP addresses - these are IP addresses with "11 10" in the high order 4 bits.
  • the host group contains zero or more IP hosts and the IP multicasting protocol transmits a multicast datagram to all members of the group to which it is addressed. Hosts may join and leave groups dynamically and the routing of multicast datagrams is supported by multicast routers and gateways.
  • the group already contains Host M which is a number of router hops away from Host N in the network Shortly after Host N has sent it join message, Host M sends a datagram to the group, but the local router of Host M has not yet been informed of the change in group membership and as a result the datagram is not forwarded to one of the particular network links connected to the local router of Host M that is the only path in the network from that router that ultimately will reach Host N.
  • Host N will receive no datagrams addressed to the group from Host M until the local router of M has its group membership information updated
  • Other related problems can also occur When a host leaves a group, messages addressed to the group will continue for some time to be routed to that host up to the local router of that host. The local router will know at least not to route the datagram onto the local network of that host. This can still result in a great deal of unnecessary datagrams being carried in a large network when there are many active message groups with rapidly changing memberships.
  • distributed multicast messaging does not sufficiently reduce the message rate between the hosts.
  • each host need only send one message addressed to the message group in order to send a message to all of other hosts in the group. This is an improvement over conventional unicast messaging where one message would need to be sent to each of the other hosts in a group.
  • distributed multicast messaging does nothing to reduce the received message rate at each of the hosts when multiple hosts in a group are sending messages to the group closely spaced in time. Let us return to the example of a group often hosts sending messages seven times per-second to the group. With conventional unicast messaging, each host will need to send 9 messages to the other hosts, seven times per-second and will receive 9 messages, seven times per-second. With distributed multicast messaging, each host will need to send only one message to the group containing all of the hosts seven times per-second, but will still receive 9 messages, seven times per-second. It is desirable to further reduce the number of received messages.
  • Figure 3 shows a network with multicast routers 39, 40, 41, 42, 43 and 44 and hosts 35,
  • the four hosts have unicast network addresses A, B, C, D and are also all members of a message group with address E.
  • a network protocol such IP with multicast extensions is assumed to be used in this example.
  • Host 35 sends packet 54 with source address A and destination multicast address E to the entire message group.
  • host 37 sends packet 55 to the group
  • host 36 sends packet 56 to the group
  • host 38 sends packet 57 to the group.
  • Packet 54 is carried over network link 45 to multicast router 39.
  • the router determines from its routing tables that the multicast packet should be sent onto network links 46 and 47 and duplicates the packet and sends to both of these network links.
  • the packet is received by multicast routers 40 and 43.
  • Multicast router 43 sends the packet onto network link 50 and router 40 sends its onto links 48 and 49.
  • the packet is then received at multicast routers 44, 42 and 41.
  • Router 41 sends the packet over network link 51 where it is received by host 36.
  • Router 42 sends the packet over network link 52 to host 37 and router 44 sends the packet over link 53 to host 38.
  • a similar process is followed for each of the other packets sent by the hosts to the multicast group E. The final packets received by each host are shown in
  • While distributed multicasting does reduce the number of messages that need to be sent by the hosts in a networked interactive application, it has no effect on the number of messages that they receive. It has the further disadvantages of poor behavior when group membership is rapidly changing and requires a special network infrastructure of multicast routers. It also has no support for message aggregation and cannot do so since message delivery is distributed. Distributed multicasting also has no support for messages that define logical operations between message groups and unicast host addresses. All of these problems can be understood when placed in context of the design goals for distributed multicast messaging. Distributed multicast messaging was not designed for interactive applications where groups are rapidly created, changed and destroyed. Instead it was optimized for applications where the groups are created, changed and destroyed over relatively long time spans perhaps measured in many minutes or even hours.
  • An example would be a video conference where all the participants agreed to connect the conference at a particular time for a conference that might last for an hour. Another would be the transmission of an audio or video program from one host to many receiving hosts, perhaps measured in the thousands or even millions. The multicast group would exist for the duration of the audio/video program. Host members would join and leave dynamically, but in this application it would be acceptable for there to be a significant time lag from joining or leaving before the connection was established or broken.
  • ATM Asynchronous Transfer Mode
  • IP IP
  • Asynchronous Transfer Mode IP
  • ATM is fundamentally connection oriented.
  • An ATM network consists of ATM switches interconnected by point-to-point links. The host systems are connected to the leaves of the network. Before any communication can occur between the hosts through the network, a virtual circuit must be setup across the network. Two forms of communication can be supported by an ATM network. Bi-directional point-to-point between two hosts and point-to-multipoint in one direction from one host to multiple hosts. ATM, however, does not directly support any form of multicasting.
  • a multicast server shown in Figure 8.
  • Host systems 1 12, 113, 114, 115 setup point-to-point connections 106, 107, 108 and 109 to a multicast server 105.
  • ATM cells are sent by the hosts to the multicast server via these links.
  • the multicast server sets up a point -to-multipoint connection 11 1 to the hosts which collectively constitute a message group. Cells sent to the server which are addressed to the group are forwarded to the point- to-multipoint link 11 1.
  • the ATM network 1 10 is responsible for the transport and switching for maintaining all of the connections between the hosts and the server.
  • the cells carried by the point-to-multipoint connection are duplicated when necessary by the ATM switches at the branching points in the network tree between and forwarded down the branching network links. Therefore, the network is responsible for the replication of the cells and their payloads, not the server.
  • This method has the same problems as distributed multicasting when used for an interactive application. Each host still receives individual cells from each of the other hosts, so there is no aggregation of the payloads of the cells targeted at a single host. There is no support for addressing cells to hosts based on logical operations on the sets of members of host groups.
  • the present invention relates to facilitating efficient communications between multiple host computers over a conventional wide area communications network to implement an interactive application such as a computer game between multiple players.
  • the hosts will be dynamically sending to each other information that the other hosts need in order to keep the interactive application operating consistently on each of the hosts.
  • the invention is comprised of a group messaging server connected to the network that maintains a set of message groups used by the hosts to communicate information between themselves.
  • the invention further comprises a server-group messaging protocol used by the hosts and the server.
  • the server-group messaging protocol is layered on top of the Transport Level Protocol (TLP) of the network and is called the Upper Level Protocol (or ULP).
  • TLP Transport Level Protocol
  • ULP Upper Level Protocol
  • the ULP can be thought of as a session layer protocol built on top of a transport or applications layer protocol.
  • the ULP protocol uses a server-group address space that is separate from the address space of the TLP. Hosts send messages to addresses in the ULP address space to a group messaging server using the underlying unicast transport protocol of the network.
  • the ULP address space is segmented into unicast addresses, implicit group messaging addresses and logical group messaging addresses.
  • the implicit and logical group messaging addresses are collectively called group messaging addresses.
  • Host systems must first establish connections to a group messaging server before sending messages to any ULP addresses The process of establishing this connection is done by sending TLP messages to the server The server establishes the connection by assigning a unicast ULP address to the host and returning this address in an acknowledgment message to the host
  • hosts can inquire about existing message groups, join existing message groups, create new message groups, leave message groups they have joined and send messages to ULP addresses known by the server
  • Each message group is assigned either an implicit or logical ULP address depending on its type
  • Figure 5 shows an example of a wide area network with a group messaging server ("GMS")
  • Hosts 58 has TLP address A and ULP address H
  • host 59 has TLP address C and ULP address J
  • host 60 has TLP address B and ULP address I
  • host 61 has TLP address D and ULP address K
  • the network is a conventional unicast network of network links 69, 70, 71, 72, 73, 74, 75, 76, and 77 and
  • Host 58 sends message 80 which contains the TLP source address A of the host and the destination TLP address S for the GMS 62.
  • the destination ULP address G is an implicit ULP address handled by the GMS and the payload Pl contains both the data to be sent and the source ULP address H of the host It is assumed that prior to sending their ULP messages to the GMS, that each host as already established a connection to the GMS and joined the message group G
  • Host 60 sends message 81 with payload P2 containing data and source ULP address I
  • Hosts 59 sends message 82 with payload P3 containing data and source ULP address J.
  • Host 61 sends message 83 with payload P4 containing data and source ULP address K.
  • the GMS receives all of these messages and sees that each message is addressed to implicit message group G with members H, I, J, and K.
  • the GMS can either process the message with or without aggregating their payloads.
  • Figure 6 shows the case where there is no aggregation and
  • Figure 7 shows the case with aggregation
  • the GMS Without aggregation, the GMS generates the outbound messages 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, and 95 which it sends to the hosts
  • the datagrams have TLP headers with the source and destination TLP addresses of the GMS and the hosts respectively
  • the next field in the datagrams is the destination ULP of the datagram Datagrams 84, 85, and sent to host 58 with TLP address A and ULP address H Datagrams 87, 88, and 89 are sent to host 60 with TLP address B and ULP address I Datagrams 90, 91 and 92 are sent to host 59 with TLP address C and ULP address J.
  • the present invention also avoids the problems that dynamic group membership causes for distributed multicasting. Group membership can be changed very rapidly. Groups can be created, joined and left by single unicast messages from hosts to the GMS. These messages will be point-to-point messages and will not have to propagate in throughout the network nor have to cause routing table changes in the routers.
  • the invention also allows aggregating message payloads of multiple messages destined to a single host into a single larger message. This can be done because of the GMS where all of the messages are received prior to being sent to the hosts.
  • Figure 7 shows an example of how this works.
  • the hosts send their messages to the GMS in exactly the same fashion as in Figure 6 using the same addresses previously defined in Figure 5.
  • Host 58 sends message 96
  • host 60 sends message 97
  • host 59 sends message 98
  • host 61 sends message 99.
  • the GMS receives all of these messages and creates four outbound messages 100, 101, 102 and 103. The process by which these messages will be explained in detail in the detailed description of the invention.
  • Each message is destined to a single host and contains an aggregated payload with multiple payload items.
  • Message 100 has a destination ULP address H for host 58 and aggregated payload P2, P3 and P4 from the messages from hosts 59, 60 and 61.
  • Message 101 is targeted at host 60
  • message 102 is targeted at host 59
  • message 103 is targeted at host 61.
  • each host sends one message and receives one message.
  • the received message is longer and contains multiple payloads, but this is a significant improvement over receiving multiple messages with the wasted overhead of multiple message headers and message processing time.
  • Overall the invention has dramatically reduced the amount of data that must be sent and received by each host. Since the bit rate over conventional phone lines using a modem is low, a reduction in the amount of data that must be sent and received directly translates into improved time and latency for message communications between the hosts.
  • Hosts create, join and leave message groups using control messages in the ULP protocol to the GMS. Hosts may also read and write application specific state information that is stored in the GMS.
  • the message When hosts send messages to other hosts, the message must be at least addressed to an implicit group address.
  • the ULP implicit address will always be the primary address in a message from one host to another.
  • the message may optionally specify auxiliary destination addresses. In many cases the implicit ULP address will be the only destination
  • a ULP send message may optionally specify an address list of auxiliary addresses in addition to the primary destination of the implicit ULP address.
  • This auxiliary address list can contain only unicast and logical ULP addresses
  • the address list can also specify set operators to be performed between the sets of host ULP addresses defined by the unicast addresses and logical groups Once the address list has been processed to yield a set of hosts, this set is intersected with the set of hosts that are members of the implicit message group specified by the primary implicit ULP address in the message This ability to perform logical set operators on message groups is very usef l in interactive applications.
  • a ULP message would be sent to the ULP implicit message group for all the players in the area with an auxiliary address of the logical message group for all the players on the selected team
  • the GMS would perform the proper set intersection prior to sending the resulting messages to the targeted hosts. The result of this will be that the message will only be delivered to the players on the selected team in the selected area of the game.
  • the present invention deals with the issues of deploying an interactive application for multiple participants on wide area networks by providing a method for reducing the overall message rate and reducing latency
  • This invention uses a server group messaging approach, as oppose to the above described "distributed multicast messaging" approach.
  • the present invention overcomes the undesirable side effects of the distributed multicast messaging approach. Further, it reduces the message rate between the hosts. As pointed out in an example discussed above, with prior art distributed multicast messaging, each host will need to send only one message to the group containing all of the hosts seven times per-second, but will still receive 9 messages, seven times per-second.
  • the present invention of server group messaging has each host sending one message, seven times per-second and receiving one message, seven times per-second.
  • the present invention is different from the multicast routing and distribution method disclosed in U.S. Patent Nos. 4,740,954, 4,864,559, 5,361,256,
  • the present invention is also different from the source to destination multicast streams approach disclosed in EP 0 637 149 A2, PCT US94/11282 and PCT/US94/1 1278.
  • the data transmission is from a source to a plurality of designations, whereas the present invention describes data transmission from a sending host to a server host system and then from the server host to the destination hosts.
  • Figure 1 shows a conventional unicast network consisting of hosts, network links and routers.
  • Figure 2 shows the unicast datagrams on a conventional unicast network that would be needed to implement an interactive application between four hosts.
  • Figure 3 shows a prior art multicast network consisting of hosts, network links and multicast routers.
  • Figure 4 shows a multicast datagrams on a prior art multicast network that would be needed to implement an interactive application between four hosts.
  • Figure 5 shows a unicast network equipped with a group messaging server in accordance with the present invention.
  • Figure 6 shows the ULP datagrams without payload aggregation on a network according to the present invention that would be needed to implement an interactive application between four hosts.
  • Figure 7 shows the ULP datagrams with payload aggregation on a network according to the present invention that would be needed to implement an interactive application between four hosts.
  • Figure 8 shows a prior art ATM network with a multicast server.
  • FIG. 9 shows the detailed datagram format and address format for ULP messages in accordance with the present invention.
  • FIG. 10 shows the internal functions of the GMS according to the present invention.
  • Figure 11 shows the host software interface and functions needed to support the ULP according to the present invention.
  • the present invention provides a method for multiple host computers to efficiently communicate information to one another over a wide area network for the purposes of implementing an interactive application between multiple users.
  • the method consists of three components: a host protocol interface, a protocol and a server.
  • the protocol is between the host protocol interface and the server and is implemented on top of the network transport protocol of a wide area network.
  • the protocol is called the Upper Level Protocol (ULP) since it is layered above the existing network Transport Level Protocol (TLP).
  • ULP Upper Level Protocol
  • TLP Transport Level Protocol
  • the protocol can be described as a Session Layer protocol on top of the Transport Layer of the network.
  • Figure 11 shows the host protocol interface, 151, relative to the interactive application, 150, and the host interface for the Transport Level Protocol , 153.
  • the network interface, 155 provides the physical connection for the host to the network.
  • the network communications stack, 154 is the communications protocol stack that provides network transport services for the host and the host interface for the Transport Level Protocol, 153, is and interface between host application software and the network transport services of the network communications stack.
  • the interactive application can send and receive conventional network messages using the host interface to the TLP.
  • the interactive application also can send and receive ULP messages through the host interface for the ULP.
  • Internal to the host interface for the ULP is a table, 152, of all ULP addresses which the host can send messages to. Each entry in the table contains a pair of addresses, a ULP address and its corresponding TLP address.
  • a core function of the ULP is group messaging where hosts send messages to message groups populated by multiple hosts. This allows a host to send a message to multiple hosts with one ULP message. Since the ULP is layered on top of the TLP, the group messaging functions of the ULP operate on a conventional unicast network where TLP messages can only be sent from one host to only one other host
  • the group based messaging is implemented through the use of a server called a group messaging server
  • All ULP messages from the hosts are sent from the hosts to a group messaging server using the TLP protocol
  • the server processes the ULP portion of the messages and takes the necessary required by the ULP message
  • Control ULP messages are processed locally by the server and may be acknowledged to the sending host ULP messages addressed to other hosts are processed by the group messaging server and then re-transmitted to the proper ULP destination hosts, again using the TLP protocol to encapsulate and transport these messages
  • hosts 58, 59, 60 and 61 send messages to one another using the ULP over a conventional unicast network using a group messaging server 62
  • the network consists of conventional routers 63, 64, 65, 66, 67 and 68 connected with conventional network links 69, 70, 71, 72, 73, 74, 75, 76 and 77
  • Host 58 can send a message to hosts 59, 60 and 61 by sending a single ULP message to the group messaging server 62 where the ULP message specifies a destination address that is a ULP message group
  • the ULP message is encapsulated in a TLP message addressed to the group messaging server This causes the message to be properly routed by router 63 to network link 71 to router 67 to the server 62
  • the group messaging server receives the ULP message and determines that the message is addressed to a message group containing hosts 59, 60 and 61 as members
  • the server sends the payload of the received message to each of the hosts in three new ULP messages individually sent to the three hosts Since each message is encapsulated in a TLP message, the messages are properly carried over the conventional unicast network
  • the first ULP message is sent by the group messaging server to host 61.
  • This message is carried by network links 71, 70, 72 and 75 and routers 67, 63, 64 and 65.
  • the second ULP message is sent by the group messaging server to host 60. This message is carried by network links 71, 70, 73 and 76 and routers 67, 63, 64 and 66.
  • the third ULP message is sent by the group messaging server to host 61. This message is carried by network links 74 and 77 and routers 67 and 68.
  • the invention can be implemented both in a datagram form and in a connection oriented form. To best understand the details of the invention, it is best to first consider a datagram implementation.
  • the ULP can be implemented as a datagram protocol by encapsulating addresses, message type information and the message payload within a datagram of the underlying network transport protocol.
  • the general form of the ULP datagram message format is shown in Figure 9 as elements 123, 124, 125, 126, 127, 128 and 129.
  • the transport header 123 is the datagram header of the TLP that is encapsulating the ULP datagram.
  • the ULP message type field 124 indicates whether it is a send or receive message, if it is a control message or a state message. The following table shows the different message types.
  • the ULP message type field must be present in a ULP datagram.
  • Send messages are always sent from a host to a group messaging server. Messages from a group server to the hosts are always receive messages.
  • Send Control messages are messages from hosts to a group messaging server requesting a control function be performed.
  • Receive Control messages are acknowledgments from a group messaging server to the hosts in response to a prior Send Control messages.
  • the Send and Receive State messages are special cases of the Send and Receive Control messages that allow hosts to read and write application specific state storage in the group messaging server. The specific control functions supported by the ULP will be explained later.
  • the destination ULP address 125 is required in ULP datagrams and specifies the primary destination of the ULP message.
  • the address count field 126 is required in ULP send message types and is not present in ULP receive message types. When the address count field in a ULP send message is non-zero, it specifies the number of auxiliary destination addresses for the send message that follow the address count field. These auxiliary destination addresses are shown as items 127 and 128, but it is understood that there are as many auxiliary ULP destination addresses as specified by the address count field. Finally there is the payload 129.
  • the payload format for ULP datagrams is defined by items 1 16, 117, 118, 119, 120, 121 and 122.
  • Item 116 is the message count and defines how many payload elements will be contained in the payload.
  • a single payload element consists of a triplet of source ULP address, data length and data.
  • Items 1 17, 1 18 and 1 19 comprise the first payload element of the payload.
  • Item 1 17 is the ULP address of the source of the payload element
  • item 118 is the data length for the data in the payload element and item 119 is the actual data.
  • Items 120, 121 and 122 comprise the last payload element in the payload.
  • ULP send messages only support payloads with a single payload element, so the message count is required to be equal to one.
  • ULP receive messages may have payloads with one or more payload elements.
  • the address space of the ULP is divided into three segments: unicast host addresses, implicit group addresses and logical group addresses. All source and destination addresses in ULP must be in this address space.
  • the ULP address space is unique to a single group messaging server. Therefore each group messaging server has a unique ULP address space. Multiple group messaging servers may be connected to the network and hosts may communicate with multiple group messaging servers without confusion since each ULP datagram contains the header of the TLP. Different group messaging servers will have unique TLP addresses which can be used by the hosts to uniquely identify multiple ULP address spaces.
  • the format for ULP addresses is shown in Figure 9 comprised of items 130, 131 and 132.
  • the address format field 130 is a variable length field used to allow multiple address lengths to be supported.
  • the address type field 131 indicates the type of ULP address: unicast host, implicit group or logical group.
  • the encoding is as follows: Address Type Encoding
  • the address format encoding determines the length of the address field and therefore the total length of the ULP address. This encoding is shown below. Note that when the address type specifies a unicast host address, the low bit of the address type field is concatenated to the address field to become the most significant bit of the address. This doubles the size of the address space for unicast host addresses which is useful since there will generally be more hosts than group messaging servers. Address Format Encoding
  • ULP unicast host addresses are assigned to each host when it first connects to a group messaging server. When a host sends a message to other ULP address, the unicast ULP address of the host will appear as the source ULP address in the received payload element. Unicast ULP host addresses can also be used as destination addresses only as auxiliary addresses in a ULP send message. They are not allowed to be used to as the primary ULP destination address. This means that hosts cannot send ULP directly to one another, but always must send the messages to one another through a group messaging server.
  • Implicit group addresses are created by a group messaging server in response to a control message to the server requesting the creation of an implicit message group.
  • the host requesting the creation of the implicit message group becomes a member of the message group when it is created
  • Other hosts can send inquiry control messages to the group messaging server to learn of its existence and then send a implicit group join message in order to join the group.
  • the group messaging server maintains a list of ULP addresses of hosts that are members of the implicit message group.
  • Implicit ULP group addresses are the only ULP addresses allowed to be the primary destination of a ULP send message. Implicit ULP addresses will never appear as ULP source addresses in a payload element
  • Logical ULP addresses are used both to address logical message groups and for specifying set operations between the group members of the auxiliary ULP addresses in a ULP send message.
  • Logical message groups are created and joined similarly to implicit message groups, however, logical ULP addresses may only be used as auxiliary ULP addresses in a ULP send message
  • Logical ULP addresses will also never appear as source ULP addresses in a payload element. The support of set operations between message groups as part of a ULP send message will be explained in a later section on ULP send messages.
  • Group Messaging Server Internal Functions The internal components of the group messaging server are shown in Figure
  • the group messaging server is a general purpose computer system with a network interface to connect it to a wide area network.
  • Item 135 is the network interface for the group messaging server and includes not only the hardware connection to the network but the communications protocol stack used to implement the TLP on the server.
  • Item 136 is an overall control function for the group messaging server. This control function is responsible for all ULP messages that are sent or received by the GMS. Internal to this control function are several important storage and processing functions. Item 137 is an address map for all hosts currently connected to the GMS. This address map is a list of the ULP host address of each host connected to GMS and its corresponding TLP address. This enables the control function to construct the necessary TLP headers for sending ULP messages to the hosts connected to the GMS. Item 138 is a list of all of the currently active implicit ULP addresses currently recognized by the GMS. Item
  • 139 is an application specific state storage and processing function. Many interactive applications deployed over a network will be able to be implemented solely with host based processing. In these cases all data that needs to be sent between the hosts can be transported using the ULP. However, some applications will need maintain a centrally stored and maintained repository of application state information. This is useful when hosts may join or leave the application dynamically. When hosts join such an application, they will need a place from which they can obtain a snapshot of the current state of the application in order to be consistent with the other hosts that already where part of the application. To read and write this state storage area, the ULP supports send and receive state message types. Within these messages, there is the ability to access a state address space so that different portions of the state can be individually accessed. Application specific processing of state written into this state storage area can also be implemented.
  • Items 140 and 141 are two of multiple ULP server processes running on the GMS. These are software processes that are at the heart of the ULP. Each implicit ULP addresses recognized by the GMS has a one-to-one correspondence to a ULP server process and to a message group maintained by the process. Since all ULP send messages must have an implicit ULP address as the primary destination address of the message, every ULP send message is sent to and processed by a ULP server process. These processes are created by the GMS control function in response to ULP control messages to create new implicit ULP addresses. They are destroyed when the last host which is a member of its message group has left the message group.
  • a ULP server process Internal to a ULP server process is a list, 142, of the ULP host addresses of the members of the message group, a set of message queues 143 for each host which is a member of the message group and a message aggregation function 149 which is used to aggregate multiple messages to a single host into a single message.
  • Item 145 maintains a list of all of the logical ULP addresses and message groups in the GMS. Items 144 and 146 represent two of multiple logical ULP addresses. For each logical ULP address, there is a corresponding list, 147 and 148 of the host ULP addresses of the members of the logical message group.
  • the logical message groups are not tied to specific ULP server processes, but are global with a GMS to all of the ULP server processes. Control Functions
  • control functions consist of connect, disconnect, create group, close group, join group, leave group, query groups, query group members, query group attributes. These control functions are implemented by a ULP send and receive control messages. The control functions are initiated by a host sending a
  • ULP send control message to a GMS. These messages only allow a primary ULP destination address in the message and do no allow auxiliary addresses.
  • the primary ULP address is inte ⁇ reted as a control address space with a unique fixed address assigned to each of the control functions enumerated above.
  • the contents of data in the payload supplies any arguments needed by the control function.
  • Returned values from the control function are returned in a ULP receive control message that is addressed to the host that sent the original control message for which data is being returned. The detailed operation of these control functions is described below.
  • This control function allows a host to connect to a GMS.
  • ULP address in the message is a fixed address that indicates the connect function.
  • the source ULP address and any data in the payload are ignored.
  • the GMS control function, 136 creates a new host address and enters the host address in the host address map 136 along with the source TLP address from the TLP header of the message.
  • the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful host connection.
  • the destination ULP address in the message is the ULP address assigned to the host. The host saves this and uses it for any future messages to the GMS. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed host connection.
  • the destination ULP address in the message is a fixed address that indicates the disconnect function.
  • the source ULP address is used to remove the host from membership in any implicit or logical groups prior to disconnecting. Any data in the payload is ignored.
  • the GMS control function also removes the entry for the host from the host address map.
  • the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful host disconnection.
  • the destination ULP address in the message is the ULP address assigned to the host. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed host disconnection.
  • Create implicit group This function allows a host to create a new implicit message group and associated implicit ULP address and server process.
  • the payload in the message may contain a single payload item whose data field holds attributes of the group. These attributes can be used to define any optional functions of the group.
  • the destination ULP address in the message is a fixed address that indicates the create implicit group function.
  • the GMS control function allocates a new implicit ULP address, adds it to the implicit ULP address list 138 and creates a new ULP server process 140.
  • the host that sends this message is added to the membership list of the implicit group. This is done by adding the source ULP address in the message to the group membership list 142 in the ULP server process.
  • the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful implicit group creation.
  • the source ULP address in the payload is the ULP address assigned to the new implicit group. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed implicit group creation.
  • Create logical group This function allows a host to create a new logical message group and associated logical ULP address.
  • the payload in the message may contain a single payload item whose data field holds attributes of the group. These attributes can be used to define any optional functions of the group
  • the destination ULP address in the message is a fixed address that indicates the create logical group function.
  • the GMS control function allocates a new logical group
  • the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful logical group creation.
  • the source ULP address in the payload is the ULP address assigned to the new logical group. If there is an error, the control fiinction returns a message to the host with a function code in the data portion of the payload indicating failed implicit group creation.
  • This function allows a host to join an existing logical or implicit message group.
  • the destination ULP address in the message is a fixed address that indicates the join group function.
  • the data portion of the payload contains the ULP address of the group that is to be joined.
  • the GMS control function looks at this address and determines if it is an implicit or logical ULP address. If it is an implicit ULP address, the GMS control function finds the ULP server process selected by the address in the message payload and adds the source ULP host address from the message to the group membership list 142. If it is a logical ULP address, the GMS control function finds the logical ULP address 144 selected by the address in the message payload and adds the source ULP host address from the message to the group membership list 147.
  • the GMS control function Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful group join.
  • the source ULP address in the payload is the ULP address of the group that was joined. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed implicit group creation.
  • This function allows a host to leave an existing logical or implicit message group that it is a member of.
  • the destination ULP address in the message is a fixed address that indicates the leave group function.
  • the data portion of the payload contains the ULP address of the group that is to be left.
  • the GMS control function looks at this address and determines if it is an implicit or logical ULP address. If it is an implicit ULP address, the GMS control function finds the ULP server process selected by the address in the message payload and removes from the group membership list 142 the source ULP host address from the message. If the host is the last member of the group, the ULP server process is terminated and the implicit ULP address is de-allocated.
  • the GMS control function finds the logical ULP address 144 selected by the address in the message payload and removes from the group membership list 147 the source ULP host address from the. If the host is the last member of the group, the ULP address is de-allocated. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful group leave. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed implicit group creation.
  • This function allows a host to get a list of all implicit and logical message groups currently active on a GMS.
  • the destination ULP address in the message is a fixed address that indicates the query groups function. Any data portion of the payload is ignored.
  • the GMS control function responds with a receive control ULP message addressed to the host along with a payload with multiple payload elements.
  • the first payload element contains a function code indicating successful query groups.
  • the source ULP address in the first payload element is ignored.
  • Each of the subsequent payload elements contain a ULP group address in the source address field of the payload element that is one of the active group addresses on the GMS. There is no data field in these subsequent payload elements. If there is an error, the control function returns a message to the host with a function code in the data portion of a payload with a single payload element indicating failed query groups.
  • This function allows a host to get a list of all hosts that are members of a message group.
  • the destination ULP address in the message is a fixed address that indicates the query group members function.
  • the data portion of the payload carries the address of the message group for the query.
  • the GMS control function responds with a receive control ULP message addressed to the host along with a payload with multiple payload elements.
  • the first payload element contains a function code indicating successful query group members.
  • the source ULP address in the first payload element is ignored.
  • Each of the subsequent payload elements contain a ULP host address in the source address field of the payload element that is one of the active group addresses on the GMS. There is no data field in these subsequent payload elements. If there is an error, the control function returns a message to the host with a function code in the data portion of a payload with a single payload element indicating failed query group members.
  • This function allows a host to get a list of the attributes of a message group.
  • the destination ULP address in the message is a fixed address that indicates the query group attributes function.
  • the data portion of the payload carries the address of the message group for the query.
  • GMS control function responds with a receive control ULP message addressed to the host along with a payload with a two payload elements.
  • the first payload element contains a function code indicating successful query group members.
  • the second payload element contains the attributes of the message group. If there is an error, the control function returns a message to the host with a function code in the data portion of a payload with a single payload element indicating failed query group attributes.
  • the auxiliary address count 126 is zero and there are no auxiliary destination addresses 127 or 128.
  • the payload consists of a message count 116 of one, the ULP of the host sending the message in the source ULP address 117 and the data length 118 and data 119.
  • Send message datagrams may only have a single payload item so their message count field 116 must always be one.
  • the host sends the send message onto the network with a TLP header addressing the datagram to the GMS that is the selected target of the message.
  • the GMS receives the message and the GMS control function 136 determines that it is a send message datagram and looks up the implicit destination address in its implicit ULP address list 138. If the address does not exist, an error message is returned to the sending host with a ULP receive message datagram. If the address is valid, the GMS control function removes the TLP header from the datagram and sends the ULP portion to the ULP server process corresponding to the destination implicit ULP address. Assume for discussion that this is the ULP server process 140. The ULP server process 140 will extract the single payload item from the message 117, 1 18 and 119 and place the payload item in each of the message queues 143. There will be one message queue for each member of the message group served by the ULP server process 140.
  • the members of the group will have their host ULP addresses listed in the host address list 142.
  • Each message queue in a ULP server process will fill with payload items that are targeted at particular destination hosts. The mechanisms by which payload items are removed from the queues and sent to the hosts will be described later.
  • auxiliary address 127 is one and the auxiliary destination addresses 127 is a unicast host ULP address.
  • the payload consists of a message count 116 of one, the ULP of the host sending the message in the source ULP address 117 and the data length 118 and data 119.
  • the host sends the send message onto the network with a TLP header addressing the datagram to the GMS that is the selected target of the message.
  • the GMS receives the message and the GMS control function 136 determines that it is a send message datagram and looks up the implicit destination address in its implicit ULP address list 138 and the unicast host ULP auxiliary address in the host address map 137. If either of addresses does not exist, an error message is returned to the sending host with a ULP receive message datagram. If the addresses are valid, the GMS control function removes the TLP header from the datagram and sends the ULP portion to the ULP server process corresponding to the destination implicit ULP address. Assume for discussion that this is the ULP server process 140. The ULP server process extracts the auxiliary ULP address from the message and determines from the address that it is a unicast host ULP address.
  • the server process then checks to see if this address is a member of the message group defined by the host address list 142. If it is not, no further action is taken and the payload item in the message is not placed in any of the message queues 143. If the host address is in the message group, the payload item in the message is placed in the single message queue corresponding to that host.
  • the net effect is that the ULP server process has performed a set intersection operation on the members of the message group selected by the implicit ULP destination address and defined by the group membership list 142 with the members of the set of hosts defined by the auxiliary address. The payload item is them sent only to the hosts that are members of this set intersection.
  • auxiliary address 127 in addition to an implicit destination 125, there is also a single auxiliary address 127 in the datagram.
  • the auxiliary address count 126 is one and the auxiliary destination addresses 127 is a logical ULP address.
  • the payload consists of a message count 116 of one, the ULP of the host sending the message in the source ULP address 117 and the data length 118 and data 119.
  • the host sends the send message onto the network with a TLP header addressing the datagram to the GMS that is the selected target of the message.
  • the GMS receives the message and the GMS control function 136 determines that it is a send message datagram and looks up the implicit destination address in its implicit ULP address list 138 and the logical ULP auxiliary address in list of logical ULP addresses 145. If either of addresses does not exist, an error message is returned to the sending host with a ULP receive message datagram. If the addresses are valid, the GMS control function removes the TLP header from the datagram and sends the ULP portion to the ULP server process corresponding to the destination implicit ULP address. Assume for discussion that this is the ULP server process 140.
  • the ULP server process extracts the auxiliary ULP address from the message and determines from the address that it is a logical ULP address. Assume for this example that this logical ULP address is the logical address 144.
  • the server process fetches the group membership list 147 corresponding to the logical address and performs a set intersection operation with the group membership list 142 of the server process. If there are no members of this set intersection, no further action is taken and the payload item in the message is not placed in any of the message queues 143. If there are members of the set intersection operation, the payload item in the message is placed in the queues corresponding to the hosts that are members of the set intersection.
  • a send message can perform set operations between the implicit message group of the ULP server process and multiple logical and unicast ULP addresses. This is done by placing multiple auxiliary destination ULP addresses in the message with logical operators imbedded in the address list.
  • the address count 126 holds a count of the total auxiliary addresses in the address list 127 and 128.
  • the auxiliary addresses are a mix of logical ULP addresses and unicast host ULP addresses. Two logical ULP addresses in the ULP address space are assigned the role of specifying set operations to be performed between the logical message groups and unicast host addresses in the message list. They are specially assigned addresses for the functions set intersection, set union. A third logical address is used to indicate set complement.
  • the payload consists of a message count 116 of one, the ULP of the host sending the message in the source ULP address 1 17 and the data length 118 and data 119.
  • the host sends the send message onto the network with a TLP header addressing the datagram to the GMS that is the selected target of the message.
  • the GMS receives the message and the GMS control function 136 determines that it is a send message datagram and looks up the implicit ULP message in the implicit ULP address list 138 and all of the addresses in the address list either in the host ULP address map 137 or in the logical ULP address list 145 as appropriate. If any of addresses does not exist, an error message is returned to the sending host with a ULP receive message datagram. If the addresses are valid, the GMS control function removes the TLP header from the datagram and sends the ULP portion to the ULP server process corresponding to the destination implicit ULP address.
  • the ULP server process extracts the auxiliary ULP address list from the message and scans it from beginning to end.
  • the scanning and processing of the set operators is done in post-fix fashion. This means that arguments are read followed by an operator that is then applied to the arguments. The result of the operator becomes the first argument of the next operation. Therefore at the start of scanning two addresses are read from the address list.
  • the next address will be an operator that is applied to the arguments and the result of this operator is the first argument to be used by the next operator. From then on a single address is read from the address list followed by a logical ULP address which is operator on the two arguments consisting of the new argument and the results of the last operator.
  • the logical address used to indicate set complement is not a set operator, by an argument qualifier since it can precede any address in the address list.
  • the meaning of the set complement argument qualifier is relative to the group membership of implicit group address in the send message. If the set complement qualifier precedes a unicast host address which is not a member of the message group selected by the implicit ULP address in the send message, the effective argument is the set of all hosts that are members of the implicit message group. If the set complement qualifier precedes a unicast host address which is a member of the message group selected by the implicit ULP address in the send message, the effective argument is the set of all hosts that are members of the implicit message group except for the original unicast host address qualified by the complement function.
  • the effective argument is the set of all hosts that are members of the implicit message group specified by the send message except hosts that are members of the logical message group preceded by the set complement modifier.
  • the delivery method is set on a per-ULP server process basis by attributes that are provided at the time that an implicit ULP message group and server process are created. It is important during the description of these methods to keep in mind that the invention is intended to provide an efficient means for a group of hosts to send messages to each other at a rapid rate during the implementation of a networked interactive application. Also assumed in the following description is that the GMS performs echo suppression when a host sends a message to a group that it belongs to.
  • the most simple delivery method is to immediately deliver the payload items to their targeted hosts as soon as they are placed in the message queues.
  • Each payload item in a message queue will contain a ULP source address, a data length and the data to be sent.
  • the ULP server process will remove a payload item from a message queue for a particular host 143.
  • the host address for this host will be obtained from the group membership list 142.
  • the payload item and the destination host address will be sent to the GMS control function 136 where it will be used to create a ULP receive message sent to the destination host.
  • the GMS control function 136 will use the destination ULP host address to look up the TLP address of the host from the host address map 137.
  • the ULP message type 124 will be ULP receive, the destination ULP address 125 will be the destination host, the address count will be 0 and there will be no auxiliary addresses.
  • the payload in this case will have a message count 116 of 1 and the payload item comprised of fields 117, 118, and 1 19 will be the payload element taken from the message queue.
  • Immediate delivery is useful when the message rate between a group of hosts is low.
  • hosts that are members of an implicit message group where each member of the group sends a message to every other member of the group at a fixed rate.
  • each host will send three messages to the other members of the group and receive three messages from the other members of the group at the fixed rate. This is acceptable is the size of the group is small and the message rate is low.
  • total message rate is the product of the underlying message rate and the total number of members of the group minus one.
  • a key concept in the present invention is the aggregation of multiple messages in a message queue into a single ULP receive message to a host that contains multiple payload items in the payload.
  • the ULP server process 140 removes payload items from a message queue 143 for a host and accumulates them in an aggregation buffer 149.
  • the aggregation buffer has buffer areas for each host for which there is a message queue. These individual host areas within the aggregation buffer are called host aggregation buffers. The start and end of this aggregation period can be controlled in a number of ways that will be described in the next sections. At the end of the aggregation period, the each host aggregation buffer may hold multiple payload items.
  • the host aggregation buffer will hold a message count of the payload items followed by the multiple payload items.
  • the contents of a host aggregation buffer along with the ULP host address of the corresponding host are sent to the GMS control function 136 where it will be used to create a ULP receive message sent to the destination host.
  • the GMS control function 136 will use the destination ULP host address to look up the TLP address of the host from the host address map 137. This will be used to create a TLP header for the message 123.
  • the payload in this case will have a message count 1 16 set by the message count value from the host aggregation buffer.
  • the payload will contain all of the payload items from the host aggregation buffer.
  • the effect of aggregation will be to greatly reduce the total message rate received by the hosts.
  • a single message to a host will be able to carry multiple payload items received from the other hosts during the aggregation period.
  • This fits very well the interactive applications of this invention where groups of hosts will be sending messages to all the other hosts in the group at a periodic rate.
  • Aggregation will be very effective in collecting together all of the messages from all of the other hosts into a single message for each member of the group. The reduces processing at each receiving host since a single message will be received rather than many separate messages.
  • Aggregation will also reduce the total data rate to the hosts since aggregation eliminates the need for separate message headers for each payload item. The savings will be significant for small payload items since there will be only one message header comprising fields 123, 124 and
  • the first data element in the first payload item can be sent in uncompressed form with each subsequent data element being compressed using some form of difference coding method.
  • a variety of known data compression methods use the concept of a predictor with differences from the predicted value being encoded.
  • the first data element in an aggregated payload can be used as this predictor with the subsequent data elements coded using such a data compression method.
  • These conventional data compression methods do not assume any knowledge of the internal structure or function of portions of a data element to compress. It is also possible to make use of application specific coding techniques that take advantage of such knowledge to potentially achieve much higher coding efficiency.
  • a ULP Server Process defines a uniform time base for defining the aggregation time period. This time base is defined by three parameters: the time period, the aggregation offset and the transmit offset. These parameters are set by the attributes provided in the create implicit group control function at the time the implicit group and the ULP server process are created.
  • the time period is a fixed time interval during which the ULP server process will accumulate messages in the message queues, aggregate the messages in the queues and send the aggregated messages to the targeted hosts.
  • the aggregation offset defines the point after the start of the time period after which arriving messages will be stored in the message queues for delivery in the next time period.
  • a snapshot will be taken of all of the messages in each message queue. New messages will continue to arrive and be entered into the queues after the aggregation offset. Only those messages in the queues before the aggregation offset point will be aggregated into outbound messages. The resulting aggregated messages will then be sent to their targeted hosts at the point in time which is the transmit offset after the start of the time period. The result is that messages arrive continuously and are stored in the message queues. Once per time period the are aggregated into single messages to each host which is the target of messages and once per time period these aggregated messages are sent to the hosts.
  • Another embodiment of the SI method is to allow the ULP server process to dynamically vary the time period based on some criteria such as the received message rates, and/or received data rate.
  • the ULP server could use a function to define the aggregation period based on the number of messages received per second or the total number of payload bytes received per second.
  • One reasonable function would be to shorten the aggregation period as the rate or received messages or data rate of the received payloads increased. This would tend to keep the size of the outbound messages from growing too much as received messages and/or received data rate grew.
  • Other possible functions could be used that varied the aggregation period based on received message rates, received payload data rates or other parameters available to the ULP server process.
  • the host synchronous or HS method of defining the aggregation time period allows the definition of a flexible time period that is controlled by the hosts. It is based on the concept of a turn which is a host sending a message to one or more members of the implicit message group which is operating is HS mode. Once every host in the message group has taken a turn, the aggregation period ends. A snapshot of the contents of the message queues is taken, the contents of each of the queues is aggregated and the aggregated messages are sent to the hosts targeted by each message queue.
  • a refinement to this technique qualifies which of the three ULP send message types to the group constitute a host turn: a send only to the implicit address of the group, a send to a unicast host address within the group or a send to a logical ULP address which shares members with the group.
  • the attributes of the group not only will define HS aggregation, but one or more ULP send message types that will be considered a host turn.
  • a further refinement sets the total number of turns that a host can take in a single aggregation time period. The default will be one turn, but multiple turns can be allowed. If a host attempts to take more turns than allowed, the messages are ignored.
  • This aggregation technique has the additional benefit of causing the hosts which are member of an HS implicit message group to have their processing functions synchronized when they are executing the same interactive application.
  • Many networked interactive applications are based on a simple overall three step operational model: wait for messages from other hosts, process the messages and the local users inputs to update the local application, send messages to the other hosts.
  • This basic application loop is repeated at a rate fast enough to provide an interactive experience such as 5 to 30 times per second. It is desirable to keep such applications synchronized so that the states of the applications is consistent on the different host machines.
  • When such applications communicate using the HS model of the present invention their operations will become naturally synchronized.
  • the HS ULP server process will wait until all of the members of the message group has completed their turns and sent a message to the group before sending the aggregated messages to the members of the group. This will cause the applications on the hosts to wait until they have received the aggregated messages.
  • the wide area network is the Internet and the TLP protocol is TCP/IP.
  • the GMS is a general pu ⁇ ose computer system connected to the Internet and the hosts are personal computers connected to the Internet.
  • TCP/IP provides an number of advantages that provide for a more efficient applications interface on the hosts 151.
  • TCP/IP supports the concept of source and destination port numbers in its header.
  • the ULP can make use of the port numbers to identify source and destination ULP connections.
  • Most ULP send messages will be from hosts to a implicit ULP group addresses and most ULP receive messages will be from the implicit ULP addresses to the ULP host addresses. All of these and the ULP message type field can represented by source and destination port addresses within the TCP/IP header. This means that for most ULP messages, the ULP message encapsulated within the TCP/IP message need only contain the payload.
  • the destination port will be the host the source port will be for the implicit ULP group address and the payload will still contain the source host ULP addresses in each the payload items.
  • TCP/IP also supports header compression for low speed dial-up lines which is also important in this application. See RFC 1144.
  • TCP/IP is a connection oriented protocol which provides reliable end-to-end transport. It handles re ⁇ transmission on errors and fragmentation and reassembly of data transparently to upper level protocols. Header compression allows much of the TCP/IP header to be omitted with each packet to be replaced by a small connection identifier. This connection ID will uniquely define a connection consisting of a source and destination IP address and source and destination TCP/IP port numbers.
  • the preferred embodiment of the ULP is as a session layer protocol
  • the application on a host opens a session with a ULP server process This session is identified with a unique session ID on the host.
  • the host application then sends data to the ULP host interface 151 tagged with this session ID.
  • the session ID defines a host and implicit ULP pair including the TCP/IP TLP address of the GMS server that is running the particular ULP server process for the implicit ULP address
  • One possible extension to the invention is to extend the ULP to support a common synchronized time base on the GMS and the hosts that are connected to it. This would be most interesting in context of the SI message aggregation mode.
  • the SI time base on the GMS could be replicated on all of the hosts and all of the hosts and the GMS could lock these time bases together.
  • ULP server processes that perform specific application specific processing on the contents of the messages that are received.
  • a variety of different application specific processing functions can be defined and implemented. A particular function would be selected by attributes provided in the create implicit group function. These functions could process the data in the message payloads and replace the data elements in the payloads with processed results. Separately, or in combination with processing the message payloads, the processing could store either raw message payload data in the application specific state storage area or could store processed results.
  • the host system need not be personal computers, but could also be dedicated game consoles or television set top boxes or any other device with a programmable controller capable of implementing the ULP protocol.
  • the wide area network used to transport the ULP protocol need not be the Internet or based on IP.
  • Other networks with some means for wide area packet or datagram transport are possible including ATM networks or a digital cable television network.

Abstract

A method for deploying interactive applications over a network containing host computers (58, 59, 60, 61) and group messaging servers (62) is disclosed. The method operates in a conventional unicast network architecture comprised of conventional network links and unicast gateways and routers. The hosts send messages containing destination group addresses by unicast to the group messaging servers. The group addresses select message groups maintained by the group messaging servers. For each message group, the group messaging servers also maintain a list (142) of all of the hosts that are members of the particular group. In its most simple implementation, the method consists of the group server receiving a message from a host containing a destination group address. Using the group address, the group messaging server then selects a message group which lists all of the host members of the group which are the targets of messages to the group. The group messaging server then forwards the message to each of the target hosts. In an interactive application, many messages will be arriving at the group server close to one another in time. Rather than simply forward each message to its targeted hosts, the group messaging server aggregates the contents of each message received during a specified time period and then sends an aggregated message to the targeted hosts. This method can reduce the message traffic between hosts in a networked interactive application and contributes to reducing the latency in the communications between the hosts.

Description

SERVER-GROUP MESSAGING SYSTEM FOR INTERACTIVE APPLICATIONS
FIELD OF THE INVENTION
The present invention relates to computer network systems, and particularly to server group messaging systems and methods for reducing message rate and latency
Background of the Invention
There are a wide range of interactive applications implemented on computer systems today All are characterized by dynamic response to the user The user provides input to the computer and the application responds quickly One popular example of interactive applications on personal computers (PCs) are games. In this case, rapid response to the user may mean redrawing the screen with a new picture in between 30ms and 100ms Interactive applications such as games control the speed of their interaction with the user through an internal time base The application uses this time base to derive rates at which the user input is sampled, the screen is redrawn and sound is played
As computers have become more powerful and common, it has become important to connect them together in networks A network is comprised of nodes and links The nodes are connected in such a way that there exists a path from each node over the links and through the other nodes to each of the other nodes in the network Each node may be connected to the network with one or more links Nodes are further categorized into hosts, gateways and routers Hosts are computer systems that are connected to the network by one link They communicate with the other nodes on the network by sending messages and receiving messages Gateways are computer systems connected to the network by more than one link. They not only communicate with the other nodes as do hosts, but they also forward messages on one of their network links to other nodes on their other network links. This processing of forwarding messages is called routing. In addition to sending and receiving messages and their routing functions, gateways may perform other functions in a network. Routers are nodes that are connected to the network by more than one link and whose sole function is the forwarding of messages on one network link to the other network links to which it is connected. A network consisting of many network links can be thought of as a network of sub-networks with gateways and/or routers connecting the sub-networks together into what is called an internet. Today the widely known example of a world wide internet is the so called "Internet" which in 1995 has over 10 million computers connected full time world-wide.
With so many computers on a single world-wide network, it is desirable to create interactive networked applications that bring together many people in a shared, networked, interactive application. Unfortunately, creating such shared, networked, interactive applications runs into the limitations of the existing network technology.
As an example, consider a game designed to be deployed over a network which is to be played by multiple players simultaneously. The game could be implemented in software on a PC connected to a network. A rate set by its internal time base, it would sample the inputs of the local user, receive messages from the network from the PCs of the other players and send messages out to the PCs of the other players. A typical rate will be ten time per second for a time period of 100ms. The messages sent between the PCs would contain information that was needed to keep the game consistent between all of the PCs. In a game that created the illusion of a spatial environment where each player could move, the packets could contain information about the new positions of the players as they moved. Today there are many commercial example of PC games that can be played between multiple players on Local Area Networks (LANs) or by two players over dial-up phone lines using modems. The network messages sent by such games contain a wide variety of information specific to the game. This can include position and velocity information of the objects in the game along with special actions taken by a player that effect the other players in the game.
The case of a two player game played over a modem is particularly simple. If the message rate is 10 messages per second, each PC sends 10 messages per second to the other PC and receives 10 messages per second. The delay introduced by the modems and phone line is small and will not be noticed in most games. Unfortunately, the case of two players is uninteresting for networked interactive applications. With the same game played with 8 players on a LAN, the message rate increases. Each PC must send 7 messages, one to each of the other 7 players every time period and will receive 7 messages from the other players in the same time period. If the messaging time period is 100ms, the total message rate will be 70 messages sent per second and 70 messages received per second. As can be seen the message rate increases linearly with the number of players in the game. The message rates and data rates supported by popular LANs are high enough to support a large number of players at reasonable message sizes. Unfortunately, LANs are only deployed in commercial applications and cannot be considered for deploying a networked interactive application to consumer users.
The wide area networks available today to consumer users all must be accessed through dial-up phone lines using modems. While modem speeds have increased rapidly, they have now reached a bit rate of 28.8 Kbits/sec which is close to the limit set by the signal-to-noise ratio of conventional phone lines.
Further speed increases are possible with ISDN, but this technology is not ready for mass market use. Other new wide area networking technologies are being discussed that would provide much higher bandwidth, but none are close to commercial operation Therefore, in deploying a networked, interactive application to consumers, it is necessary to do so in a way that operates with existing networking and communications infrastructures
In the example of the 8 player networked game, consider a wide area network implementation where the PCs of each of the players is connected to the network with a 28 8 Kbit/sec modem Assume that the network used in this example is the Internet so that all of the network protocols and routing behavior is well defined and understood If the game uses TCP/IP to send its messages between the PCs in the game, the PPP protocol over the dial-up phone lines can be advantageously used to compress the TCP/IP headers Even so, a typical message will be approximately 25 bytes in size Sent through the modem, this is 250 bits The messages are sent 10 times per second to each of the other PCs in the game and received 10 times per second from the other PCs This is 35 0 Kbits/sec which exceeds the capabilities of the modem by 20% If the messages are reduced to 20 bytes, just 8 players can be supported, but this approach clearly cannot support networked interactive applications with large numbers of participants There are other problems beyond just the bandwidth of the network connection There is the loading on each PC caused by the high packet rates and there is the latency introduced by the time needed to send all of the outbound packets Each packet sent or received by a PC will require some amount of processing time As the packet rate increases with the number of players in the game, less and less of the processor will be available for running the game software itself Latency is important in an interactive application because it defines the responsiveness of the system When a player provides a new input on their system, it is desirable for that input to immediately affect the game on all of the other players systems This is particularly important in any game where the game outcome depends on players shooting at targets that are moved by the actions of the other players Latency in this case will be the time from when a player acts to move a target to the time that the target has moved on the screens of the other players in the game. A major portion of this latency will come from the time needed to send the messages to the other seven players in the game. In this example the time to send the messages to the other 7 players will be approximately 50 ms. While the first player of the seven will receive the message quickly, it will not be until 50 ms have passed that the last player of the seven will have received the message. Internet Protocol Multicasting
As mentioned before, the Internet is a widely known example of a wide area network. The Internet is based on a protocol appropriately called the Internet
Protocol (IP). In the OSI reference model for layers of network protocols, IP corresponds to a layer 3 or Network layer protocol. It provides services for transmission and routing of packets between two nodes in an internet. The addressing model provides a 32 bit address for all nodes in the network and all packets carry source and destination addresses. IP also defines the routing of packets between network links in an inter-network. Gateways and routers maintain tables that are used to lookup routing information based on the destination addresses of the packets they receive. The routing information tells the gateway/router whether the destination of the packet is directly reachable on a local network link connected to the gateway/router or if not, the address of another gateway/router on one of the local network links to which the packet should be forwarded. On top of IP are the layer 4 transport protocols TCP and UDP. UDP provides datagram delivery services to applications that does not guarantee reliable or in-order delivery of the datagrams. TCP is a connection oriented service to applications that does provide reliable delivery of a data stream. It handles division of the stream into packets and ensures reliable, in- order delivery. See the Internet Society RFCs: RFC-791 "Internet Protocol", RFC-793 "Transmission Control Protocol" and RFC- 1180 "A TCP/IP Tutorial". IP, TCP and UDP are unicast protocols: packets, streams or datagrams are transmitted from a source to a single destination.
As an example, consider Figures 1 and 2. Figure 1 shows a conventional unicast network with hosts 1, 2, 3 and 4 and network links 11, 12, 13, 14, 15,16,17, 18 and 19 and routers 5, 6, 7, 8, 9 and 10. In this example, each host wants to send a data payload to each of the other hosts. Host 1 has network address A, host 2 has network address C, host 3 has network address B and host 4 has network address D. Existing network protocols are typically based on packet formats that contain a source address, destination address and a payload. This is representative of commonly used wide area network protocols such as IP.
There are other components in an actual IP packet, but for sake of this example, only these items will be considered. Figure 2 shows the example packets that are sent by the hosts to one another using a conventional unicast network protocol such as IP. Host 1 send packets 20, to host 3, packet 21 to host 2 and packet 22 to host 4. Host 1 wants to send the same data Pl to each of the other three hosts, therefore the payload in all three packets is the same. Packet 20 travels over network links 11, 12, 15 and 18 and through routers 5, 6, and 8 to reach host 3. In a similar fashion host 3 sends packets 23 to host 1, packet 24 to host 2 and packet 25 to host 4. Host 2 and host 4 send packets 26, 27, 28 and 29, 30, 31 respectively to the other three hosts. All of these packets are carried by the unicast network individually from the source host to the destination host. So in this example each host must send three packets and receive three packets in order for each host to send its payload to the other three hosts.
As can be seen, each host must send a packet to every other host that it wishes to communicate with in an interactive application. Further, it receives a packet from every other host that wishes to communicate with it. In an interactive application, this will happen at a regular and high rate. All of the hosts that wish to communicate with one another will need to send packets to each other eight to ten times per second. With four hosts communicating with one another as in this example, each host will send three messages and receive three messages eight to ten times per second. As the number of hosts in the application that need to communicate with one another grows, the message rate will reach a rate that cannot be supported by conventional dial-up lines. This makes unicast transport protocols unsuitable for delivering interactive applications for multiple participants since their use will result in the problem of high packet rates that grow with the number of participants.
Work has been done to attempt to extend the IP protocol to support multicasting. See RFC-1112 "Host Extensions for IP Multicasting ". This document describes a set of extensions to the IP protocol that enable IP multicasting. IP multicasting supports the transmission of a IP datagram to a host group by addressing the datagram to a single destination address. Multicast addresses are a subset of the IP address space and identified by class D IP addresses - these are IP addresses with "11 10" in the high order 4 bits. The host group contains zero or more IP hosts and the IP multicasting protocol transmits a multicast datagram to all members of the group to which it is addressed. Hosts may join and leave groups dynamically and the routing of multicast datagrams is supported by multicast routers and gateways. It is proper to describe this general approach to multicast messaging as "distributed multicast messaging". It is a distributed technique because the job of message delivery and duplication is distributed throughout the network to all of the multicast routers. For distributed multicast messaging to work in a wide area network, all of the routers handling datagrams for multicast hosts must support the routing of multicast datagrams. Such multicast routers must be aware of the multicast group membership of all of the hosts locally connected to the router in order to deliver multicast datagrams to local hosts. Multicast routers must also be able to forward multicast packets to routers on their local network links Multicast routers must also decide to which if any local routers they must forward multicast datagrams. When a multicast datagram is received, by a multicast router, its group address is compared to a list for each local multicast router of group addresses. When there is a match, the datagram is then forwarded to that local multicast router Therefore, the multicast routers in the network must maintain an accurate and up to date list of group addresses for which they are to forward datagrams to These lists are updated when hosts join or leave multicast groups Hosts do this by sending messages using Internet Group Management Protocol (IGMP) to their immediately-neighboring multicast routers A further attribute of distributed multicast messaging is that the routers must propagate the group membership information for a particular group throughout the network to all of the other routers that will be forwarding traffic for that group RFC-1 112 does not describe how this is to be done Many different approaches have been defined for solving this problem that will be mentioned later in descriptions of related prior art Despite their differences, all of these approaches are methods for propagation of multicast routing information between the multicast routers and techniques for routing the multicast datagrams in an inter-network supporting distributed multicast messaging The distributed multicast messaging approach has a number of undesirable side effects. The process of propagation of group membership information to all of the relevant routers is not instantaneous In a large complex network it can even take quite a period of time depending on the number of routers that must receive that updated group membership information and how many routers the information for the group membership update must past through This process can easily take many seconds and even minutes depending on the specifics of the algorithm that is used RFC-1112 mentions this problem and some of the side effects that must be handled by an implementation of a practical routing algorithm for multicast messaging. One problem results when groups are dynamically created and destroyed. Since there is no central authority in the network for assigning group addresses, it is easily possible in a distributed network for there to be duplication of group address assignment. This will result in incorrect datagram delivery, where hosts will receive unwanted datagrams from the duplicate group. This requires a method at each host to filter out the unwanted datagrams. Another set of problems result from the time delay from when a group is created, destroyed or its membership changed to when all of the routers needed to route the datagrams to the member hosts have been informed of these changes. Imagine the case where Host N joins an existing group by sending a join message to its local router. The group already contains Host M which is a number of router hops away from Host N in the network Shortly after Host N has sent it join message, Host M sends a datagram to the group, but the local router of Host M has not yet been informed of the change in group membership and as a result the datagram is not forwarded to one of the particular network links connected to the local router of Host M that is the only path in the network from that router that ultimately will reach Host N. The result is that Host N will receive no datagrams addressed to the group from Host M until the local router of M has its group membership information updated Other related problems can also occur When a host leaves a group, messages addressed to the group will continue for some time to be routed to that host up to the local router of that host. The local router will know at least not to route the datagram onto the local network of that host. This can still result in a great deal of unnecessary datagrams being carried in a large network when there are many active message groups with rapidly changing memberships.
Finally, distributed multicast messaging does not sufficiently reduce the message rate between the hosts. With distributed multicast messaging, each host need only send one message addressed to the message group in order to send a message to all of other hosts in the group. This is an improvement over conventional unicast messaging where one message would need to be sent to each of the other hosts in a group. However, distributed multicast messaging does nothing to reduce the received message rate at each of the hosts when multiple hosts in a group are sending messages to the group closely spaced in time. Let us return to the example of a group often hosts sending messages seven times per-second to the group. With conventional unicast messaging, each host will need to send 9 messages to the other hosts, seven times per-second and will receive 9 messages, seven times per-second. With distributed multicast messaging, each host will need to send only one message to the group containing all of the hosts seven times per-second, but will still receive 9 messages, seven times per-second. It is desirable to further reduce the number of received messages.
An example of distributed multicasting is shown in Figures 3 and 4. Figure 3 shows a network with multicast routers 39, 40, 41, 42, 43 and 44 and hosts 35,
36, 37, 38 and network links 45, 46, 47, 48, 49, 50, 51, 52 and 53. The four hosts have unicast network addresses A, B, C, D and are also all members of a message group with address E. In advance the message group was created and each of the hosts joined the message group so that each of the multicast routers is aware of the message group and has the proper routing information. A network protocol such IP with multicast extensions is assumed to be used in this example. Host 35 sends packet 54 with source address A and destination multicast address E to the entire message group. In the same manner host 37 sends packet 55 to the group, host 36 sends packet 56 to the group and host 38 sends packet 57 to the group. As the packets are handled by the multicast routers they are replicated as necessary in order to deliver them to all the members of the group. Let us consider how a packets sent by host 35 is ultimately delivered to the other hosts. Packet 54 is carried over network link 45 to multicast router 39. The router determines from its routing tables that the multicast packet should be sent onto network links 46 and 47 and duplicates the packet and sends to both of these network links. The packet is received by multicast routers 40 and 43. Multicast router 43 sends the packet onto network link 50 and router 40 sends its onto links 48 and 49. The packet is then received at multicast routers 44, 42 and 41. Router 41 sends the packet over network link 51 where it is received by host 36. Router 42 sends the packet over network link 52 to host 37 and router 44 sends the packet over link 53 to host 38. A similar process is followed for each of the other packets sent by the hosts to the multicast group E. The final packets received by each host are shown in
Figure 4.
While distributed multicasting does reduce the number of messages that need to be sent by the hosts in a networked interactive application, it has no effect on the number of messages that they receive. It has the further disadvantages of poor behavior when group membership is rapidly changing and requires a special network infrastructure of multicast routers. It also has no support for message aggregation and cannot do so since message delivery is distributed. Distributed multicasting also has no support for messages that define logical operations between message groups and unicast host addresses. All of these problems can be understood when placed in context of the design goals for distributed multicast messaging. Distributed multicast messaging was not designed for interactive applications where groups are rapidly created, changed and destroyed. Instead it was optimized for applications where the groups are created, changed and destroyed over relatively long time spans perhaps measured in many minutes or even hours. An example would be a video conference where all the participants agreed to connect the conference at a particular time for a conference that might last for an hour. Another would be the transmission of an audio or video program from one host to many receiving hosts, perhaps measured in the thousands or even millions. The multicast group would exist for the duration of the audio/video program. Host members would join and leave dynamically, but in this application it would be acceptable for there to be a significant time lag from joining or leaving before the connection was established or broken.
While IP and multicast extensions to IP are based on the routing of packets, another form of wide area networking technology called Asynchronous Transfer Mode (ATM) is based on switching fixed sized cells through switches. Unlike IP which supports both datagram and connection oriented services, ATM is fundamentally connection oriented. An ATM network consists of ATM switches interconnected by point-to-point links. The host systems are connected to the leaves of the network. Before any communication can occur between the hosts through the network, a virtual circuit must be setup across the network. Two forms of communication can be supported by an ATM network. Bi-directional point-to-point between two hosts and point-to-multipoint in one direction from one host to multiple hosts. ATM, however, does not directly support any form of multicasting. There are a number of proposals for layering multicasting on top of ATM. One approach is called a multicast server, shown in Figure 8. Host systems 1 12, 113, 114, 115 setup point-to-point connections 106, 107, 108 and 109 to a multicast server 105. ATM cells are sent by the hosts to the multicast server via these links. The multicast server sets up a point -to-multipoint connection 11 1 to the hosts which collectively constitute a message group. Cells sent to the server which are addressed to the group are forwarded to the point- to-multipoint link 11 1. The ATM network 1 10 is responsible for the transport and switching for maintaining all of the connections between the hosts and the server. The cells carried by the point-to-multipoint connection are duplicated when necessary by the ATM switches at the branching points in the network tree between and forwarded down the branching network links. Therefore, the network is responsible for the replication of the cells and their payloads, not the server. This method has the same problems as distributed multicasting when used for an interactive application. Each host still receives individual cells from each of the other hosts, so there is no aggregation of the payloads of the cells targeted at a single host. There is no support for addressing cells to hosts based on logical operations on the sets of members of host groups.
Related Prior Art
There are a number of existing patents and European patent applications that are related to the area of the invention. These can be organized into two separate categories: multicast routing/distribution and source to destination multicast streams. Multicast routing and distribution
These patents are US 4,740,954 by Cotton et al, US 4,864,559 by Perlman, US 5,361,256 by Doeringer et al, US 5,079,767 by Perlman and US 5,309,433 by Cidon et al. Collectively these patents cover various algorithms for the routing and distribution of the datagrams in distributed multicast networks. None deal with the problems described previously for this class of multicast routing and message distribution such as poor behaviors when the message groups change rapidly. In all of these patents, messages are transmitted from a host via a distributed network of routers to a plurality of destination hosts which are members of a group. Since these patents deal only with variants of distributed multicasting they provide no means to reduce the received message rate, no method to aggregate messages and provide no method in the messages to perform logical operation on message groups. Source to destination multicast streams
These are PCTs and a European patent application. They are EP 0 637 149 A2 by Perlman et al, PCT/US94/11282 by Danneels et al and PCT/US94/11278 by Sivakumar et al. These three patent applications deal with the transmission of data streams from a source to a group of destinations. In none of these patent applications, is a method described for transmitting data between multiple members of a group. In all of these applications, the data transmission is from a source to a plurality of designations. Since these patent applications deal only with point-to-multipoint messaging, they can provide no means to reduce the received message rate, no method to aggregate messages and provide no method in the messages to perform logical operation on message groups.
SUMMARY OF THE INVENTION
The present invention relates to facilitating efficient communications between multiple host computers over a conventional wide area communications network to implement an interactive application such as a computer game between multiple players. In such an application, the hosts will be dynamically sending to each other information that the other hosts need in order to keep the interactive application operating consistently on each of the hosts. The invention is comprised of a group messaging server connected to the network that maintains a set of message groups used by the hosts to communicate information between themselves. The invention further comprises a server-group messaging protocol used by the hosts and the server. The server-group messaging protocol is layered on top of the Transport Level Protocol (TLP) of the network and is called the Upper Level Protocol (or ULP). In the OSI reference model the ULP can be thought of as a session layer protocol built on top of a transport or applications layer protocol. The ULP protocol uses a server-group address space that is separate from the address space of the TLP. Hosts send messages to addresses in the ULP address space to a group messaging server using the underlying unicast transport protocol of the network. The ULP address space is segmented into unicast addresses, implicit group messaging addresses and logical group messaging addresses. The implicit and logical group messaging addresses are collectively called group messaging addresses. Host systems must first establish connections to a group messaging server before sending messages to any ULP addresses The process of establishing this connection is done by sending TLP messages to the server The server establishes the connection by assigning a unicast ULP address to the host and returning this address in an acknowledgment message to the host Once connected, hosts can inquire about existing message groups, join existing message groups, create new message groups, leave message groups they have joined and send messages to ULP addresses known by the server Each message group is assigned either an implicit or logical ULP address depending on its type Figure 5 shows an example of a wide area network with a group messaging server ("GMS") Hosts 58 has TLP address A and ULP address H, host 59 has TLP address C and ULP address J, host 60 has TLP address B and ULP address I and host 61 has TLP address D and ULP address K The network is a conventional unicast network of network links 69, 70, 71, 72, 73, 74, 75, 76, and 77 and unicast routers 63, 64, 65, 66, 67, and 68 The group messaging server 62 receives messages from the hosts addressed to a message group and send the contents of the messages to the members of the message group Figure 6 shows an example of datagrams sent from the hosts to a message group that they are members of As before, a TLP such as IP (where the message header contain the source and destination TLP addresses) is assumed to be used here
Host 58 sends message 80 which contains the TLP source address A of the host and the destination TLP address S for the GMS 62. The destination ULP address G is an implicit ULP address handled by the GMS and the payload Pl contains both the data to be sent and the source ULP address H of the host It is assumed that prior to sending their ULP messages to the GMS, that each host as already established a connection to the GMS and joined the message group G Host 60 sends message 81 with payload P2 containing data and source ULP address I Hosts 59 sends message 82 with payload P3 containing data and source ULP address J. Host 61 sends message 83 with payload P4 containing data and source ULP address K. The GMS receives all of these messages and sees that each message is addressed to implicit message group G with members H, I, J, and K. The GMS can either process the message with or without aggregating their payloads. Figure 6 shows the case where there is no aggregation and Figure 7 shows the case with aggregation
Without aggregation, the GMS generates the outbound messages 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, and 95 which it sends to the hosts The datagrams have TLP headers with the source and destination TLP addresses of the GMS and the hosts respectively The next field in the datagrams is the destination ULP of the datagram Datagrams 84, 85, and sent to host 58 with TLP address A and ULP address H Datagrams 87, 88, and 89 are sent to host 60 with TLP address B and ULP address I Datagrams 90, 91 and 92 are sent to host 59 with TLP address C and ULP address J. Datagrams 93, 94 and 95 are sent to host 61 with TLP address D and ULP address K respectively As can be seen from the payloads that each host has received, each host has received the payloads from the other three hosts Note that each host has not received a copy of its own original message This is because the GMS has performed echo suppression This is selectable attribute of the GMS since in some applications it is useful for the hosts to receive and echo of each message that they send to a group that they are also members of. In the example of Figure 6, it has been shown how the present invention can achieve the same message delivery as distributed multicasting without its disadvantages. Without aggregation, the present invention enables a host to send a single message to multiple other hosts that are members of a message group. It reduces the message traffic that a host must process in an interactive application by reducing the number of messages that each host must send to the others. Without aggregation, however, there is no reduction in the number of messages received by the hosts Without aggregation we can achieve the same message rate as distributed multicasting without the need for a network with multicast routers, we can use a conventional unicast network such as the Internet. The present invention also avoids the problems that dynamic group membership causes for distributed multicasting. Group membership can be changed very rapidly. Groups can be created, joined and left by single unicast messages from hosts to the GMS. These messages will be point-to-point messages and will not have to propagate in throughout the network nor have to cause routing table changes in the routers. This ability to rapidly and accurately change group membership is critical to the implementation of networked interactive applications. Consider a computer game for multiple players that supports hundreds of players that are spread throughout a three dimensional space created by the game. At any time only a few players will be able to see and effect one another in the game since other players will be in other areas that are out of sight. Using conventional phone lines to carry the data from each players computer to the network, it will not be possible to send all actions of each player to all of the other players, but because only a few players will be in close proximity at any one time, it will not be necessary to do so. It is only necessary to send data between the players that are in close proximity to one another. These "groups" of players naturally map onto the message groups of the invention. As players move about the three dimensional space of the game, game will cause them to join and leave message groups as necessary. If this does not happen rapidly it will limit the interactivity of the game or cause inconsistent results for the different players in the game.
The invention also allows aggregating message payloads of multiple messages destined to a single host into a single larger message. This can be done because of the GMS where all of the messages are received prior to being sent to the hosts. Figure 7 shows an example of how this works. The hosts send their messages to the GMS in exactly the same fashion as in Figure 6 using the same addresses previously defined in Figure 5. Host 58 sends message 96, host 60 sends message 97, host 59 sends message 98 and host 61 sends message 99. The GMS receives all of these messages and creates four outbound messages 100, 101, 102 and 103. The process by which these messages will be explained in detail in the detailed description of the invention. Each message is destined to a single host and contains an aggregated payload with multiple payload items. Message 100 has a destination ULP address H for host 58 and aggregated payload P2, P3 and P4 from the messages from hosts 59, 60 and 61. Message 101 is targeted at host 60, message 102 is targeted at host 59 and message 103 is targeted at host 61. As can be seen, each host sends one message and receives one message. The received message is longer and contains multiple payloads, but this is a significant improvement over receiving multiple messages with the wasted overhead of multiple message headers and message processing time. Overall the invention has dramatically reduced the amount of data that must be sent and received by each host. Since the bit rate over conventional phone lines using a modem is low, a reduction in the amount of data that must be sent and received directly translates into improved time and latency for message communications between the hosts.
Hosts create, join and leave message groups using control messages in the ULP protocol to the GMS. Hosts may also read and write application specific state information that is stored in the GMS. When hosts send messages to other hosts, the message must be at least addressed to an implicit group address. The ULP implicit address will always be the primary address in a message from one host to another. The message may optionally specify auxiliary destination addresses. In many cases the implicit ULP address will be the only destination
ULP address in the message. The GMS will handle delivery of the ULP messages addressed to the implicit message group to all of the hosts that are members of the group. A ULP send message may optionally specify an address list of auxiliary addresses in addition to the primary destination of the implicit ULP address. This auxiliary address list can contain only unicast and logical ULP addresses The address list can also specify set operators to be performed between the sets of host ULP addresses defined by the unicast addresses and logical groups Once the address list has been processed to yield a set of hosts, this set is intersected with the set of hosts that are members of the implicit message group specified by the primary implicit ULP address in the message This ability to perform logical set operators on message groups is very usef l in interactive applications. It allows a single ULP message to selectively deliver a message to hosts that fit a set of computed criteria without the sending host having to know the anything about the members of the groups in the address list Recall the example of a networked game with hundreds of players in a three dimensional environment created by the game Consider an implicit message group consisting of all of the game players in a certain area of the game where all of the players can interact with one another Consider that the players are organized into multiple teams Logical message groups could be created for each team within the game. To send a message to all the players within the area that were on one team, a ULP message would be sent to the ULP implicit message group for all the players in the area with an auxiliary address of the logical message group for all the players on the selected team The GMS would perform the proper set intersection prior to sending the resulting messages to the targeted hosts. The result of this will be that the message will only be delivered to the players on the selected team in the selected area of the game.
In summary, the present invention deals with the issues of deploying an interactive application for multiple participants on wide area networks by providing a method for reducing the overall message rate and reducing latency This invention uses a server group messaging approach, as oppose to the above described "distributed multicast messaging" approach. The present invention overcomes the undesirable side effects of the distributed multicast messaging approach. Further, it reduces the message rate between the hosts. As pointed out in an example discussed above, with prior art distributed multicast messaging, each host will need to send only one message to the group containing all of the hosts seven times per-second, but will still receive 9 messages, seven times per-second. The present invention of server group messaging has each host sending one message, seven times per-second and receiving one message, seven times per-second.
The present invention is different from the multicast routing and distribution method disclosed in U.S. Patent Nos. 4,740,954, 4,864,559, 5,361,256,
5,079,767 and 5,309,433. Since these patents deal only with variants of distributed multicasting they provide no means to reduce the received message rate, no method to aggregate messages and provide no method in the messages to perform logical operation on message groups. This differs from the present invention where messages from multiple hosts addressed to a message group are received by a group server which processes the contents of the messages and transmits the results to the destination hosts.
The present invention is also different from the source to destination multicast streams approach disclosed in EP 0 637 149 A2, PCT US94/11282 and PCT/US94/1 1278. In all of these references, the data transmission is from a source to a plurality of designations, whereas the present invention describes data transmission from a sending host to a server host system and then from the server host to the destination hosts.
These and other features and advantages of the present invention can be understood from the following detailed description of the invention together with the accompanying drawings. DESCRIPTION OF DRAWINGS
Figure 1 shows a conventional unicast network consisting of hosts, network links and routers.
Figure 2 shows the unicast datagrams on a conventional unicast network that would be needed to implement an interactive application between four hosts.
Figure 3 shows a prior art multicast network consisting of hosts, network links and multicast routers.
Figure 4 shows a multicast datagrams on a prior art multicast network that would be needed to implement an interactive application between four hosts. Figure 5 shows a unicast network equipped with a group messaging server in accordance with the present invention.
Figure 6 shows the ULP datagrams without payload aggregation on a network according to the present invention that would be needed to implement an interactive application between four hosts. Figure 7 shows the ULP datagrams with payload aggregation on a network according to the present invention that would be needed to implement an interactive application between four hosts.
Figure 8 shows a prior art ATM network with a multicast server.
Figure 9 shows the detailed datagram format and address format for ULP messages in accordance with the present invention.
Figure 10 shows the internal functions of the GMS according to the present invention.
Figure 11 shows the host software interface and functions needed to support the ULP according to the present invention.
DETAILED DESCRIPTION OF THE INVENTION
The present invention provides a method for multiple host computers to efficiently communicate information to one another over a wide area network for the purposes of implementing an interactive application between multiple users. The method consists of three components: a host protocol interface, a protocol and a server. The protocol is between the host protocol interface and the server and is implemented on top of the network transport protocol of a wide area network. The protocol is called the Upper Level Protocol (ULP) since it is layered above the existing network Transport Level Protocol (TLP). In the OSI reference model the protocol can be described as a Session Layer protocol on top of the Transport Layer of the network. Figure 11 shows the host protocol interface, 151, relative to the interactive application, 150, and the host interface for the Transport Level Protocol , 153. The network interface, 155, provides the physical connection for the host to the network. The network communications stack, 154, is the communications protocol stack that provides network transport services for the host and the host interface for the Transport Level Protocol, 153, is and interface between host application software and the network transport services of the network communications stack. The interactive application can send and receive conventional network messages using the host interface to the TLP. The interactive application also can send and receive ULP messages through the host interface for the ULP. Internal to the host interface for the ULP is a table, 152, of all ULP addresses which the host can send messages to. Each entry in the table contains a pair of addresses, a ULP address and its corresponding TLP address. When the host sends a message to a ULP address, that message is encapsulated in a TLP message sent to the TLP address corresponding to that ULP address. This allows the ULP messages to be handled transparently by the transport mechanisms of the existing network. A core function of the ULP is group messaging where hosts send messages to message groups populated by multiple hosts. This allows a host to send a message to multiple hosts with one ULP message. Since the ULP is layered on top of the TLP, the group messaging functions of the ULP operate on a conventional unicast network where TLP messages can only be sent from one host to only one other host
The group based messaging is implemented through the use of a server called a group messaging server All ULP messages from the hosts are sent from the hosts to a group messaging server using the TLP protocol The server processes the ULP portion of the messages and takes the necessary required by the ULP message Control ULP messages are processed locally by the server and may be acknowledged to the sending host ULP messages addressed to other hosts are processed by the group messaging server and then re-transmitted to the proper ULP destination hosts, again using the TLP protocol to encapsulate and transport these messages
In Figure 5, hosts 58, 59, 60 and 61 send messages to one another using the ULP over a conventional unicast network using a group messaging server 62 The network consists of conventional routers 63, 64, 65, 66, 67 and 68 connected with conventional network links 69, 70, 71, 72, 73, 74, 75, 76 and 77
Host 58 can send a message to hosts 59, 60 and 61 by sending a single ULP message to the group messaging server 62 where the ULP message specifies a destination address that is a ULP message group The ULP message is encapsulated in a TLP message addressed to the group messaging server This causes the message to be properly routed by router 63 to network link 71 to router 67 to the server 62 The group messaging server receives the ULP message and determines that the message is addressed to a message group containing hosts 59, 60 and 61 as members The server sends the payload of the received message to each of the hosts in three new ULP messages individually sent to the three hosts Since each message is encapsulated in a TLP message, the messages are properly carried over the conventional unicast network The first ULP message is sent by the group messaging server to host 61. This message is carried by network links 71, 70, 72 and 75 and routers 67, 63, 64 and 65. The second ULP message is sent by the group messaging server to host 60. This message is carried by network links 71, 70, 73 and 76 and routers 67, 63, 64 and 66. The third ULP message is sent by the group messaging server to host 61. This message is carried by network links 74 and 77 and routers 67 and 68. The invention can be implemented both in a datagram form and in a connection oriented form. To best understand the details of the invention, it is best to first consider a datagram implementation.
Datagram Transport Implementation
The ULP can be implemented as a datagram protocol by encapsulating addresses, message type information and the message payload within a datagram of the underlying network transport protocol. The general form of the ULP datagram message format is shown in Figure 9 as elements 123, 124, 125, 126, 127, 128 and 129. The transport header 123 is the datagram header of the TLP that is encapsulating the ULP datagram. The ULP message type field 124 indicates whether it is a send or receive message, if it is a control message or a state message. The following table shows the different message types. The ULP message type field must be present in a ULP datagram.
Message Types
1 Send
2 Receive
3 Send Control
4 Receive Control
5 Send State
6 Receive State
Send messages are always sent from a host to a group messaging server. Messages from a group server to the hosts are always receive messages. Send Control messages are messages from hosts to a group messaging server requesting a control function be performed. Receive Control messages are acknowledgments from a group messaging server to the hosts in response to a prior Send Control messages. The Send and Receive State messages are special cases of the Send and Receive Control messages that allow hosts to read and write application specific state storage in the group messaging server. The specific control functions supported by the ULP will be explained later.
The destination ULP address 125 is required in ULP datagrams and specifies the primary destination of the ULP message. The address count field 126 is required in ULP send message types and is not present in ULP receive message types. When the address count field in a ULP send message is non-zero, it specifies the number of auxiliary destination addresses for the send message that follow the address count field. These auxiliary destination addresses are shown as items 127 and 128, but it is understood that there are as many auxiliary ULP destination addresses as specified by the address count field. Finally there is the payload 129.
The payload format for ULP datagrams is defined by items 1 16, 117, 118, 119, 120, 121 and 122. Item 116 is the message count and defines how many payload elements will be contained in the payload. A single payload element consists of a triplet of source ULP address, data length and data. Items 1 17, 1 18 and 1 19 comprise the first payload element of the payload. Item 1 17 is the ULP address of the source of the payload element, item 118 is the data length for the data in the payload element and item 119 is the actual data. Items 120, 121 and 122 comprise the last payload element in the payload. ULP send messages only support payloads with a single payload element, so the message count is required to be equal to one. ULP receive messages may have payloads with one or more payload elements. ULP Address Space
The address space of the ULP is divided into three segments: unicast host addresses, implicit group addresses and logical group addresses. All source and destination addresses in ULP must be in this address space. The ULP address space is unique to a single group messaging server. Therefore each group messaging server has a unique ULP address space. Multiple group messaging servers may be connected to the network and hosts may communicate with multiple group messaging servers without confusion since each ULP datagram contains the header of the TLP. Different group messaging servers will have unique TLP addresses which can be used by the hosts to uniquely identify multiple ULP address spaces. The format for ULP addresses is shown in Figure 9 comprised of items 130, 131 and 132. The address format field 130 is a variable length field used to allow multiple address lengths to be supported. The address type field 131 indicates the type of ULP address: unicast host, implicit group or logical group. The encoding is as follows: Address Type Encoding
0 0 Unicast Host Address
0 1 Unicast Host Address 1 0 Implicit Group Address
1 1 Logical Group Address
The address format encoding determines the length of the address field and therefore the total length of the ULP address. This encoding is shown below. Note that when the address type specifies a unicast host address, the low bit of the address type field is concatenated to the address field to become the most significant bit of the address. This doubles the size of the address space for unicast host addresses which is useful since there will generally be more hosts than group messaging servers. Address Format Encoding
0 29 Bit Address Field
1 0 4 Bit Address Field
1 1 0 1 1 Bit Address Field
ULP unicast host addresses are assigned to each host when it first connects to a group messaging server. When a host sends a message to other ULP address, the unicast ULP address of the host will appear as the source ULP address in the received payload element. Unicast ULP host addresses can also be used as destination addresses only as auxiliary addresses in a ULP send message. They are not allowed to be used to as the primary ULP destination address. This means that hosts cannot send ULP directly to one another, but always must send the messages to one another through a group messaging server.
Implicit group addresses are created by a group messaging server in response to a control message to the server requesting the creation of an implicit message group. The host requesting the creation of the implicit message group becomes a member of the message group when it is created Other hosts can send inquiry control messages to the group messaging server to learn of its existence and then send a implicit group join message in order to join the group. The group messaging server maintains a list of ULP addresses of hosts that are members of the implicit message group. Implicit ULP group addresses are the only ULP addresses allowed to be the primary destination of a ULP send message. Implicit ULP addresses will never appear as ULP source addresses in a payload element
Logical ULP addresses are used both to address logical message groups and for specifying set operations between the group members of the auxiliary ULP addresses in a ULP send message. Logical message groups are created and joined similarly to implicit message groups, however, logical ULP addresses may only be used as auxiliary ULP addresses in a ULP send message Logical ULP addresses will also never appear as source ULP addresses in a payload element. The support of set operations between message groups as part of a ULP send message will be explained in a later section on ULP send messages.
Group Messaging Server Internal Functions The internal components of the group messaging server are shown in Figure
10.
In the preferred embodiment, the group messaging server is a general purpose computer system with a network interface to connect it to a wide area network. Item 135 is the network interface for the group messaging server and includes not only the hardware connection to the network but the communications protocol stack used to implement the TLP on the server.
Item 136 is an overall control function for the group messaging server. This control function is responsible for all ULP messages that are sent or received by the GMS. Internal to this control function are several important storage and processing functions. Item 137 is an address map for all hosts currently connected to the GMS. This address map is a list of the ULP host address of each host connected to GMS and its corresponding TLP address. This enables the control function to construct the necessary TLP headers for sending ULP messages to the hosts connected to the GMS. Item 138 is a list of all of the currently active implicit ULP addresses currently recognized by the GMS. Item
139 is an application specific state storage and processing function. Many interactive applications deployed over a network will be able to be implemented solely with host based processing. In these cases all data that needs to be sent between the hosts can be transported using the ULP. However, some applications will need maintain a centrally stored and maintained repository of application state information. This is useful when hosts may join or leave the application dynamically. When hosts join such an application, they will need a place from which they can obtain a snapshot of the current state of the application in order to be consistent with the other hosts that already where part of the application. To read and write this state storage area, the ULP supports send and receive state message types. Within these messages, there is the ability to access a state address space so that different portions of the state can be individually accessed. Application specific processing of state written into this state storage area can also be implemented.
Items 140 and 141 are two of multiple ULP server processes running on the GMS. These are software processes that are at the heart of the ULP. Each implicit ULP addresses recognized by the GMS has a one-to-one correspondence to a ULP server process and to a message group maintained by the process. Since all ULP send messages must have an implicit ULP address as the primary destination address of the message, every ULP send message is sent to and processed by a ULP server process. These processes are created by the GMS control function in response to ULP control messages to create new implicit ULP addresses. They are destroyed when the last host which is a member of its message group has left the message group. Internal to a ULP server process is a list, 142, of the ULP host addresses of the members of the message group, a set of message queues 143 for each host which is a member of the message group and a message aggregation function 149 which is used to aggregate multiple messages to a single host into a single message.
Item 145 maintains a list of all of the logical ULP addresses and message groups in the GMS. Items 144 and 146 represent two of multiple logical ULP addresses. For each logical ULP address, there is a corresponding list, 147 and 148 of the host ULP addresses of the members of the logical message group. The logical message groups are not tied to specific ULP server processes, but are global with a GMS to all of the ULP server processes. Control Functions
The control functions consist of connect, disconnect, create group, close group, join group, leave group, query groups, query group members, query group attributes. These control functions are implemented by a ULP send and receive control messages. The control functions are initiated by a host sending a
ULP send control message to a GMS. These messages only allow a primary ULP destination address in the message and do no allow auxiliary addresses. The primary ULP address is inteφreted as a control address space with a unique fixed address assigned to each of the control functions enumerated above. The contents of data in the payload supplies any arguments needed by the control function. Returned values from the control function are returned in a ULP receive control message that is addressed to the host that sent the original control message for which data is being returned. The detailed operation of these control functions is described below. Cjarineci
This control function allows a host to connect to a GMS. The destination
ULP address in the message is a fixed address that indicates the connect function. The source ULP address and any data in the payload are ignored. Upon receiving this message, the GMS control function, 136, creates a new host address and enters the host address in the host address map 136 along with the source TLP address from the TLP header of the message. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful host connection. The destination ULP address in the message is the ULP address assigned to the host. The host saves this and uses it for any future messages to the GMS. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed host connection. Disconnect
This function allows a host to disconnect from a GMS. The destination ULP address in the message is a fixed address that indicates the disconnect function.
The source ULP address is used to remove the host from membership in any implicit or logical groups prior to disconnecting. Any data in the payload is ignored. The GMS control function also removes the entry for the host from the host address map. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful host disconnection. The destination ULP address in the message is the ULP address assigned to the host. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed host disconnection. Create implicit group This function allows a host to create a new implicit message group and associated implicit ULP address and server process. The payload in the message may contain a single payload item whose data field holds attributes of the group. These attributes can be used to define any optional functions of the group. The destination ULP address in the message is a fixed address that indicates the create implicit group function. The GMS control function allocates a new implicit ULP address, adds it to the implicit ULP address list 138 and creates a new ULP server process 140. The host that sends this message is added to the membership list of the implicit group. This is done by adding the source ULP address in the message to the group membership list 142 in the ULP server process. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful implicit group creation. The source ULP address in the payload is the ULP address assigned to the new implicit group. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed implicit group creation.
Create logical group This function allows a host to create a new logical message group and associated logical ULP address. The payload in the message may contain a single payload item whose data field holds attributes of the group. These attributes can be used to define any optional functions of the group The destination ULP address in the message is a fixed address that indicates the create logical group function. The GMS control function allocates a new logical
ULP address and adds it to the logical ULP address list 145. The host that sends this message is added to the membership list of the logical group. This is done by adding the source ULP address in the message to the group membership list 147 for the new logical message group 144. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful logical group creation. The source ULP address in the payload is the ULP address assigned to the new logical group. If there is an error, the control fiinction returns a message to the host with a function code in the data portion of the payload indicating failed implicit group creation.
Join group
This function allows a host to join an existing logical or implicit message group. The destination ULP address in the message is a fixed address that indicates the join group function. The data portion of the payload contains the ULP address of the group that is to be joined. The GMS control function looks at this address and determines if it is an implicit or logical ULP address. If it is an implicit ULP address, the GMS control function finds the ULP server process selected by the address in the message payload and adds the source ULP host address from the message to the group membership list 142. If it is a logical ULP address, the GMS control function finds the logical ULP address 144 selected by the address in the message payload and adds the source ULP host address from the message to the group membership list 147. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful group join. The source ULP address in the payload is the ULP address of the group that was joined. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed implicit group creation.
Leave group
This function allows a host to leave an existing logical or implicit message group that it is a member of. The destination ULP address in the message is a fixed address that indicates the leave group function. The data portion of the payload contains the ULP address of the group that is to be left. The GMS control function looks at this address and determines if it is an implicit or logical ULP address. If it is an implicit ULP address, the GMS control function finds the ULP server process selected by the address in the message payload and removes from the group membership list 142 the source ULP host address from the message. If the host is the last member of the group, the ULP server process is terminated and the implicit ULP address is de-allocated. If it is a logical ULP address, the GMS control function finds the logical ULP address 144 selected by the address in the message payload and removes from the group membership list 147 the source ULP host address from the. If the host is the last member of the group, the ULP address is de-allocated. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a function code in the data portion of the payload that indicates successful group leave. If there is an error, the control function returns a message to the host with a function code in the data portion of the payload indicating failed implicit group creation.
Query groups
This function allows a host to get a list of all implicit and logical message groups currently active on a GMS. The destination ULP address in the message is a fixed address that indicates the query groups function. Any data portion of the payload is ignored. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a payload with multiple payload elements. The first payload element contains a function code indicating successful query groups. The source ULP address in the first payload element is ignored. Each of the subsequent payload elements contain a ULP group address in the source address field of the payload element that is one of the active group addresses on the GMS. There is no data field in these subsequent payload elements. If there is an error, the control function returns a message to the host with a function code in the data portion of a payload with a single payload element indicating failed query groups.
Query group members
This function allows a host to get a list of all hosts that are members of a message group. The destination ULP address in the message is a fixed address that indicates the query group members function. The data portion of the payload carries the address of the message group for the query. Upon successful completion, the GMS control function responds with a receive control ULP message addressed to the host along with a payload with multiple payload elements. The first payload element contains a function code indicating successful query group members. The source ULP address in the first payload element is ignored. Each of the subsequent payload elements contain a ULP host address in the source address field of the payload element that is one of the active group addresses on the GMS. There is no data field in these subsequent payload elements. If there is an error, the control function returns a message to the host with a function code in the data portion of a payload with a single payload element indicating failed query group members.
Query group attributes This function allows a host to get a list of the attributes of a message group.
The destination ULP address in the message is a fixed address that indicates the query group attributes function. The data portion of the payload carries the address of the message group for the query. Upon successful completion, the
GMS control function responds with a receive control ULP message addressed to the host along with a payload with a two payload elements. The first payload element contains a function code indicating successful query group members. The second payload element contains the attributes of the message group. If there is an error, the control function returns a message to the host with a function code in the data portion of a payload with a single payload element indicating failed query group attributes.
Send Message Operation
In order to fully understand the operations of the send message function, a number of individual cases are worth considering.
Single implicit destination The most simple case is a send message to a single implicit ULP address. In all send message datagrams, the destination ULP address 125 must be an implicit
ULP address. In this case of a single implicit destination, this is the only destination address in the datagram. The auxiliary address count 126 is zero and there are no auxiliary destination addresses 127 or 128. The payload consists of a message count 116 of one, the ULP of the host sending the message in the source ULP address 117 and the data length 118 and data 119.
Send message datagrams may only have a single payload item so their message count field 116 must always be one. The host sends the send message onto the network with a TLP header addressing the datagram to the GMS that is the selected target of the message.
The GMS receives the message and the GMS control function 136 determines that it is a send message datagram and looks up the implicit destination address in its implicit ULP address list 138. If the address does not exist, an error message is returned to the sending host with a ULP receive message datagram. If the address is valid, the GMS control function removes the TLP header from the datagram and sends the ULP portion to the ULP server process corresponding to the destination implicit ULP address. Assume for discussion that this is the ULP server process 140. The ULP server process 140 will extract the single payload item from the message 117, 1 18 and 119 and place the payload item in each of the message queues 143. There will be one message queue for each member of the message group served by the ULP server process 140. The members of the group will have their host ULP addresses listed in the host address list 142. Each message queue in a ULP server process will fill with payload items that are targeted at particular destination hosts. The mechanisms by which payload items are removed from the queues and sent to the hosts will be described later.
Auxiliary unicast destination In this case in addition to an implicit destination 125, there is also a single auxiliary address 127 in the datagram. The auxiliary address count 126 is one and the auxiliary destination addresses 127 is a unicast host ULP address. The payload consists of a message count 116 of one, the ULP of the host sending the message in the source ULP address 117 and the data length 118 and data 119. The host sends the send message onto the network with a TLP header addressing the datagram to the GMS that is the selected target of the message. The GMS receives the message and the GMS control function 136 determines that it is a send message datagram and looks up the implicit destination address in its implicit ULP address list 138 and the unicast host ULP auxiliary address in the host address map 137. If either of addresses does not exist, an error message is returned to the sending host with a ULP receive message datagram. If the addresses are valid, the GMS control function removes the TLP header from the datagram and sends the ULP portion to the ULP server process corresponding to the destination implicit ULP address. Assume for discussion that this is the ULP server process 140. The ULP server process extracts the auxiliary ULP address from the message and determines from the address that it is a unicast host ULP address. The server process then checks to see if this address is a member of the message group defined by the host address list 142. If it is not, no further action is taken and the payload item in the message is not placed in any of the message queues 143. If the host address is in the message group, the payload item in the message is placed in the single message queue corresponding to that host. The net effect is that the ULP server process has performed a set intersection operation on the members of the message group selected by the implicit ULP destination address and defined by the group membership list 142 with the members of the set of hosts defined by the auxiliary address. The payload item is them sent only to the hosts that are members of this set intersection. Auxiliary logical destination
In this case in addition to an implicit destination 125, there is also a single auxiliary address 127 in the datagram. The auxiliary address count 126 is one and the auxiliary destination addresses 127 is a logical ULP address. The payload consists of a message count 116 of one, the ULP of the host sending the message in the source ULP address 117 and the data length 118 and data 119.
The host sends the send message onto the network with a TLP header addressing the datagram to the GMS that is the selected target of the message. The GMS receives the message and the GMS control function 136 determines that it is a send message datagram and looks up the implicit destination address in its implicit ULP address list 138 and the logical ULP auxiliary address in list of logical ULP addresses 145. If either of addresses does not exist, an error message is returned to the sending host with a ULP receive message datagram. If the addresses are valid, the GMS control function removes the TLP header from the datagram and sends the ULP portion to the ULP server process corresponding to the destination implicit ULP address. Assume for discussion that this is the ULP server process 140. The ULP server process extracts the auxiliary ULP address from the message and determines from the address that it is a logical ULP address. Assume for this example that this logical ULP address is the logical address 144. The server process fetches the group membership list 147 corresponding to the logical address and performs a set intersection operation with the group membership list 142 of the server process. If there are no members of this set intersection, no further action is taken and the payload item in the message is not placed in any of the message queues 143. If there are members of the set intersection operation, the payload item in the message is placed in the queues corresponding to the hosts that are members of the set intersection.
Multiple auxiliary addresses with logical operations In its most sophisticated form, a send message can perform set operations between the implicit message group of the ULP server process and multiple logical and unicast ULP addresses. This is done by placing multiple auxiliary destination ULP addresses in the message with logical operators imbedded in the address list. The address count 126 holds a count of the total auxiliary addresses in the address list 127 and 128. The auxiliary addresses are a mix of logical ULP addresses and unicast host ULP addresses. Two logical ULP addresses in the ULP address space are assigned the role of specifying set operations to be performed between the logical message groups and unicast host addresses in the message list. They are specially assigned addresses for the functions set intersection, set union. A third logical address is used to indicate set complement. The payload consists of a message count 116 of one, the ULP of the host sending the message in the source ULP address 1 17 and the data length 118 and data 119.
The host sends the send message onto the network with a TLP header addressing the datagram to the GMS that is the selected target of the message. The GMS receives the message and the GMS control function 136 determines that it is a send message datagram and looks up the implicit ULP message in the implicit ULP address list 138 and all of the addresses in the address list either in the host ULP address map 137 or in the logical ULP address list 145 as appropriate. If any of addresses does not exist, an error message is returned to the sending host with a ULP receive message datagram. If the addresses are valid, the GMS control function removes the TLP header from the datagram and sends the ULP portion to the ULP server process corresponding to the destination implicit ULP address. Assume for discussion that this is the ULP server process 140. The ULP server process extracts the auxiliary ULP address list from the message and scans it from beginning to end. The scanning and processing of the set operators is done in post-fix fashion. This means that arguments are read followed by an operator that is then applied to the arguments. The result of the operator becomes the first argument of the next operation. Therefore at the start of scanning two addresses are read from the address list. The next address will be an operator that is applied to the arguments and the result of this operator is the first argument to be used by the next operator. From then on a single address is read from the address list followed by a logical ULP address which is operator on the two arguments consisting of the new argument and the results of the last operator. The logical address used to indicate set complement is not a set operator, by an argument qualifier since it can precede any address in the address list. The meaning of the set complement argument qualifier is relative to the group membership of implicit group address in the send message. If the set complement qualifier precedes a unicast host address which is not a member of the message group selected by the implicit ULP address in the send message, the effective argument is the set of all hosts that are members of the implicit message group. If the set complement qualifier precedes a unicast host address which is a member of the message group selected by the implicit ULP address in the send message, the effective argument is the set of all hosts that are members of the implicit message group except for the original unicast host address qualified by the complement function. If the set complement qualifier precedes a logical ULP address the effective argument is the set of all hosts that are members of the implicit message group specified by the send message except hosts that are members of the logical message group preceded by the set complement modifier. Once the entire address list has been processed to a single result set of hosts, a set intersection operation is performed on this set and the set of members of the implicit message group 142 defined by the implicit address in the send message. If there are no members of this set intersection, no further action is taken and the payload item in the message is not placed in any of the message queues 143. If there are members of the set intersection operation, the payload item in the message is placed in the queues corresponding to the hosts that are members of the set intersection.
Message Delivery and Aggregation
Once messages are entered into the message queues in the ULP server processes, there are a variety of ways that they can ultimately be delivered to the targeted hosts. In the invention, the delivery method is set on a per-ULP server process basis by attributes that are provided at the time that an implicit ULP message group and server process are created. It is important during the description of these methods to keep in mind that the invention is intended to provide an efficient means for a group of hosts to send messages to each other at a rapid rate during the implementation of a networked interactive application. Also assumed in the following description is that the GMS performs echo suppression when a host sends a message to a group that it belongs to. This means that the host will not receive a copy of its own message to the group either as a single un-aggregated message or as a payload item in an aggregated message This is controlled by a ULP server process attribute that can be changed to stop echo suppression, but echo suppression is the default. Immediate Delivery
The most simple delivery method is to immediately deliver the payload items to their targeted hosts as soon as they are placed in the message queues. Each payload item in a message queue will contain a ULP source address, a data length and the data to be sent. To implement immediate delivery, the ULP server process will remove a payload item from a message queue for a particular host 143. The host address for this host will be obtained from the group membership list 142. The payload item and the destination host address will be sent to the GMS control function 136 where it will be used to create a ULP receive message sent to the destination host. The GMS control function 136 will use the destination ULP host address to look up the TLP address of the host from the host address map 137. This will be used to create a TLP header for the message 123. The ULP message type 124 will be ULP receive, the destination ULP address 125 will be the destination host, the address count will be 0 and there will be no auxiliary addresses. The payload in this case will have a message count 116 of 1 and the payload item comprised of fields 117, 118, and 1 19 will be the payload element taken from the message queue.
Immediate delivery is useful when the message rate between a group of hosts is low. Consider four hosts that are members of an implicit message group where each member of the group sends a message to every other member of the group at a fixed rate. With immediate delivery, each host will send three messages to the other members of the group and receive three messages from the other members of the group at the fixed rate. This is acceptable is the size of the group is small and the message rate is low. However, it is obvious that total message rate is the product of the underlying message rate and the total number of members of the group minus one. Clearly this will result in unacceptably high message rates for large groups and highly interactive message rates A group of 20 members that had an underlying message rate of 10 messages per second would yield a total message rate at each host of 190 messages sent and 190 messages received every second. This message rate will be unsupportable over a conventional dial-up connection to a conventional wide area network such as the internet
Aggregation A key concept in the present invention is the aggregation of multiple messages in a message queue into a single ULP receive message to a host that contains multiple payload items in the payload. The ULP server process 140 removes payload items from a message queue 143 for a host and accumulates them in an aggregation buffer 149. The aggregation buffer has buffer areas for each host for which there is a message queue. These individual host areas within the aggregation buffer are called host aggregation buffers. The start and end of this aggregation period can be controlled in a number of ways that will be described in the next sections. At the end of the aggregation period, the each host aggregation buffer may hold multiple payload items. The host aggregation buffer will hold a message count of the payload items followed by the multiple payload items. The contents of a host aggregation buffer along with the ULP host address of the corresponding host are sent to the GMS control function 136 where it will be used to create a ULP receive message sent to the destination host. The GMS control function 136 will use the destination ULP host address to look up the TLP address of the host from the host address map 137. This will be used to create a TLP header for the message 123. The ULP message type
124 will be ULP receive, the destination ULP address 125 will be the destination host, the address count will be 0 and there will be no auxiliary addresses. The payload in this case will have a message count 1 16 set by the message count value from the host aggregation buffer. The payload will contain all of the payload items from the host aggregation buffer.
The effect of aggregation will be to greatly reduce the total message rate received by the hosts. A single message to a host will be able to carry multiple payload items received from the other hosts during the aggregation period. This fits very well the interactive applications of this invention where groups of hosts will be sending messages to all the other hosts in the group at a periodic rate. Aggregation will be very effective in collecting together all of the messages from all of the other hosts into a single message for each member of the group. The reduces processing at each receiving host since a single message will be received rather than many separate messages. Aggregation will also reduce the total data rate to the hosts since aggregation eliminates the need for separate message headers for each payload item. The savings will be significant for small payload items since there will be only one message header comprising fields 123, 124 and
125 for multiple payload items. In cases where a group of hosts are sending messages to the group at a periodic rate, it is often the case in many interactive applications that the data being sent by each host to the group is very similar to the messages sent by the other hosts. This affords the opportunity within an aggregated payload of multiple payload items to apply a data compression method across the multiple data elements of the payload elements. A wide variety of known data compression methods will lend themselves to this application. The first data element in the first payload item can be sent in uncompressed form with each subsequent data element being compressed using some form of difference coding method. A variety of known data compression methods use the concept of a predictor with differences from the predicted value being encoded. The first data element in an aggregated payload can be used as this predictor with the subsequent data elements coded using such a data compression method. These conventional data compression methods do not assume any knowledge of the internal structure or function of portions of a data element to compress. It is also possible to make use of application specific coding techniques that take advantage of such knowledge to potentially achieve much higher coding efficiency.
Server Isochronous
One method by which the aggregation time period can be defined is called Server Isochronous or SI. In this method, A ULP Server Process defines a uniform time base for defining the aggregation time period. This time base is defined by three parameters: the time period, the aggregation offset and the transmit offset. These parameters are set by the attributes provided in the create implicit group control function at the time the implicit group and the ULP server process are created. The time period is a fixed time interval during which the ULP server process will accumulate messages in the message queues, aggregate the messages in the queues and send the aggregated messages to the targeted hosts. The aggregation offset defines the point after the start of the time period after which arriving messages will be stored in the message queues for delivery in the next time period. Therefore, at the aggregation offset after the start of the time period, a snapshot will be taken of all of the messages in each message queue. New messages will continue to arrive and be entered into the queues after the aggregation offset. Only those messages in the queues before the aggregation offset point will be aggregated into outbound messages. The resulting aggregated messages will then be sent to their targeted hosts at the point in time which is the transmit offset after the start of the time period. The result is that messages arrive continuously and are stored in the message queues. Once per time period the are aggregated into single messages to each host which is the target of messages and once per time period these aggregated messages are sent to the hosts.
Another embodiment of the SI method is to allow the ULP server process to dynamically vary the time period based on some criteria such as the received message rates, and/or received data rate. The ULP server could use a function to define the aggregation period based on the number of messages received per second or the total number of payload bytes received per second. One reasonable function would be to shorten the aggregation period as the rate or received messages or data rate of the received payloads increased. This would tend to keep the size of the outbound messages from growing too much as received messages and/or received data rate grew. Other possible functions could be used that varied the aggregation period based on received message rates, received payload data rates or other parameters available to the ULP server process. Host Synchronous
The host synchronous or HS method of defining the aggregation time period allows the definition of a flexible time period that is controlled by the hosts. It is based on the concept of a turn which is a host sending a message to one or more members of the implicit message group which is operating is HS mode. Once every host in the message group has taken a turn, the aggregation period ends. A snapshot of the contents of the message queues is taken, the contents of each of the queues is aggregated and the aggregated messages are sent to the hosts targeted by each message queue. A refinement to this technique qualifies which of the three ULP send message types to the group constitute a host turn: a send only to the implicit address of the group, a send to a unicast host address within the group or a send to a logical ULP address which shares members with the group. The attributes of the group not only will define HS aggregation, but one or more ULP send message types that will be considered a host turn. A further refinement sets the total number of turns that a host can take in a single aggregation time period. The default will be one turn, but multiple turns can be allowed. If a host attempts to take more turns than allowed, the messages are ignored.
This aggregation technique has the additional benefit of causing the hosts which are member of an HS implicit message group to have their processing functions synchronized when they are executing the same interactive application.
Many networked interactive applications are based on a simple overall three step operational model: wait for messages from other hosts, process the messages and the local users inputs to update the local application, send messages to the other hosts. This basic application loop is repeated at a rate fast enough to provide an interactive experience such as 5 to 30 times per second. It is desirable to keep such applications synchronized so that the states of the applications is consistent on the different host machines. When such applications communicate using the HS model of the present invention their operations will become naturally synchronized. The HS ULP server process will wait until all of the members of the message group has completed their turns and sent a message to the group before sending the aggregated messages to the members of the group. This will cause the applications on the hosts to wait until they have received the aggregated messages. They will all then start processing these messages along with the local user inputs. Even if they perform their processing at different speeds and send their next messages to the group at different times, the HS ULP server will wait until all have completed their processing and reported in with a message to the group. This will keep all of the host applications synchronized in that every host will be at the same application loop iteration as all of the others. This will keep the application state consistent on all of the hosts. Only network propagation delays from the GMS to the hosts and different processing speeds of the hosts will cause the start and completion of their processing to begin at different times. It is not a requirement in networked applications to keep all of the hosts precisely synchronized, only that application state is consistent. The HS method provides a natural way to do this in the context of the present invention. Preferred Embodiment
The detailed description of the invention has described a datagram implementation of the invention as the best way to explain the invention. The preferred embodiment of the invention is as follows.
In the preferred embodiment, the wide area network is the Internet and the TLP protocol is TCP/IP. The GMS is a general puφose computer system connected to the Internet and the hosts are personal computers connected to the Internet.
TCP/IP provides an number of advantages that provide for a more efficient applications interface on the hosts 151. TCP/IP supports the concept of source and destination port numbers in its header. The ULP can make use of the port numbers to identify source and destination ULP connections. Most ULP send messages will be from hosts to a implicit ULP group addresses and most ULP receive messages will be from the implicit ULP addresses to the ULP host addresses. All of these and the ULP message type field can represented by source and destination port addresses within the TCP/IP header. This means that for most ULP messages, the ULP message encapsulated within the TCP/IP message need only contain the payload. There is the slight complication of the aggregated ULP receive messages sent from a ULP server process to a hosts. Here the destination port will be the host the source port will be for the implicit ULP group address and the payload will still contain the source host ULP addresses in each the payload items.
TCP/IP also supports header compression for low speed dial-up lines which is also important in this application. See RFC 1144. TCP/IP is a connection oriented protocol which provides reliable end-to-end transport. It handles re¬ transmission on errors and fragmentation and reassembly of data transparently to upper level protocols. Header compression allows much of the TCP/IP header to be omitted with each packet to be replaced by a small connection identifier. This connection ID will uniquely define a connection consisting of a source and destination IP address and source and destination TCP/IP port numbers.
At the interface to the application on the hosts, the preferred embodiment of the ULP is as a session layer protocol In the preferred embodiment the application on a host opens a session with a ULP server process This session is identified with a unique session ID on the host. The host application then sends data to the ULP host interface 151 tagged with this session ID. The session ID defines a host and implicit ULP pair including the TCP/IP TLP address of the GMS server that is running the particular ULP server process for the implicit ULP address By binding the transport address of the GMS of a ULP server process to the session ID, we can transparently to the application support multiple group messaging servers on the network and a single host can have multiple active sessions with different physical group messaging servers This avoids any address space collision problems that could arise from the fact that the ULP address space is unique to each GMS.
Alternate Embodiments One possible extension to the invention is to extend the ULP to support a common synchronized time base on the GMS and the hosts that are connected to it. This would be most interesting in context of the SI message aggregation mode. The SI time base on the GMS could be replicated on all of the hosts and all of the hosts and the GMS could lock these time bases together. There are known methods to synchronize time bases on multiple computer systems. One such method is called NTP.
Another extension to the invention is to define ULP server processes that perform specific application specific processing on the contents of the messages that are received. A variety of different application specific processing functions can be defined and implemented. A particular function would be selected by attributes provided in the create implicit group function. These functions could process the data in the message payloads and replace the data elements in the payloads with processed results. Separately, or in combination with processing the message payloads, the processing could store either raw message payload data in the application specific state storage area or could store processed results.
Clearly, the host system need not be personal computers, but could also be dedicated game consoles or television set top boxes or any other device with a programmable controller capable of implementing the ULP protocol.
The wide area network used to transport the ULP protocol need not be the Internet or based on IP. Other networks with some means for wide area packet or datagram transport are possible including ATM networks or a digital cable television network.
The invention now being fully described, it will be apparent to one of ordinary skill in the art that any changes and modifications can be made thereto without departing from the spirit or scope of the invention as set forth herein. Accordingly, the present invention is to be limited solely by the scope of the appended claims.

Claims

WHAT IS CLAIMED IS:
1. A method for providing group messages to a plurality of host computers connected over a unicast wide area communication network, comprising the steps of: providing a group messaging server coupled to said network, said server communicating with said plurality of host computers using said unicast network and maintaining a list of message groups, each message group containing at least one host computer; sending, by a first host computer belonging to a first message group, a message to said server via said unicast network, said message containing a payload portion and a portion for identifying said first message group; and transmitting, by said server via said unicast network, said payload portion to selected host computers belonging to said first group.
2. The method of claim 1 wherein said selected host computers comprising all host computers belong to said first group except said first host computer.
3. The method of claim 1 wherein said message also contains a portion for identifying a second message group, said selected host computers being selected from a set operation of members in said first and said second message groups.
4. The method of claim 1 further comprising the step of creating, by a second host computer, said first message group by sending a first control message to said server via said unicast network.
5. The method of claim 4 further comprising the step of joining, by said first host computer, said first message group by sending via said unicast network a second control message to said server specifying said first message group.
6. The method of claim 1 wherein said network is Internet and said server communicates with said plurality of host computers using a session layer protocol
7. A method for providing group messages to a plurality of host computers connected over a unicast wide area communication network, comprising the steps of: providing a group messaging server coupled to said network, said server communicating with said plurality of host computers using said unicast network and maintaining a list of message groups, each message group containing at least one host computer; sending, by a plurality of host computers belonging to a first message group, messages to said server via said unicast network, said messages containing a payload portion and a portion for identifying said first message group; aggregating, by said server in a time interval determined in accordance with a predefined criterion, said payload portions of said messages to create an aggregated payload; forming an aggregated message using said aggregated payload; and transmitting, by said server via said unicast network, said aggregated message to a recipient host computer belonging to said first message group.
8. The method of claim 7 wherein said time interval is a fixed period of time.
9. The method of claim 7 wherein said time interval corresponds to a time for said server to receive at least one message from each host computer belonging to said first message group.
10. The method of claim 7 further comprising the step of creating, by one of said plurality of host computers, said first message group by sending a first control message to said server via said unicast network.
11. The method of claim 10 further comprising the step of joining, by some of said plurality of host computers, said first message group by sending control messages via said unicast network to said server specifying said first message group.
12. The method of claim 7 wherein said network is Internet and said server communicates with said plurality of host computers using a session layer protocol
13. A method for providing group messages to a plurality of host computers connected over a unicast wide area communication network, comprising the steps of: providing a group messaging server coupled to said network, said server communicating with said plurality of host computers using said unicast network and maintaining a list of message groups, each message group containing at least one host computer; dynamically joining, by a first host computer, message groups on said list by sending a first control message to said server via said unicast network, said first control message specifying a message group desired to be joined by said first host computer; and dynamically leaving, by said first host computer, message groups on said list by sending a second control message to said server via said unicast network, said second control message specifying a message group said first host computer desires to leave.
14. The method of claim 13 wherein said first host computer belongs to a first message group, said method further comprising the steps of: sending, by said first host computer, a message to said server via said unicast network, said message containing a payload portion and a portion for identifying said first message group; and transmitting, by said server via said unicast network, said payload portion to selected host computers belonging to said first group.
15. The method of claim 14 wherein said selected host computers comprising all host computers belong to said first group except said first host computer.
16. The method of claim 14 wherein said message also contains a portion for identifying a second message group, said selected host computers being selected from a set operation of members in said first and said second message groups.
PCT/US1997/000567 1996-02-01 1997-01-21 Server-group messaging system for interactive applications WO1997028502A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
AU17471/97A AU1747197A (en) 1996-02-01 1997-01-21 Server-group messaging system for interactive applications
EP97904759A EP1012724A1 (en) 1996-02-01 1997-01-21 Server-group messaging system for interactive applications
JP9527663A JP2000504133A (en) 1996-02-01 1997-01-21 Group server communication system for interactive applications

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US08/595,323 1996-02-01
US08/595,323 US5822523A (en) 1996-02-01 1996-02-01 Server-group messaging system for interactive applications

Publications (1)

Publication Number Publication Date
WO1997028502A1 true WO1997028502A1 (en) 1997-08-07

Family

ID=24382776

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1997/000567 WO1997028502A1 (en) 1996-02-01 1997-01-21 Server-group messaging system for interactive applications

Country Status (5)

Country Link
US (3) US5822523A (en)
EP (1) EP1012724A1 (en)
JP (1) JP2000504133A (en)
AU (1) AU1747197A (en)
WO (1) WO1997028502A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7006616B1 (en) 1999-05-21 2006-02-28 Terayon Communication Systems, Inc. Teleconferencing bridge with EdgePoint mixing
US7079502B2 (en) 2000-05-16 2006-07-18 Nec Corporation Packet communication system, mobile communication system, and addressing method for communication
US10856031B2 (en) 2003-04-15 2020-12-01 MedialP, Inc. Method and apparatus for generating interactive programming in a communication network
USRE48579E1 (en) 2002-04-15 2021-06-01 Media Ip, Inc. Method and apparatus for internet-based interactive programming

Families Citing this family (310)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8352400B2 (en) 1991-12-23 2013-01-08 Hoffberg Steven M Adaptive pattern recognition based controller apparatus and method and human-factored interface therefore
US6240451B1 (en) * 1995-05-25 2001-05-29 Punch Networks Corporation Method and apparatus for automatically disseminating information over a network
US8574074B2 (en) 2005-09-30 2013-11-05 Sony Computer Entertainment America Llc Advertising impression determination
US7895076B2 (en) 1995-06-30 2011-02-22 Sony Computer Entertainment Inc. Advertisement insertion, profiling, impression, and feedback
US6088749A (en) * 1995-11-30 2000-07-11 Excel Switching Corp. Universal API with generic call processing message having user-defined PPL event ID and generic response message for communications between telecommunications switch and host application
US6343313B1 (en) * 1996-03-26 2002-01-29 Pixion, Inc. Computer conferencing system with real-time multipoint, multi-speed, multi-stream scalability
US5956491A (en) 1996-04-01 1999-09-21 Marks; Daniel L. Group communications multiplexing system
US5905871A (en) * 1996-10-10 1999-05-18 Lucent Technologies Inc. Method of multicasting
US6758755B2 (en) * 1996-11-14 2004-07-06 Arcade Planet, Inc. Prize redemption system for games executed over a wide area network
US20080227538A1 (en) * 1996-11-14 2008-09-18 Bally Gaming Inc. Game prize controller and system
US8235821B2 (en) * 1996-11-14 2012-08-07 Bally Gaming, Inc. Progressive controller and TCP/IP in gaming system
US6041343A (en) * 1996-12-19 2000-03-21 International Business Machines Corp. Method and system for a hybrid peer-server communications structure
US6175831B1 (en) * 1997-01-17 2001-01-16 Six Degrees, Inc. Method and apparatus for constructing a networking database and system
RU2144274C1 (en) * 1997-02-07 2000-01-10 Самсунг Электроникс Ко., Лтд. Method for transmission and processing of message groups in electronic mail system
US6055364A (en) * 1997-07-31 2000-04-25 Cisco Technology, Inc. Content-based filtering of multicast information
US6850484B1 (en) * 1997-08-21 2005-02-01 At&T Corp. Packet redirection and message stream management
US6259701B1 (en) * 1997-09-11 2001-07-10 At&T Corp. Method and system for a unicast endpoint client to access a multicast internet protocol (IP) session
DK0909068T3 (en) * 1997-10-13 2001-05-07 X Way Rights B V Method and apparatus for structured communication
US6216167B1 (en) * 1997-10-31 2001-04-10 Nortel Networks Limited Efficient path based forwarding and multicast forwarding
US6185623B1 (en) * 1997-11-07 2001-02-06 International Business Machines Corporation Method and system for trivial file transfer protocol (TFTP) subnet broadcast
US6108652A (en) * 1997-12-01 2000-08-22 At&T Corp. Multicast probability-base grouping of nodes in switched network for improved broadcast search
US6295560B1 (en) * 1997-12-05 2001-09-25 Kabushiki Kaisha Toshiba Data delivery system with load distribution among data delivery units using shared lower address and unique lower layer address
EP1040645B1 (en) * 1997-12-16 2018-03-28 Nokia Solutions and Networks GmbH & Co. KG Method and apparatus for receiving full-motion digital video multi-casts, interactive data and interactive voice via a dsl circuit
US6519686B2 (en) * 1998-01-05 2003-02-11 Intel Corporation Information streaming in a multi-process system using shared memory
US6118785A (en) * 1998-04-07 2000-09-12 3Com Corporation Point-to-point protocol with a signaling channel
US6112245A (en) * 1998-04-07 2000-08-29 3Com Corporation Session establishment for static links in Point-to-Point Protocol sessions
US6301229B1 (en) 1998-04-07 2001-10-09 3Com Corporation Distribution of protocol processes from network elements to end stations
US6097720A (en) * 1998-04-07 2000-08-01 3Com Corporation Enabling multicast distribution efficiencies in a dialup access environment
JPH11331191A (en) 1998-05-18 1999-11-30 Nec Corp Subscriber access device
JP3620010B2 (en) * 1998-05-22 2005-02-16 富士通株式会社 Apparatus and program recording medium used in wireless communication system
US6163810A (en) * 1998-06-02 2000-12-19 At&T Corp. System and method for managing the exchange of information between multicast and unicast hosts
US6098100A (en) * 1998-06-08 2000-08-01 Silicon Integrated Systems Corp. Method and apparatus for detecting a wake packet issued by a network device to a sleeping node
US6125116A (en) * 1998-07-27 2000-09-26 Nec Corporation Message sets for a flexible programmable ATM access multiplexer using functional separation of line and protocol related functions
FR2781860B1 (en) * 1998-07-31 2000-09-01 Inst Francais Du Petrole ALTERNATE COMMUNICATION SYSTEM OF AT LEAST FOUR FLUIDS AND ITS APPLICATION IN A SIMULATED MOBILE BED SEPARATION PROCESS
EP0993163A1 (en) 1998-10-05 2000-04-12 Backweb Technologies Ltd. Distributed client-based data caching system and method
JP3519616B2 (en) * 1998-10-21 2004-04-19 株式会社日立製作所 Relay device
US6366913B1 (en) * 1998-10-21 2002-04-02 Netscape Communications Corporation Centralized directory services supporting dynamic group membership
US6272522B1 (en) * 1998-11-17 2001-08-07 Sun Microsystems, Incorporated Computer data packet switching and load balancing system using a general-purpose multiprocessor architecture
US6625813B1 (en) 1999-01-12 2003-09-23 Sony Corporation Digital television broadcast architecture for massively multi-user application
US6502140B1 (en) * 1999-01-29 2002-12-31 International Business Machines Corporation Multicast support for small groups
US6415312B1 (en) * 1999-01-29 2002-07-02 International Business Machines Corporation Reliable multicast for small groups
US7904187B2 (en) 1999-02-01 2011-03-08 Hoffberg Steven M Internet appliance system and method
WO2000072532A1 (en) * 1999-05-24 2000-11-30 Rutgers, The State University Of New Jersey System and method for network packet reduction
US7100000B1 (en) * 1999-05-28 2006-08-29 International Business Machines Corporation System and methods for processing audio using multiple speech technologies
US7009969B1 (en) * 1999-06-08 2006-03-07 Cisco Technology, Inc. Local area network and message packet for a telecommunications device
US6625773B1 (en) * 1999-06-09 2003-09-23 International Business Machines Corporation System for multicast communications in packet switched networks
US20040193722A1 (en) * 1999-08-30 2004-09-30 Donovan Kevin Remington Joseph Bartholomew Universal instant messaging system for the internet
US6640244B1 (en) * 1999-08-31 2003-10-28 Accenture Llp Request batcher in a transaction services patterns environment
US6415317B1 (en) 1999-10-01 2002-07-02 Joshua Michael Yelon Software system for reducing the appearance of latency in a multi-user environment
US6917963B1 (en) * 1999-10-05 2005-07-12 Veritas Operating Corporation Snapshot image for the application state of unshareable and shareable data
WO2001026267A1 (en) * 1999-10-05 2001-04-12 Ejasent Inc. Virtual port multiplexing
US6716103B1 (en) 1999-10-07 2004-04-06 Nintendo Co., Ltd. Portable game machine
ATE382220T1 (en) * 1999-10-12 2008-01-15 Alcatel Lucent APPARATUS AND METHOD FOR COMPRESSING MULTI-MESSAGE DESTINATION ADDRESSES
US7079495B1 (en) 2000-01-04 2006-07-18 Cisco Technology, Inc. System and method for enabling multicast telecommunications
US6804254B1 (en) 2000-01-04 2004-10-12 Cisco Technology, Inc. System and method for maintaining a communication link
US7069432B1 (en) * 2000-01-04 2006-06-27 Cisco Technology, Inc. System and method for providing security in a telecommunication network
US7006494B1 (en) 2000-01-04 2006-02-28 Cisco Technology, Inc. System and method for a virtual telephony intermediary
WO2001052072A1 (en) * 2000-01-14 2001-07-19 Multitude, Inc. Apparatus and method for creating moderated forums
WO2001055879A1 (en) * 2000-01-28 2001-08-02 Ibeam Broadcasting Corporation A system and method for determining optimal server in a distributed network for serving content streams
US7508753B2 (en) * 2000-01-31 2009-03-24 At&T Intellectual Property, Ii, L.P. Packet redirection and message stream management
US20020029287A1 (en) * 2000-02-02 2002-03-07 Yechiam Yemini Method and apparatus for dynamically addressing a circuits based network
US6513003B1 (en) 2000-02-03 2003-01-28 Fair Disclosure Financial Network, Inc. System and method for integrated delivery of media and synchronized transcription
US7302497B2 (en) * 2000-02-08 2007-11-27 Telefonaktiebolaget Lm Ericsson (Publ) Using internet protocol (IP) in radio access network
US6724770B1 (en) * 2000-02-17 2004-04-20 Kenneth P. Birman Multicast protocol with reduced buffering requirements
US6651086B1 (en) 2000-02-22 2003-11-18 Yahoo! Inc. Systems and methods for matching participants to a conversation
US7330824B1 (en) 2000-02-25 2008-02-12 Navic Systems, Inc. Method and system for content profiling and activation
US6714992B1 (en) * 2000-02-25 2004-03-30 Navic Systems, Inc. Method and system for embedded network device installation
US7392281B1 (en) 2000-02-25 2008-06-24 Navic Systems, Inc. System and method for providing guaranteed delivery of messages to embedded devices over a data network
US6845396B1 (en) 2000-02-25 2005-01-18 Navic Systems, Inc. Method and system for content deployment and activation
US6751218B1 (en) * 2000-02-26 2004-06-15 Avaya Technology Corp. Method and system for ATM-coupled multicast service over IP networks
US6807562B1 (en) * 2000-02-29 2004-10-19 Microsoft Corporation Automatic and selective assignment of channels to recipients of voice chat data
US7240093B1 (en) * 2000-02-29 2007-07-03 Microsoft Corporation Use of online messaging to facilitate selection of participants in game play
EP1266507B1 (en) * 2000-03-17 2004-06-02 America Online, Inc. Home-networking
US7725523B2 (en) 2000-04-11 2010-05-25 Bolnick David A System, method and computer program product for gathering and delivering personalized user information
US8818855B2 (en) * 2000-04-18 2014-08-26 Intellectual Ventures I Llc System and method for aggregating information over a wide area network
US6826621B1 (en) 2000-04-24 2004-11-30 International Business Machines Corporation Method and system for aggregating interface addresses
US7065079B1 (en) * 2000-05-04 2006-06-20 Cisco Technology, Inc. VC sharing for multicast in a computer network
US7151773B1 (en) 2000-05-05 2006-12-19 Fujitsu Limited System and method for connectionless/connection oriented signal transport
US7133403B1 (en) * 2000-05-05 2006-11-07 Fujitsu Limited Transport network and method
US7173912B2 (en) * 2000-05-05 2007-02-06 Fujitsu Limited Method and system for modeling and advertising asymmetric topology of a node in a transport network
US6775229B1 (en) 2000-05-05 2004-08-10 Fujitsu Network Communications, Inc. Method and system for providing a protection path for connection-oriented signals in a telecommunications network
US7385917B1 (en) 2000-05-05 2008-06-10 Fujitsu Limited Method and system for providing a protection path for connectionless signals in a telecommunications network
US6515966B1 (en) 2000-05-05 2003-02-04 Fujitsu Network Communications, Inc. System and method for application object transport
US7047176B2 (en) * 2000-05-05 2006-05-16 Fujitsu Limited Method and system for hardware simulation
US7075927B2 (en) * 2000-05-05 2006-07-11 Fujitsu Limited Method and system for quality of service (QoS) support in a packet-switched network
US7058730B2 (en) * 2000-05-05 2006-06-06 Fujitsu Limited Unique address space and method for a transport network
US6693909B1 (en) 2000-05-05 2004-02-17 Fujitsu Network Communications, Inc. Method and system for transporting traffic in a packet-switched network
US6845389B1 (en) * 2000-05-12 2005-01-18 Nortel Networks Limited System and method for broadband multi-user communication sessions
US20060067500A1 (en) * 2000-05-15 2006-03-30 Christofferson Frank C Teleconferencing bridge with edgepoint mixing
FR2809559B1 (en) * 2000-05-25 2002-08-16 Cit Alcatel METHOD FOR MUTUAL TRANSFERS OF CONTROL PARAMETERS THROUGH A COMMUNICATION NETWORK
US7056217B1 (en) 2000-05-31 2006-06-06 Nintendo Co., Ltd. Messaging service for video game systems with buddy list that displays game being played
US8131867B1 (en) * 2000-06-01 2012-03-06 Qualcomm Incorporated Dynamic layer congestion control for multicast transport
AU2001265079A1 (en) * 2000-06-09 2001-12-24 Rashida A. Karmali Application specific live streaming multimedia mixer apparatus, systems and methods
DE10030272A1 (en) * 2000-06-20 2002-01-03 Siemens Ag Procedure for the transmission of short messages
US20020001310A1 (en) * 2000-06-29 2002-01-03 Khanh Mai Virtual multicasting
US6862430B1 (en) * 2000-07-05 2005-03-01 Echelon Corporation System and method for selecting repeaters
US7111163B1 (en) 2000-07-10 2006-09-19 Alterwan, Inc. Wide area network using internet with quality of service
US7103016B1 (en) * 2000-08-11 2006-09-05 Echelon Corporation System and method for providing transaction control on a data network
ATE326097T1 (en) * 2000-08-25 2006-06-15 Cit Alcatel METHOD FOR PROVIDING A BIDIRECTIONAL CONNECTION IN A NETWORK FOR THE MULTIPLE TRANSMISSION OF DATA STREAMS USING THE INTERNET PROTOCOL AND NETWORK FOR APPLYING THE METHOD
US20020026480A1 (en) * 2000-08-25 2002-02-28 Takanori Terada E-mail system
US7596784B2 (en) * 2000-09-12 2009-09-29 Symantec Operating Corporation Method system and apparatus for providing pay-per-use distributed computing resources
US6742028B1 (en) * 2000-09-15 2004-05-25 Frank Wang Content management and sharing
US7870183B1 (en) * 2000-10-25 2011-01-11 International Business Machines Corporation Multicast enabled mail
US7244181B2 (en) * 2000-11-14 2007-07-17 Netamin Communication Corp. Multi-player game employing dynamic re-sequencing
US20020065929A1 (en) * 2000-11-28 2002-05-30 Navic Systems Inc. Protocol extensions to increase reliability of bulk data transmissions
US7047273B2 (en) 2000-11-28 2006-05-16 Navic Systems, Inc. Load balancing in set top cable box environment
US7653743B2 (en) * 2000-11-28 2010-01-26 Microsoft Corporation Protocol for throttling high volume messages
US6760330B2 (en) * 2000-12-18 2004-07-06 Sun Microsystems, Inc. Community separation control in a multi-community node
US6931028B1 (en) * 2000-12-28 2005-08-16 Cisco Technology, Inc. Scaleable RSVP signaling between VoIP dial-peers for tandem voice solutions
US8751310B2 (en) 2005-09-30 2014-06-10 Sony Computer Entertainment America Llc Monitoring advertisement impressions
US6963919B1 (en) * 2001-02-28 2005-11-08 Agilent Technologies, Inc. Method and system for improving computer network performance
US20020129159A1 (en) * 2001-03-09 2002-09-12 Michael Luby Multi-output packet server with independent streams
US7177950B2 (en) * 2001-04-26 2007-02-13 Microsoft Corporation Synchronization and recovery of peers in a peer to peer environment
WO2003105006A1 (en) * 2001-04-30 2003-12-18 America Online, Inc. Load balancing with direct terminal response
US7124166B2 (en) 2001-04-30 2006-10-17 Aol Llc Duplicating digital streams for digital conferencing using switching technologies
US7266609B2 (en) * 2001-04-30 2007-09-04 Aol Llc Generating multiple data streams from a single data source
US8572278B2 (en) * 2001-04-30 2013-10-29 Facebook, Inc. Generating multiple data streams from a single data source
US7292571B2 (en) * 2001-04-30 2007-11-06 Aol Llc, A Delaware Limited Liability Company Load balancing with direct terminal response
US7237033B2 (en) * 2001-04-30 2007-06-26 Aol Llc Duplicating switch for streaming data units to a terminal
US7227658B2 (en) * 2001-08-01 2007-06-05 Canon Kabushiki Kaisha Content management in a broadband printing system
US20030037102A1 (en) * 2001-08-14 2003-02-20 Philippe Eckert Message broker
US7103011B2 (en) * 2001-08-30 2006-09-05 Motorola, Inc. Use of IP-multicast technology for 2-party calls in mobile communication networks
US7389359B2 (en) * 2001-10-19 2008-06-17 Foundry Networks, Inc. Method and system for intelligently forwarding multicast packets
US8387111B2 (en) 2001-11-01 2013-02-26 International Business Machines Corporation Type independent permission based access control
US6971064B2 (en) 2001-11-14 2005-11-29 Lite-On Technology Corporation Method for automatically forwarding and replying short message
US8147334B2 (en) * 2003-09-04 2012-04-03 Jean-Marie Gatto Universal game server
US20030120800A1 (en) * 2001-12-06 2003-06-26 Edwards Systens Technology, Inc. Network layer protocol
JP4386732B2 (en) 2002-01-08 2009-12-16 セブン ネットワークス, インコーポレイテッド Mobile network connection architecture
WO2003065242A1 (en) * 2002-01-25 2003-08-07 Tibco Software, Inc. Single applet to communicate with multiple html elements contained inside of multiple categories on a page
US8959230B2 (en) * 2002-01-28 2015-02-17 Qualcomm Incorporated Method and apparatus for negotiation of transmission parameters for broadcast/multicast services
US7313135B2 (en) * 2002-01-31 2007-12-25 Mosaid Technologies, Inc. Trunking in a matrix
US6922718B2 (en) * 2002-02-01 2005-07-26 Dell Products L.P. Method and system for participating locations in a multi-point video conference
EP1333693A1 (en) * 2002-02-04 2003-08-06 Koninklijke KPN N.V. Method and system for transmitting messages to terminal groups
US7143135B2 (en) * 2002-02-08 2006-11-28 Microsoft Corporation Automatic participant evaluation in computer mediated persistent conversations
US20030236120A1 (en) * 2002-04-19 2003-12-25 Kenneth Reece Method and device for determining the physical location and identity of a user
AU2002305963B2 (en) * 2002-06-25 2010-05-13 Chien Liang Chua Networking system
US8028092B2 (en) 2002-06-28 2011-09-27 Aol Inc. Inserting advertising content
US7383339B1 (en) 2002-07-31 2008-06-03 Aol Llc, A Delaware Limited Liability Company Local proxy server for establishing device controls
US20040107298A1 (en) * 2002-08-14 2004-06-03 Cedric Westphal Layered compression architecture for multi-hop header compression
US7571256B2 (en) * 2002-09-24 2009-08-04 Sun Microsystems, Inc. Data communication execution thread
ATE373269T1 (en) * 2002-10-08 2007-09-15 Koninkl Philips Electronics Nv INTEGRATED CIRCUIT AND METHOD FOR SENDING REQUESTS
GB0227048D0 (en) * 2002-11-20 2002-12-24 3Com Corp Network units for use in and organisation of cascade systems
US7263102B2 (en) * 2002-11-27 2007-08-28 At&T Intellectual Property, Inc. Multi-path gateway communications device
US7379464B2 (en) * 2002-11-27 2008-05-27 At&T Bls Intellectual Property, Inc. Personal digital gateway
US7224698B2 (en) * 2002-11-27 2007-05-29 Bellsouth Intellectual Property Corporation Edge side assembler
US7359939B2 (en) * 2002-12-06 2008-04-15 Alcatel Canada, Inc. Fast service restoration for lost IGMP leave requests
US7853563B2 (en) 2005-08-01 2010-12-14 Seven Networks, Inc. Universal data aggregation
US8468126B2 (en) 2005-08-01 2013-06-18 Seven Networks, Inc. Publishing data in an information community
US7917468B2 (en) 2005-08-01 2011-03-29 Seven Networks, Inc. Linking of personal information management data
US9110853B2 (en) * 2003-03-10 2015-08-18 Oracle America, Inc. Computer system with multiple classes of device IDs
US20040186888A1 (en) * 2003-03-19 2004-09-23 International Business Machines Corporation Method and system for transferring real-time messages between multiple non-connected messaging servers
US20040225738A1 (en) * 2003-05-09 2004-11-11 Patel Dipak M. Methods, systems and computer program products for replicating servers and network traffic for problem determination and/or tuning
US7455590B2 (en) * 2003-05-09 2008-11-25 Microsoft Corporation Sending messages in response to events occurring on a gaming service
US7769885B1 (en) * 2003-05-23 2010-08-03 Juniper Networks, Inc. Determining liveness of protocols and interfaces
US7385937B2 (en) * 2003-07-23 2008-06-10 International Business Machines Corporation Method and system for determining a path between two points of an IP network over which datagrams are transmitted
US7111047B2 (en) * 2003-08-08 2006-09-19 Teamon Systems, Inc. Communications system providing message aggregation features and related methods
US7437457B1 (en) 2003-09-08 2008-10-14 Aol Llc, A Delaware Limited Liability Company Regulating concurrent logins associated with a single account
US7743367B1 (en) * 2003-12-30 2010-06-22 Sap Ag Registration method for supporting bytecode modification
US7508814B1 (en) 2003-12-30 2009-03-24 At&T Intellectual Property, Ii, L.P. Electronic loop provisioning methods and systems
US8838743B2 (en) * 2004-02-13 2014-09-16 Intel Corporation Apparatus and method for a dynamically extensible virtual switch
US7418491B2 (en) * 2004-02-19 2008-08-26 International Business Machines Corporation Architecture for a centralized management system
US20050198437A1 (en) * 2004-03-08 2005-09-08 Mukherjee Shubhendu S. Method and system for coalescing coherence messages
US20050262205A1 (en) * 2004-04-30 2005-11-24 Nikolov Radoslav I Delivering messages in an enterprise messaging system using message selector hierarchy
US7519669B2 (en) * 2004-04-30 2009-04-14 Sap Aktiengesellschaft Prioritizing producers and consumers of an enterprise messaging system
US20060031367A1 (en) * 2004-06-04 2006-02-09 John Buford Instant messaging and presence using virtual devices
US20060036689A1 (en) * 2004-06-04 2006-02-16 John Buford Personal messaging proxy
US20060031317A1 (en) * 2004-06-04 2006-02-09 Wanrong Lin System for synthesizing instant message and presence attributes from polled, queried and event information sources
US7785197B2 (en) * 2004-07-29 2010-08-31 Nintendo Co., Ltd. Voice-to-text chat conversion for remote video game play
US7491123B2 (en) * 2004-07-29 2009-02-17 Nintendo Co., Ltd. Video game voice chat with amplitude-based virtual ranging
US20060041624A1 (en) * 2004-08-18 2006-02-23 International Business Machines Corporation System and method for distributing an electronic message
US8763157B2 (en) 2004-08-23 2014-06-24 Sony Computer Entertainment America Llc Statutory license restricted digital media playback on portable devices
JP2006074379A (en) * 2004-09-01 2006-03-16 Ntt Docomo Inc Server, transmission terminal, mobile communication system and mobile communication method
US8010082B2 (en) 2004-10-20 2011-08-30 Seven Networks, Inc. Flexible billing architecture
US7441271B2 (en) 2004-10-20 2008-10-21 Seven Networks Method and apparatus for intercepting events in a communication system
US7706781B2 (en) 2004-11-22 2010-04-27 Seven Networks International Oy Data security in a mobile e-mail service
FI117152B (en) 2004-12-03 2006-06-30 Seven Networks Internat Oy E-mail service provisioning method for mobile terminal, involves using domain part and further parameters to generate new parameter set in list of setting parameter sets, if provisioning of e-mail service is successful
US7969959B2 (en) 2004-12-16 2011-06-28 Critical Response Systems, Inc. Method and apparatus for efficient and deterministic group alerting
US20080274810A1 (en) * 2005-02-25 2008-11-06 Sawako-Eeva Hayashi Controlling Communications Between Players of a Multi-Player Game
US7752633B1 (en) 2005-03-14 2010-07-06 Seven Networks, Inc. Cross-platform event engine
US8438633B1 (en) 2005-04-21 2013-05-07 Seven Networks, Inc. Flexible real-time inbox access
US7796742B1 (en) 2005-04-21 2010-09-14 Seven Networks, Inc. Systems and methods for simplified provisioning
WO2006136660A1 (en) 2005-06-21 2006-12-28 Seven Networks International Oy Maintaining an ip connection in a mobile network
US7596141B2 (en) * 2005-06-30 2009-09-29 Intel Corporation Packet classification using encoded addresses
US8069166B2 (en) 2005-08-01 2011-11-29 Seven Networks, Inc. Managing user-to-user contact with inferred presence information
US8626584B2 (en) 2005-09-30 2014-01-07 Sony Computer Entertainment America Llc Population of an advertisement reference list
US20070118425A1 (en) 2005-10-25 2007-05-24 Podbridge, Inc. User device agent for asynchronous advertising in time and space shifted media network
US8676900B2 (en) 2005-10-25 2014-03-18 Sony Computer Entertainment America Llc Asynchronous advertising placement based on metadata
US11004089B2 (en) 2005-10-25 2021-05-11 Sony Interactive Entertainment LLC Associating media content files with advertisements
US10657538B2 (en) 2005-10-25 2020-05-19 Sony Interactive Entertainment LLC Resolution of advertising rules
US20070104186A1 (en) * 2005-11-04 2007-05-10 Bea Systems, Inc. System and method for a gatekeeper in a communications network
CN101346634B (en) * 2005-11-04 2012-10-24 甲骨文国际公司 System and method for a gatekeeper in a communications network
US7769395B2 (en) 2006-06-20 2010-08-03 Seven Networks, Inc. Location-based operations and messaging
US8204005B2 (en) * 2006-03-09 2012-06-19 Intel Corporation Wireless communication device and method for dynamic bidirectional aggregation of MAC frames with delayed channel access in a wireless network
WO2007120741A2 (en) * 2006-04-12 2007-10-25 Netamin Communication Corp. Team-based networked video gaming and automatic event management
US8140618B2 (en) * 2006-05-04 2012-03-20 Citrix Online Llc Methods and systems for bandwidth adaptive N-to-N communication in a distributed system
US8769019B2 (en) * 2006-05-04 2014-07-01 Citrix Systems, Inc. Methods and systems for managing shared state within a distributed system with varying consistency and consensus semantics
JP5313882B2 (en) 2006-05-05 2013-10-09 ソニー コンピュータ エンタテインメント アメリカ リミテッド ライアビリテイ カンパニー Device for displaying main content and auxiliary content
US8001250B2 (en) * 2006-05-16 2011-08-16 Oracle International Corporation SIP and HTTP convergence in network computing environments
US8112525B2 (en) * 2006-05-16 2012-02-07 Oracle International Corporation Engine near cache for reducing latency in a telecommunications environment
US8171466B2 (en) * 2006-05-16 2012-05-01 Oracle International Corporation Hitless application upgrade for SIP server architecture
US8219697B2 (en) * 2006-05-17 2012-07-10 Oracle International Corporation Diameter protocol and SH interface support for SIP server architecture
GB2438454B (en) * 2006-05-26 2008-08-06 Motorola Inc Method and system for communication
US7945620B2 (en) * 2006-06-13 2011-05-17 International Business Machines Corporation Chat tool for concurrently chatting over more than one interrelated chat channels
US7725577B2 (en) * 2006-07-31 2010-05-25 Sap Ag Method and system to adaptively manage the quality of service of interactions between smart item networks and enterprise applications
US20080072292A1 (en) * 2006-09-01 2008-03-20 Narjala Ranjit S Secure device introduction with capabilities assessment
US7661027B2 (en) * 2006-10-10 2010-02-09 Bea Systems, Inc. SIP server architecture fault tolerance and failover
US8078737B2 (en) * 2006-12-13 2011-12-13 Oracle International Corporation System and method for efficient storage of long-lived session state in a SIP server
US20080147551A1 (en) * 2006-12-13 2008-06-19 Bea Systems, Inc. System and Method for a SIP Server with Online Charging
US9667430B2 (en) * 2006-12-13 2017-05-30 Oracle International Corporation System and method for a SIP server with offline charging
US20080159286A1 (en) * 2006-12-28 2008-07-03 Moore Martin T Contextualized broadcast message channel for activity-centric collaborative computing
US8805425B2 (en) 2007-06-01 2014-08-12 Seven Networks, Inc. Integrated messaging
US8693494B2 (en) 2007-06-01 2014-04-08 Seven Networks, Inc. Polling
US20090049128A1 (en) * 2007-08-17 2009-02-19 Sony Computer Entertainment America Inc. Schemes for game chat routing and taunt control
US7761525B2 (en) * 2007-08-23 2010-07-20 International Business Machines Corporation System and method for providing improved time references in documents
US7921686B2 (en) * 2007-08-28 2011-04-12 Cisco Technology, Inc. Highly scalable architecture for application network appliances
US8416247B2 (en) 2007-10-09 2013-04-09 Sony Computer Entertaiment America Inc. Increasing the number of advertising impressions in an interactive environment
US8364181B2 (en) 2007-12-10 2013-01-29 Seven Networks, Inc. Electronic-mail filtering for mobile devices
US8793305B2 (en) 2007-12-13 2014-07-29 Seven Networks, Inc. Content delivery to a mobile device from a content service
US9002828B2 (en) 2007-12-13 2015-04-07 Seven Networks, Inc. Predictive content delivery
US8107921B2 (en) 2008-01-11 2012-01-31 Seven Networks, Inc. Mobile virtual network operator
US8862657B2 (en) 2008-01-25 2014-10-14 Seven Networks, Inc. Policy based content service
US20090193338A1 (en) 2008-01-28 2009-07-30 Trevor Fiatal Reducing network and battery consumption during content delivery and playback
US8769558B2 (en) 2008-02-12 2014-07-01 Sony Computer Entertainment America Llc Discovery and analytics for episodic downloaded media
JP5149709B2 (en) * 2008-06-18 2013-02-20 株式会社エヌ・ティ・ティ・ドコモ Short message service management system, short message transfer device, short message transfer form regulation information supply device, and short message transfer method
US8787947B2 (en) 2008-06-18 2014-07-22 Seven Networks, Inc. Application discovery on mobile devices
US8078158B2 (en) 2008-06-26 2011-12-13 Seven Networks, Inc. Provisioning applications for a mobile device
SG157990A1 (en) * 2008-07-04 2010-01-29 3Rd Brand Pte Ltd System and method for facilitating the growth of a mobile community
WO2010014778A1 (en) 2008-07-31 2010-02-04 Juma Technology Corp. Method for overcoming address conflicts among disparate networks in a network management system
US8909759B2 (en) 2008-10-10 2014-12-09 Seven Networks, Inc. Bandwidth measurement
US8176200B2 (en) * 2008-10-24 2012-05-08 Microsoft Corporation Distributed aggregation on an overlay network
US8145711B2 (en) * 2008-11-05 2012-03-27 At&T Intellectual Property I, L.P. Multi-player game data via multicast transmission
US8160060B2 (en) * 2008-11-10 2012-04-17 Infosys Technologies Limited System and method for transferring data using variance based messaging
US20100160047A1 (en) * 2008-12-22 2010-06-24 Microsoft Corporation Scalable Game Primitives / Distributed Real-Time Aggregation Of Player Data
US8423088B2 (en) * 2009-07-22 2013-04-16 Microsoft Corporation Aggregated, interactive communication timeline
US8763090B2 (en) 2009-08-11 2014-06-24 Sony Computer Entertainment America Llc Management of ancillary content delivery and presentation
US9049617B2 (en) * 2009-09-23 2015-06-02 At&T Intellectual Property I, L.P. Signaling-less dynamic call setup and teardown by utilizing observed session state information
US9361165B2 (en) * 2009-12-03 2016-06-07 International Business Machines Corporation Automated merger of logically associated messages in a message queue
US9205328B2 (en) 2010-02-18 2015-12-08 Activision Publishing, Inc. Videogame system and method that enables characters to earn virtual fans by completing secondary objectives
JP5477095B2 (en) * 2010-03-19 2014-04-23 富士通株式会社 Information processing system, apparatus, method, and program
TW201209697A (en) 2010-03-30 2012-03-01 Michael Luna 3D mobile user interface with configurable workspace management
US9682324B2 (en) 2010-05-12 2017-06-20 Activision Publishing, Inc. System and method for enabling players to participate in asynchronous, competitive challenges
CN102986166B (en) * 2010-07-23 2016-07-06 瑞典爱立信有限公司 Record controls plane event
US9043433B2 (en) 2010-07-26 2015-05-26 Seven Networks, Inc. Mobile network traffic coordination across multiple applications
US8838783B2 (en) 2010-07-26 2014-09-16 Seven Networks, Inc. Distributed caching for resource and mobile network traffic management
CA2857458A1 (en) 2010-07-26 2012-02-09 Michael Luna Mobile application traffic optimization
US9077630B2 (en) 2010-07-26 2015-07-07 Seven Networks, Inc. Distributed implementation of dynamic wireless traffic policy
WO2012050618A1 (en) 2010-10-16 2012-04-19 James Charles Vago Multimedia methods, devices and systems
CN107071725B (en) 2010-10-27 2020-12-25 瑞典爱立信有限公司 Method, network entity, mobile terminal and medium for cellular network services
US8484314B2 (en) 2010-11-01 2013-07-09 Seven Networks, Inc. Distributed caching in a wireless network of content delivered for a mobile application over a long-held request
WO2012061430A2 (en) 2010-11-01 2012-05-10 Michael Luna Distributed management of keep-alive message signaling for mobile network resource conservation and optimization
US9330196B2 (en) 2010-11-01 2016-05-03 Seven Networks, Llc Wireless traffic management system cache optimization using http headers
US8843153B2 (en) 2010-11-01 2014-09-23 Seven Networks, Inc. Mobile traffic categorization and policy for network use optimization while preserving user experience
US8190701B2 (en) 2010-11-01 2012-05-29 Seven Networks, Inc. Cache defeat detection and caching of content addressed by identifiers intended to defeat cache
WO2012060995A2 (en) 2010-11-01 2012-05-10 Michael Luna Distributed caching in a wireless network of content delivered for a mobile application over a long-held request
GB2499534B (en) 2010-11-01 2018-09-19 Seven Networks Llc Caching adapted for mobile application behavior and network conditions
US8166164B1 (en) 2010-11-01 2012-04-24 Seven Networks, Inc. Application and network-based long poll request detection and cacheability assessment therefor
US9060032B2 (en) 2010-11-01 2015-06-16 Seven Networks, Inc. Selective data compression by a distributed traffic management system to reduce mobile data traffic and signaling traffic
GB2500327B (en) 2010-11-22 2019-11-06 Seven Networks Llc Optimization of resource polling intervals to satisfy mobile device requests
CN103404193B (en) 2010-11-22 2018-06-05 七网络有限责任公司 The connection that adjustment data transmission is established with the transmission being optimized for through wireless network
US9325662B2 (en) 2011-01-07 2016-04-26 Seven Networks, Llc System and method for reduction of mobile network traffic used for domain name system (DNS) queries
WO2012145541A2 (en) 2011-04-19 2012-10-26 Seven Networks, Inc. Social caching for device resource sharing and management
GB2505585B (en) 2011-04-27 2015-08-12 Seven Networks Inc Detecting and preserving state for satisfying application requests in a distributed proxy and cache system
CA2797631C (en) 2011-04-27 2013-11-19 Seven Networks, Inc. System and method for making requests on behalf of a mobile device based on atomic processes for mobile network traffic relief
US8984581B2 (en) 2011-07-27 2015-03-17 Seven Networks, Inc. Monitoring mobile application activities for malicious traffic on a mobile device
TWI591992B (en) 2011-11-16 2017-07-11 Alibaba Group Holding Ltd Group communication relationship establishment, web page display method and related device
US9679009B2 (en) * 2011-11-17 2017-06-13 Sap Se Component independent process integration message search
US8934414B2 (en) 2011-12-06 2015-01-13 Seven Networks, Inc. Cellular or WiFi mobile traffic optimization based on public or private network destination
US8868753B2 (en) 2011-12-06 2014-10-21 Seven Networks, Inc. System of redundantly clustered machines to provide failover mechanisms for mobile traffic management and network resource conservation
WO2013086447A1 (en) 2011-12-07 2013-06-13 Seven Networks, Inc. Radio-awareness of mobile device for sending server-side control signals using a wireless network optimized transport protocol
US9009250B2 (en) 2011-12-07 2015-04-14 Seven Networks, Inc. Flexible and dynamic integration schemas of a traffic management system with various network operators for network traffic alleviation
WO2013090821A1 (en) 2011-12-14 2013-06-20 Seven Networks, Inc. Hierarchies and categories for management and deployment of policies for distributed wireless traffic optimization
US20130159511A1 (en) 2011-12-14 2013-06-20 Seven Networks, Inc. System and method for generating a report to a network operator by distributing aggregation of data
US9832095B2 (en) 2011-12-14 2017-11-28 Seven Networks, Llc Operation modes for mobile traffic optimization and concurrent management of optimized and non-optimized traffic
WO2013103988A1 (en) 2012-01-05 2013-07-11 Seven Networks, Inc. Detection and management of user interactions with foreground applications on a mobile device in distributed caching
US9203864B2 (en) 2012-02-02 2015-12-01 Seven Networks, Llc Dynamic categorization of applications for network access in a mobile network
WO2013116852A1 (en) 2012-02-03 2013-08-08 Seven Networks, Inc. User as an end point for profiling and optimizing the delivery of content and data in a wireless network
US9020888B1 (en) 2012-04-04 2015-04-28 Nectar Services Corp. Data replicating systems and data replication methods
US8812695B2 (en) 2012-04-09 2014-08-19 Seven Networks, Inc. Method and system for management of a virtual network connection without heartbeat messages
US20130268656A1 (en) 2012-04-10 2013-10-10 Seven Networks, Inc. Intelligent customer service/call center services enhanced using real-time and historical mobile application and traffic-related statistics collected by a distributed caching system in a mobile network
US8775631B2 (en) 2012-07-13 2014-07-08 Seven Networks, Inc. Dynamic bandwidth adjustment for browsing or streaming activity in a wireless network based on prediction of user behavior when interacting with mobile applications
US9161258B2 (en) 2012-10-24 2015-10-13 Seven Networks, Llc Optimized and selective management of policy deployment to mobile clients in a congested network to prevent further aggravation of network congestion
US9307493B2 (en) 2012-12-20 2016-04-05 Seven Networks, Llc Systems and methods for application management of mobile device radio state promotion and demotion
US9271238B2 (en) 2013-01-23 2016-02-23 Seven Networks, Llc Application or context aware fast dormancy
US8874761B2 (en) 2013-01-25 2014-10-28 Seven Networks, Inc. Signaling optimization in a wireless network for traffic utilizing proprietary and non-proprietary protocols
US8750123B1 (en) 2013-03-11 2014-06-10 Seven Networks, Inc. Mobile device equipped with mobile network congestion recognition to make intelligent decisions regarding connecting to an operator network
US9332404B2 (en) 2013-03-15 2016-05-03 Spireon, Inc. Server abstracted messaging system
US9065765B2 (en) 2013-07-22 2015-06-23 Seven Networks, Inc. Proxy server associated with a mobile carrier for enhancing mobile traffic management in a mobile network
CN103684808B (en) * 2013-12-30 2017-02-08 腾讯科技(深圳)有限公司 Method for sending broadcast messages and server
WO2015140745A2 (en) * 2014-03-19 2015-09-24 Ascom Deutschland Gmbh A system and method for managing workflows associated with a document exchanged between a first service provider and a second service provider
US10322351B2 (en) 2014-07-03 2019-06-18 Activision Publishing, Inc. Matchmaking system and method for multiplayer video games
GB2530781A (en) 2014-10-02 2016-04-06 Ibm Processing messages for retrieval from a message queuing system
US10118099B2 (en) 2014-12-16 2018-11-06 Activision Publishing, Inc. System and method for transparently styling non-player characters in a multiplayer video game
US10303782B1 (en) 2014-12-29 2019-05-28 Veritas Technologies Llc Method to allow multi-read access for exclusive access of virtual disks by using a virtualized copy of the disk
US10315113B2 (en) 2015-05-14 2019-06-11 Activision Publishing, Inc. System and method for simulating gameplay of nonplayer characters distributed across networked end user devices
US10471348B2 (en) 2015-07-24 2019-11-12 Activision Publishing, Inc. System and method for creating and sharing customized video game weapon configurations in multiplayer video games via one or more social networks
CN105516487B (en) * 2015-12-04 2019-07-16 上海斐讯数据通信技术有限公司 Broadcast receiving method, system and equipment
US10846779B2 (en) 2016-11-23 2020-11-24 Sony Interactive Entertainment LLC Custom product categorization of digital media content
US10500498B2 (en) 2016-11-29 2019-12-10 Activision Publishing, Inc. System and method for optimizing virtual games
US10860987B2 (en) 2016-12-19 2020-12-08 Sony Interactive Entertainment LLC Personalized calendar for digital media content-related events
US11040286B2 (en) 2017-09-27 2021-06-22 Activision Publishing, Inc. Methods and systems for improved content generation in multiplayer gaming environments
US10561945B2 (en) 2017-09-27 2020-02-18 Activision Publishing, Inc. Methods and systems for incentivizing team cooperation in multiplayer gaming environments
US10974150B2 (en) 2017-09-27 2021-04-13 Activision Publishing, Inc. Methods and systems for improved content customization in multiplayer gaming environments
US10765948B2 (en) 2017-12-22 2020-09-08 Activision Publishing, Inc. Video game content aggregation, normalization, and publication systems and methods
US10931991B2 (en) 2018-01-04 2021-02-23 Sony Interactive Entertainment LLC Methods and systems for selectively skipping through media content
US11679330B2 (en) 2018-12-18 2023-06-20 Activision Publishing, Inc. Systems and methods for generating improved non-player characters
US11097193B2 (en) 2019-09-11 2021-08-24 Activision Publishing, Inc. Methods and systems for increasing player engagement in multiplayer gaming environments
US11712627B2 (en) 2019-11-08 2023-08-01 Activision Publishing, Inc. System and method for providing conditional access to virtual gaming items
US11457057B2 (en) * 2020-03-11 2022-09-27 Microsoft Technology Licensing, Llc Systems and methods for establishing highly secure and resilient persistent communication connections
US11524234B2 (en) 2020-08-18 2022-12-13 Activision Publishing, Inc. Multiplayer video games with virtual characters having dynamically modified fields of view
US11351459B2 (en) 2020-08-18 2022-06-07 Activision Publishing, Inc. Multiplayer video games with virtual characters having dynamically generated attribute profiles unconstrained by predefined discrete values

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5079767A (en) * 1988-09-27 1992-01-07 Digital Equipment Corporation Method of multicast message distribution
US5309433A (en) * 1992-06-18 1994-05-03 International Business Machines Corp. Methods and apparatus for routing packets in packet transmission networks
US5309437A (en) * 1990-06-29 1994-05-03 Digital Equipment Corporation Bridge-like internet protocol router
US5329619A (en) * 1992-10-30 1994-07-12 Software Ag Cooperative processing interface and communication broker for heterogeneous computing environments
US5361256A (en) * 1992-11-27 1994-11-01 International Business Machines Corporation Inter-domain multicast routing

Family Cites Families (51)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4572509A (en) 1982-09-30 1986-02-25 Sitrick David H Video game network
US4470954A (en) * 1983-06-13 1984-09-11 Chiknas Steven G Rotor or carrier for centrifugal analyzer and bead washer
US4740954A (en) 1986-12-31 1988-04-26 Bell Communications Research, Inc. Multicast routing algorithm
US4984235A (en) 1987-04-27 1991-01-08 Thinking Machines Corporation Method and apparatus for routing message packets and recording the roofing sequence
US5117420A (en) 1987-04-27 1992-05-26 Thinking Machines Corporation Method and apparatus for routing message packets
US4807224A (en) 1987-08-21 1989-02-21 Naron Steven E Multicast data distribution system and method
US4998199A (en) 1987-10-02 1991-03-05 Namco Ltd. Game machine system with machine grouping feature
US5083800A (en) 1989-06-09 1992-01-28 Interactive Network, Inc. Game of skill or chance playable by several participants remote from each other in conjunction with a common event
US5089813A (en) 1989-07-19 1992-02-18 Motorola, Inc. Method of super battery saving in a selective call receiver
US4991171A (en) 1989-09-26 1991-02-05 At&T Bell Laboratories Broadcast packet switch network
US5150464A (en) * 1990-06-06 1992-09-22 Apple Computer, Inc. Local area network device startup process
JPH077975B2 (en) 1990-08-20 1995-01-30 インターナショナル・ビジネス・マシーンズ・コーポレイション System and method for controlling data transmission
US5251205A (en) 1990-09-04 1993-10-05 Digital Equipment Corporation Multiple protocol routing
US5245608A (en) * 1990-09-26 1993-09-14 International Business Machines Corporation Logical grouping of layer entities in a layered communication architecture
CA2048306A1 (en) * 1990-10-02 1992-04-03 Steven P. Miller Distributed configuration profile for computing system
US5297143A (en) 1990-12-03 1994-03-22 Echelon Systems, Corp. Network communication protocol including a reliable multicasting technique
US5150410A (en) 1991-04-11 1992-09-22 Itt Corporation Secure digital conferencing system
GB2254980B (en) 1991-04-16 1995-03-08 Roke Manor Research Improvements in or relating to multicast server apparatus
US5257113A (en) 1991-09-20 1993-10-26 International Business Machines Corporation Video mixing technique using JPEG compressed data
US5590281A (en) 1991-10-28 1996-12-31 The United States Of Americas As Represented By The Secretary Of The Navy Asynchronous bidirectional application program processes interface for a distributed heterogeneous multiprocessor system
US5502726A (en) 1992-01-31 1996-03-26 Nellcor Incorporated Serial layered medical network
CA2094410C (en) 1992-06-18 1998-05-05 Joshua Seth Auerbach Distributed management communications network
US5289460A (en) 1992-07-31 1994-02-22 International Business Machines Corp. Maintenance of message distribution trees in a communications network
US5365523A (en) 1992-11-16 1994-11-15 International Business Machines Corporation Forming and maintaining access groups at the lan/wan interface
US5481735A (en) 1992-12-28 1996-01-02 Apple Computer, Inc. Method for modifying packets that meet a particular criteria as the packets pass between two layers in a network
JPH06253308A (en) 1993-03-01 1994-09-09 Fujitsu Ltd Video communication control system
ZA943336B (en) 1993-05-19 1995-08-11 Menashe Julian Interactive computerised gaming system with remote terminals
US5511168A (en) 1993-07-01 1996-04-23 Digital Equipment Corporation Virtual circuit manager for multicast messaging
US5483664A (en) * 1993-07-26 1996-01-09 Motorola, Inc. Cellular communications with scheduled handoffs
WO1995010911A1 (en) * 1993-10-12 1995-04-20 Intel Corporation Method and system for multicasting related data streams on a computer network
AU7966594A (en) * 1993-10-12 1995-05-04 Intel Corporation Method and system for multicasting formatted data on a computer network
US5453780A (en) 1994-04-28 1995-09-26 Bell Communications Research, Inc. Continous presence video signal combiner
US5558339A (en) 1994-05-05 1996-09-24 Perlman; Stephen G. Network architecture to support recording and playback of real-time video games
WO1995031061A1 (en) 1994-05-05 1995-11-16 Catapult Entertainment, Inc. Network architecture for real-time video games
US5586257A (en) 1994-05-05 1996-12-17 Perlman; Stephen G. Network architecture to support multiple site real-time video games
US5740231A (en) * 1994-09-16 1998-04-14 Octel Communications Corporation Network-based multimedia communications and directory system and method of operation
US5805830A (en) 1994-09-30 1998-09-08 Intel Corporation Method and apparatus for managing communications with selective subsets of conference participants
US5517494A (en) * 1994-09-30 1996-05-14 Apple Computer, Inc. Method and system of multicast routing for groups with a single transmitter
US5685775A (en) 1994-10-28 1997-11-11 International Business Machines Corporation Networking video games over telephone network
US5630757A (en) 1994-11-29 1997-05-20 Net Game Limited Real-time multi-user game communication system using existing cable television infrastructure
JP3121221B2 (en) 1995-02-07 2000-12-25 株式会社日立製作所 Communication method of information processing system and information processing system
US5594732A (en) 1995-03-03 1997-01-14 Intecom, Incorporated Bridging and signalling subsystems and methods for private and hybrid communications systems including multimedia systems
US5674127A (en) 1995-03-07 1997-10-07 Habilas, Inc. Multisite multiplayer interactive electronic entertainment system having a partially player defined universe
US5581552A (en) 1995-05-23 1996-12-03 At&T Multimedia server
US5930259A (en) 1995-08-25 1999-07-27 Kabushiki Kaisha Toshiba Packet transmission node device realizing packet transfer scheme and control information transfer scheme using multiple virtual connections
US5784568A (en) 1995-08-31 1998-07-21 Intel Corporation Multi-party audio chat system which allows individual user utterances to be staged separately to render received utterances in order
US5729540A (en) 1995-10-19 1998-03-17 Qualcomm Incorporated System and method for scheduling messages on a common channel
US5684800A (en) 1995-11-15 1997-11-04 Cabletron Systems, Inc. Method for establishing restricted broadcast groups in a switched network
US5812552A (en) 1996-03-19 1998-09-22 At & T Corp Method and apparatus for dynamically forming multimedia emulated local area networks
US5778187A (en) 1996-05-09 1998-07-07 Netcast Communications Corp. Multicasting method and apparatus
US5761436A (en) 1996-07-01 1998-06-02 Sun Microsystems, Inc. Method and apparatus for combining truncated hyperlinks to form a hyperlink aggregate

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5079767A (en) * 1988-09-27 1992-01-07 Digital Equipment Corporation Method of multicast message distribution
US5309437A (en) * 1990-06-29 1994-05-03 Digital Equipment Corporation Bridge-like internet protocol router
US5309433A (en) * 1992-06-18 1994-05-03 International Business Machines Corp. Methods and apparatus for routing packets in packet transmission networks
US5329619A (en) * 1992-10-30 1994-07-12 Software Ag Cooperative processing interface and communication broker for heterogeneous computing environments
US5361256A (en) * 1992-11-27 1994-11-01 International Business Machines Corporation Inter-domain multicast routing

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7006616B1 (en) 1999-05-21 2006-02-28 Terayon Communication Systems, Inc. Teleconferencing bridge with EdgePoint mixing
US7079502B2 (en) 2000-05-16 2006-07-18 Nec Corporation Packet communication system, mobile communication system, and addressing method for communication
USRE48579E1 (en) 2002-04-15 2021-06-01 Media Ip, Inc. Method and apparatus for internet-based interactive programming
US10856031B2 (en) 2003-04-15 2020-12-01 MedialP, Inc. Method and apparatus for generating interactive programming in a communication network
US11076190B2 (en) 2003-04-15 2021-07-27 MedialP, Inc. Method and apparatus for generating interactive programming in a communication network
US11477506B2 (en) 2003-04-15 2022-10-18 MediaIP, LLC Method and apparatus for generating interactive programming in a communication network
US11575955B2 (en) 2003-04-15 2023-02-07 MediaIP, LLC Providing interactive video on demand

Also Published As

Publication number Publication date
AU1747197A (en) 1997-08-22
US5822523A (en) 1998-10-13
EP1012724A1 (en) 2000-06-28
JP2000504133A (en) 2000-04-04
US6226686B1 (en) 2001-05-01
US6018766A (en) 2000-01-25

Similar Documents

Publication Publication Date Title
US5822523A (en) Server-group messaging system for interactive applications
Aweya IP router architectures: an overview
US6553028B1 (en) Method and apparatus for multicast switching using a centralized switching engine
JP3495234B2 (en) Method and system for reducing server load on network computer system
EP0963082A2 (en) Quality of service improvement of internet real-time media transmission by transmitting redundant voice/media frames
EP1194876B1 (en) Method and apparatus in a communication network
JPH10150470A (en) Method for transferring connection management information in request and response of world wide web
Yang et al. A multimedia synchronization model and its implementation in transport protocols
US6697872B1 (en) Distributed packet processing using encapsulation and decapsulation chains
EP1146722A2 (en) Method and apparatus for providing telephony services switch-based processing of media streams
Yano et al. The breadcrumb forwarding service: A synthesis of PGM and EXPRESS to improve and simplify global IP multicast
CN1330137C (en) Method for sending address analytic protocol request message into specific object in local area network
JP4028302B2 (en) Packet relay method, relay device, and network system using the relay device
Cisco Designing Internetworks for Multimedia
US8131867B1 (en) Dynamic layer congestion control for multicast transport
Cisco Designing Internetworks for Multimedia
Cisco Designing Internetworks for Multimedia
Cisco Designing Internetworks for Multimedia
Cisco Designing Internetworks for Multimedia
JP3560559B2 (en) IP server load distribution device and its load distribution method in LAN emulation
JP2001326685A (en) Multicast transmission system and its destination database management system
JP2001320367A (en) Multicast limiting/distributing method, its device and medium recording its program
JP3805884B2 (en) ATM relay device
Broll et al. DWTP: a basis for networked VR on the Internet
Boggs et al. A Specific Internetwork Architecture (Pup)

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES FI GB GE HU IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW MX NO NZ PL PT RO RU SD SE SG SI SK TJ TM TR TT UA UG UZ VN AM AZ BY KG KZ MD RU TJ TM

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): KE LS MW SD SZ UG AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 1997904759

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

WWP Wipo information: published in national office

Ref document number: 1997904759

Country of ref document: EP

WWW Wipo information: withdrawn in national office

Ref document number: 1997904759

Country of ref document: EP