CA2395282C - Preserving consistency of passively-replicated non-deterministic objects - Google Patents
Preserving consistency of passively-replicated non-deterministic objects Download PDFInfo
- Publication number
- CA2395282C CA2395282C CA002395282A CA2395282A CA2395282C CA 2395282 C CA2395282 C CA 2395282C CA 002395282 A CA002395282 A CA 002395282A CA 2395282 A CA2395282 A CA 2395282A CA 2395282 C CA2395282 C CA 2395282C
- Authority
- CA
- Canada
- Prior art keywords
- request
- transaction
- client
- primary replica
- replicated object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1474—Saving, restoring, recovering or retrying in transactions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/202—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where processing functionality is redundant
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/2097—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements maintaining the standby controller/processing unit updated
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99951—File or database maintenance
- Y10S707/99952—Coherency, e.g. same view to multiple users
- Y10S707/99953—Recoverability
Abstract
Techniques are provided for executing an operation in which a client invokes a replicated object. According to one technique, a primary replica of a replicated object receives a first request from a client, wherein the first request includes a request identifier. Rather than immediately attempting to process the request, the primary replica determines whether a record exists that corresponds to the request identifier. If a record exists that corresponds t o the request identifier, then the primary replica responds to the first request with a reply associated with the record. If no record exis ts that corresponds to the request identifier, then the primary replica performs the steps of: starting a transaction; as part of the transaction, processing the request; as part of the transaction, storing a record associated with the request identifier and a reply to the request; committing the transaction; and delivering the reply to the client. The client may itself be a replicated object. If the client i s nondeterministic, then the transaction initiated by the primary replica may be a nested transaction relative to a transaction executed by th e client, or be executed as part of the same transaction as the client. If the transaction executed by the primary replica is executed as pa rt of the same transaction as the client, then a savepoint may be established before the primary replica processes the request, so that a failure will not necessarily require all processing performed by the client to be rolled back.
Description
PRESERVING CONSISTENCY OF PASSIVELY-REPLICATED
NON-DETERMINISTIC OBJECTS
The present invention relates to replication and, more specifically, to techniques for preserving consistency of passively-replicated non-deterministic objects.
BACKGROUND OF THE INVENTION
It has become critical that the electronic systems on which society relies remain available and consistent. With respect to availability, a single instance of unavailability may tarnish or even cripple a company that relies on electronic commerce. With respect to consistency, there is a certain class of operations that should be performed if and only if other operations are also performed. For example, in a transfer of funds between bank accounts, the withdrawal of money from one account should occur if and only if the deposit of money into the other account also occurs.
The definition of consistency must be refined in the context of highly-available mission-critical applications. Online shopping, for instance, involves critical interactions between clients and the business application on the electronic commerce site.
Unfortunately, when an error occurs during request processing (such as the failure of the client, the network, or the server), the client has generally no way to know whether his request has been processed or not.
Application servers generally provide at-most-once semantics, guaranteeing that, if the request has been processed, it has been processed only once. This quality of service is not sufficient since it puts the responsibility on the client to decide whether or not the request must be reissued.
The quality of service required by mission-critical distributed applications is exactly-once. Ideally, the client wants the guarantee that its request will eventually be processed, and that processing will happen only once. This problem is known as end-to-end reliability. The "all-or-nothing" property necessary for application consistency should ideally become "all".
End-to-end reliability can be defmed as the guarantee that a request sent by a client to a server will be eventually processed and that the client will get a reply, despite the failure of any server component in the system. The request will be processed exactly once by the server. In addition, if the client fails it can still obtain the reply after recovering.
A typical scenario is that of an end-user buying airplane tickets through an electronic-commerce site. If the site fails before the user checks out, then he will have to start his selection over, but he will not be billed for what he had selected before the failure. However, if a failure occurs after the user has confirmed his purchase but before the reception of the receipt, then he cannot usually know whether the request has been processed or not, the site has to provide journalization mechanisms that can be assessed by end users.
In this scenario, end-to-end reliability means that the user has the guarantee that his request will be eventually processed once it has been submitted, if the client application (e.g., web browser) does not fail. If the client fails, then the request can be re-issued upon recovery with the guarantee that it will not be processed twice.
This scenario can be extended to involve nested invocations. For example, end-users can plan their vacations through a site that mediates the purchase of airline tickets, car rentals, and hotel reservations. This example illustrates the use of nested invocations between applications servers: the vacations site acts as a client of other sites for booking plane tickets, cars, and hotel rooms.
This scenario shows problems that may happen with partial request execution.
Consider the case of an end-user who wants to travel to London. If the vacations planning server books a car and a hotel room in London but fails before obtaining the plane ticket, the first two reservations are meaningless. Note that, in this case, end-to-end reliability does not mean that all three reservations will succeed. It rather means that the client's request will be processed completely. The vacations planner site may for instance cancel the car and hotel reservations if all planes to London are full.
Replication is a technique that is widely used to increase the availability of systems.
In general, replication involves maintaining copies ("replicas") of a resource so that if one replica fails, another replica may be used. For example, many clients may require access to a particular database table. To increase availability, many copies of the table may be managed by many different database servers. If one of the copies of the table becomes corrupt, or if one of the database servers fails, all clients that were accessing the copy of the table that is no longer available may continue to access the table using a different copy thereof.
A replicated object is represented by a set of copies. This set may be static or dynamic. Static replication requires that the number and the identity of the copies do not change during the lifetime of the replicated object, while dynamic replication allows copies to be added or removed at runtime.
In distributed systems, the two best-known replication policies are active replication and passive replication (also called primary-backup replication). With active replication, all copies of the replicated object play the same role. Thus, when a client sends a request to an actively-replicated object, all replicas of the object receive each request, process the request, update their state in response to the request, and send a response back to the client. Because the requests are always being sent to every replica, the failure of one of the replicas is transparent to the client.
With passive replication, one replica is designated as the primary replica, while all other replicas are backups. Clients perform requests by sending messages only to the primary replica, which executes the request, updates the other replicas, and sends the response to the client. If the primary replica fails, then one of the backup replicas takes over the responsibility of being the primary replica.
The main problem with replication is that conventional replication techniques require replicated objects to be deterministic. An object is said to be deterministic if the outcome of a request issued to the object (an "invocation" of the object) depends only of the state of the object prior to the invocation and the parameters of the invocation. Thus, two deterministic objects having identical states will keep identical states if they both receive the same set of invocations in the same order.
The integrity of a system may be compromised if replicated objects are not deterministic. For example, in an active replication system, if two replicas arrive at different states based on the same input, then switching between the replicas may result in unpredictable behavior. In passive replication systems, the problem created by non-deterministic objects is less apparent, but just as troublesome, in particular when the non-deterministic objects interact with other entities.
FIG. 1 illustrates a scenario in which a client C invokes a single replicated object X, which in turn invokes another replicated object Y and then a non-replicated object Z. The set of nested invocations form an "invocation tree". In this scenario, Y is aware of replication since it is replicated itself, while Z may not be aware of replication at all.
Consequently, it may be assumed that only Y implements mechanisms for dealing with replicated invocations.
The main problem when dealing with replicated objects is to maintain the consistency of the replicated state, i.e., to ensure that all replicas agree on a common state. Consistency must be preserved at all levels of the invocation tree. For instance, in FIG.
1, it is not acceptable that Y receives and processes a request while Z does not because of the failure of X. The replication mechanisms must ensure that either all objects in the invocation tree process their request, or that none of them does. This all-or-nothing property is similar to the atomicity property of a transactional system.
One approach for guaranteeing atomicity of invocations in scenarios such as that shown FIG. 1 is referred to as the roll-forward approach. The roll-forward approach uses redundancy to ensure that another replica will transparently take over upon failure of the primary without any loss of information. Consistency is maintained by guaranteeing that the invocation will succeed despite failure (by "rolling forward"). Passive replication techniques 29-01-2002 , , US003523 that use a roll-forward approach ensure that only the primary replica processes the requests.
Updates are sent to the backup replicas. If the primary replica fails during the processing of a request, a backup replica is chosen to be the new primary replica. The request during which the failure occurred is then sent to the new primary replica, which continues processing at that point.
It is commonly believed that a passive replication technique with a roll-forward approach is sufficient to support non-deterministic servers. However, these techniques are sufficient only for failure-free case, because a non-deterministic object may interact with other objects, the identity of which depends on non-deterministic factors. For instance, in Figure 1, Xmay invoke Y if some condition is met (e.g., some timer has not yet expired) and Z otherwise. In this situation, X may crash after having invoked Y, but before having updated the backups. The backup that takes over and processes the invocation may invoke Z instead of Y and leave the system in an inconsistent state.
Unfortunately, in many real-world situations and systems, detenninisrn cannot be guaranteed. Thus, there is a need for a system and technique for providing the benefits of object replication while maintaining the accuracy of results in an environment that cannot guarantee that the replicated objects are deterministic.
Frank et al: "Semantic ACID Properties in Multidatabases Using Remote Procedure Calls and Update Propagations", Software - Practice and Experience, GB, John Wiley and Sons Ltd., Chichester, Vol. 28, no. 1, 1998, pages 77 - 98 discloses a method of providing database transactions with global Semantic ACID properties (atomicity, consistency, isolation and durability), in a client-server architecture.
US-A-5864854 discloses a method of caching data items for a group of clients using a group cache look-up table that includes an entry for each data item cached by any member of the group and a list of client identifiers for each client caching the data items.
NON-DETERMINISTIC OBJECTS
The present invention relates to replication and, more specifically, to techniques for preserving consistency of passively-replicated non-deterministic objects.
BACKGROUND OF THE INVENTION
It has become critical that the electronic systems on which society relies remain available and consistent. With respect to availability, a single instance of unavailability may tarnish or even cripple a company that relies on electronic commerce. With respect to consistency, there is a certain class of operations that should be performed if and only if other operations are also performed. For example, in a transfer of funds between bank accounts, the withdrawal of money from one account should occur if and only if the deposit of money into the other account also occurs.
The definition of consistency must be refined in the context of highly-available mission-critical applications. Online shopping, for instance, involves critical interactions between clients and the business application on the electronic commerce site.
Unfortunately, when an error occurs during request processing (such as the failure of the client, the network, or the server), the client has generally no way to know whether his request has been processed or not.
Application servers generally provide at-most-once semantics, guaranteeing that, if the request has been processed, it has been processed only once. This quality of service is not sufficient since it puts the responsibility on the client to decide whether or not the request must be reissued.
The quality of service required by mission-critical distributed applications is exactly-once. Ideally, the client wants the guarantee that its request will eventually be processed, and that processing will happen only once. This problem is known as end-to-end reliability. The "all-or-nothing" property necessary for application consistency should ideally become "all".
End-to-end reliability can be defmed as the guarantee that a request sent by a client to a server will be eventually processed and that the client will get a reply, despite the failure of any server component in the system. The request will be processed exactly once by the server. In addition, if the client fails it can still obtain the reply after recovering.
A typical scenario is that of an end-user buying airplane tickets through an electronic-commerce site. If the site fails before the user checks out, then he will have to start his selection over, but he will not be billed for what he had selected before the failure. However, if a failure occurs after the user has confirmed his purchase but before the reception of the receipt, then he cannot usually know whether the request has been processed or not, the site has to provide journalization mechanisms that can be assessed by end users.
In this scenario, end-to-end reliability means that the user has the guarantee that his request will be eventually processed once it has been submitted, if the client application (e.g., web browser) does not fail. If the client fails, then the request can be re-issued upon recovery with the guarantee that it will not be processed twice.
This scenario can be extended to involve nested invocations. For example, end-users can plan their vacations through a site that mediates the purchase of airline tickets, car rentals, and hotel reservations. This example illustrates the use of nested invocations between applications servers: the vacations site acts as a client of other sites for booking plane tickets, cars, and hotel rooms.
This scenario shows problems that may happen with partial request execution.
Consider the case of an end-user who wants to travel to London. If the vacations planning server books a car and a hotel room in London but fails before obtaining the plane ticket, the first two reservations are meaningless. Note that, in this case, end-to-end reliability does not mean that all three reservations will succeed. It rather means that the client's request will be processed completely. The vacations planner site may for instance cancel the car and hotel reservations if all planes to London are full.
Replication is a technique that is widely used to increase the availability of systems.
In general, replication involves maintaining copies ("replicas") of a resource so that if one replica fails, another replica may be used. For example, many clients may require access to a particular database table. To increase availability, many copies of the table may be managed by many different database servers. If one of the copies of the table becomes corrupt, or if one of the database servers fails, all clients that were accessing the copy of the table that is no longer available may continue to access the table using a different copy thereof.
A replicated object is represented by a set of copies. This set may be static or dynamic. Static replication requires that the number and the identity of the copies do not change during the lifetime of the replicated object, while dynamic replication allows copies to be added or removed at runtime.
In distributed systems, the two best-known replication policies are active replication and passive replication (also called primary-backup replication). With active replication, all copies of the replicated object play the same role. Thus, when a client sends a request to an actively-replicated object, all replicas of the object receive each request, process the request, update their state in response to the request, and send a response back to the client. Because the requests are always being sent to every replica, the failure of one of the replicas is transparent to the client.
With passive replication, one replica is designated as the primary replica, while all other replicas are backups. Clients perform requests by sending messages only to the primary replica, which executes the request, updates the other replicas, and sends the response to the client. If the primary replica fails, then one of the backup replicas takes over the responsibility of being the primary replica.
The main problem with replication is that conventional replication techniques require replicated objects to be deterministic. An object is said to be deterministic if the outcome of a request issued to the object (an "invocation" of the object) depends only of the state of the object prior to the invocation and the parameters of the invocation. Thus, two deterministic objects having identical states will keep identical states if they both receive the same set of invocations in the same order.
The integrity of a system may be compromised if replicated objects are not deterministic. For example, in an active replication system, if two replicas arrive at different states based on the same input, then switching between the replicas may result in unpredictable behavior. In passive replication systems, the problem created by non-deterministic objects is less apparent, but just as troublesome, in particular when the non-deterministic objects interact with other entities.
FIG. 1 illustrates a scenario in which a client C invokes a single replicated object X, which in turn invokes another replicated object Y and then a non-replicated object Z. The set of nested invocations form an "invocation tree". In this scenario, Y is aware of replication since it is replicated itself, while Z may not be aware of replication at all.
Consequently, it may be assumed that only Y implements mechanisms for dealing with replicated invocations.
The main problem when dealing with replicated objects is to maintain the consistency of the replicated state, i.e., to ensure that all replicas agree on a common state. Consistency must be preserved at all levels of the invocation tree. For instance, in FIG.
1, it is not acceptable that Y receives and processes a request while Z does not because of the failure of X. The replication mechanisms must ensure that either all objects in the invocation tree process their request, or that none of them does. This all-or-nothing property is similar to the atomicity property of a transactional system.
One approach for guaranteeing atomicity of invocations in scenarios such as that shown FIG. 1 is referred to as the roll-forward approach. The roll-forward approach uses redundancy to ensure that another replica will transparently take over upon failure of the primary without any loss of information. Consistency is maintained by guaranteeing that the invocation will succeed despite failure (by "rolling forward"). Passive replication techniques 29-01-2002 , , US003523 that use a roll-forward approach ensure that only the primary replica processes the requests.
Updates are sent to the backup replicas. If the primary replica fails during the processing of a request, a backup replica is chosen to be the new primary replica. The request during which the failure occurred is then sent to the new primary replica, which continues processing at that point.
It is commonly believed that a passive replication technique with a roll-forward approach is sufficient to support non-deterministic servers. However, these techniques are sufficient only for failure-free case, because a non-deterministic object may interact with other objects, the identity of which depends on non-deterministic factors. For instance, in Figure 1, Xmay invoke Y if some condition is met (e.g., some timer has not yet expired) and Z otherwise. In this situation, X may crash after having invoked Y, but before having updated the backups. The backup that takes over and processes the invocation may invoke Z instead of Y and leave the system in an inconsistent state.
Unfortunately, in many real-world situations and systems, detenninisrn cannot be guaranteed. Thus, there is a need for a system and technique for providing the benefits of object replication while maintaining the accuracy of results in an environment that cannot guarantee that the replicated objects are deterministic.
Frank et al: "Semantic ACID Properties in Multidatabases Using Remote Procedure Calls and Update Propagations", Software - Practice and Experience, GB, John Wiley and Sons Ltd., Chichester, Vol. 28, no. 1, 1998, pages 77 - 98 discloses a method of providing database transactions with global Semantic ACID properties (atomicity, consistency, isolation and durability), in a client-server architecture.
US-A-5864854 discloses a method of caching data items for a group of clients using a group cache look-up table that includes an entry for each data item cached by any member of the group and a list of client identifiers for each client caching the data items.
AMENDED SHEET
29-01-2002 4 , US003523( StJNCMARY OF THE TNVENTION
Techniques are provided for executing an operation in which a client invokes a replicated object. According to one technique, a primary replica of the replicated object receives a first request from the client, wherein the first request includes a request identifier.
Rather than immediately attempting to process the request, the primary replica detennines whether a record exists that corresponds to the request identifier. If a record exists that corresponds to the request identifier, then the primary replica responds to the first request with a reply-associated with the record. If no record exists that corresponds to the request identifier, then the primary replica perforrns the steps of: starting a transaction; as part of the AMENDED SHEET
transaction, processing the request; as part of the transaction, storing a record associated with the request identifier and a reply to the request; committing the transaction;
and delivering the reply to the client.
The client may itself be a replicated object. If the client is nondeterministic, then the transaction initiated by the primary replica may be a nested transaction relative to a transaction executed by the client, or be executed as part of the same transaction as the client.
If the transaction executed by the primary replica is executed as part of the same transaction as the client, then a savepoint may be established before the primary replica processes the request, so that a failure will not necessarily require all processing performed by the client to be rolled back.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
FIG. 1 is a block diagram of a system in which a client invokes a replicated object, which in turn invokes two other objects;
FIG. 2A is a flowchart of steps performed by a client according to an embodiment of the invention;
FIG. 2B is a flowchart of steps performed by the primary replica of a replicated object according to an embodiment of the invention;
FIG. 3 is a block diagram illustrating a no-failure scenario according to an embodiment of the invention;
FIG. 4 is a block diagram illustrating a scenario in which an object called with a nested invocation fails after initiating a transaction, according to an embodiment of the invention;
29-01-2002 4 , US003523( StJNCMARY OF THE TNVENTION
Techniques are provided for executing an operation in which a client invokes a replicated object. According to one technique, a primary replica of the replicated object receives a first request from the client, wherein the first request includes a request identifier.
Rather than immediately attempting to process the request, the primary replica detennines whether a record exists that corresponds to the request identifier. If a record exists that corresponds to the request identifier, then the primary replica responds to the first request with a reply-associated with the record. If no record exists that corresponds to the request identifier, then the primary replica perforrns the steps of: starting a transaction; as part of the AMENDED SHEET
transaction, processing the request; as part of the transaction, storing a record associated with the request identifier and a reply to the request; committing the transaction;
and delivering the reply to the client.
The client may itself be a replicated object. If the client is nondeterministic, then the transaction initiated by the primary replica may be a nested transaction relative to a transaction executed by the client, or be executed as part of the same transaction as the client.
If the transaction executed by the primary replica is executed as part of the same transaction as the client, then a savepoint may be established before the primary replica processes the request, so that a failure will not necessarily require all processing performed by the client to be rolled back.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
FIG. 1 is a block diagram of a system in which a client invokes a replicated object, which in turn invokes two other objects;
FIG. 2A is a flowchart of steps performed by a client according to an embodiment of the invention;
FIG. 2B is a flowchart of steps performed by the primary replica of a replicated object according to an embodiment of the invention;
FIG. 3 is a block diagram illustrating a no-failure scenario according to an embodiment of the invention;
FIG. 4 is a block diagram illustrating a scenario in which an object called with a nested invocation fails after initiating a transaction, according to an embodiment of the invention;
FIG. 5 is a block diagram illustrating a scenario in which an object called with a nested invocation fails after committing a transaction, according to an embodiment of the invention;
FIG. 6 is a block diagram illustrating a scenario in which an object makes a nested invocation and fails after the object invoked in the nested transaction commits the transaction, according to an embodiment of the invention;
FIG. 7 is a block diagram illustrating a scenario in which an object makes a nested invocation and fails after the object invoked in the nested transaction commits the transaction, according to an embodiment of the invention; and FIG. 8 is a block diagram of a computer system upon which embodiments of the invention may be implemented according to an embodiment of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
A method and apparatus for executing an operation in which a client invokes a replicated object is described. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuiing the present invention.
FUNCTIONAL OVERVIEW
Techniques are provided for performing passive replication using a "roll-backwards"
approach that does not require determinism from replicated objects. The techniques can be applied, for example, in the context of distributed middleware to provide transparent replication and fail-over. The technique is based on transactions, and has different properties than the traditional approaches that are based on group cornmunication and roll-forward techniques.
TRANSACTIONS
A transaction is a sequence of operations on objects or data items that satisfies the following for properties (ACID properties):
Atomicity: all the operations of the transaction are performed, or none of them is performed.
Consistency: a transaction must be correct, by transforming the data from one consistent state to another.
Isolation: when several transactions are executed concurrently, each of them must appear as if it is executed alone.
Durability: despite crashes, the effect of the performed operations is permanent.
If a failure occurs in the system or the transaction can not be committed for some reason, it will roll back, i.e., all modifications performed up to this point will be undone and the system will revert to its previous state. Some systems support nested transactions, where transactions can be started in the scope of another transaction. If the nested transaction fails, the enclosing transaction does not automatically roll back and the application can try to correct the problem and retry the transaction. If the enclosing transaction rolls back, then all nested transactions also roll back.
TRANSACTION-BASED PASSIVE REPLICATION
Techniques are provided for performing transaction-based passive replication in a manner that achieves accurate results even when a failure occurs in a non-deterministic replicated object. For the purpose of illustration, the techniques shall be described with reference to a system that includes an invocation tree that involves three entities. The first entity in the invocation tree shall be referred to as the client, or "C". The second entity in the -~-invocation tree (the entity called by the client) shall be referred to as object X, or "X". The third entity in the invocation tree (the entity called by X) shall be referred to as object Y, or Y,~
It should be noted that the designations given to these entities are merely for the purpose of explanation, and do not in any way limit the nature of the entities involved.
Further, the techniques are in no way limited to invocation trees that involve three entities.
However, a three-entity invocation tree is sufficient to illustrate how all entities in an invocation tree may respond relative to the entity that invokes them and the entities that they invoke.
Figure 2A is a flowchart that illustrates the steps performed by a client C in a transaction-based passive replication system according to one embodiment of the invention.
Referring to Figure 2A, at step 202, the client C invokes the primary replica of object X. The primary replica of object X shall be referred to herein as X1. After invoking X1, client C
waits for a response from X1. If X1 does not respond within a predetelmined amount of time (step 204), client C assumes that XI has failed, and selects a new primary replica (step 206).
Client C then invokes the new primary replica (202). This sequence is repeated until one of the replicas of object X successfully responds to the invocation within the specified amount of time.
Figure 2B is a flowchart that illustrates the steps performed by the primary replica of object X in a transaction-based passive replication system according to one embodiment of the invention. Referring to Figure 2B, at step 250, the primary replica of object X (i.e. X1) receives the request from client C. At step 252, X1 determines whether the received request is identical to a request that was already processed by object X. According to one embodiment, this determination is made by (1) reading a request identifier from the request, and (2) comparing the request identifier against request identifiers contained in records stored in a log, where the log contains records of all replies that have been committed by all replicas of object X. If the request identifier in the request matches the request identifier in a log record, then Xl responds to the request by returning the reply indicated by the log record (step 258).
If the request identifier in the request does not match the request identifier in any log record, then control passes to step 254. At step 254, Xl begins a transaction TX. After beginning the transaction, X1 performs, as part of the transaction, all of the operations required to service the request. For the purpose of illustration, it shall be assumed that one of those operations involves invoking another replicated object Y. Thus, at step 256, Xl sends a request to the primary replica of object Y(Yl). At step 260, X1 receives a reply from Y1.
At step 262, Xl logs the reply from Yl. At step 264, after all of the operations required by the request have been performed by X1, X1 commits the transaction. After committing the transaction, X1 sends the reply to the request to C.
It should be noted that, relative to Y1, X1 is a client. Consequently, when X1 sends a request to Yl, X1 performs the steps shown in FIG. 2A. Specifically, Xl waits for a response. If the response does not come within a predetermined period of time from the transmission of the request to Yl, then Xl selects a different replica of Y to be the primary replica, and sends the request to the new primary replica of Y. This process is repeated until Xl successfully receives a reply from one of the replicas of Y.
As shall be illustrated by the scenarios described hereafter, in a system where clients and primary replicas are configured to operate as shown in Figures 2A and 2B, respectively, the clients receive correct responses even when failures occur in non-deterministic replicated obj ects.
NO-FAILURE SCENARIO
Figure 3 is a block diagram that illustrates the sequence of events when no error occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. Referring to Figure 3, the client C
invokes the primary replica of X(X 1). The invocation includes a request identifier (rid l).
Upon receiving the invocation, X1 performs a lookup operation to determine whether a log contains a record associated with ridl. In the present example, C has not previously issued the request, so the log will not contain any record associated with ridl.
Upon determining that the log does not have a record for ridl, X1 begins a transaction TX and then processes the request as part of the transaction. As part of processing the request, Xl invokes the primary replica of Y (Yl). Yl The invocation of Yl includes a request identifier (rid2). Upon receiving the invocation, Yl performs a lookup operation to determine whether a log contains a record associated with rid2. In the present example, X1 has not previously issued the request, so the log will not contain any record associated with rid2.
Upon determining that the log does not have a record for rid2, Y1 begins a transaction and then processes the request as part of the transaction. According to one embodiment, the transaction started by Yl is nested within the transaction started by Xl.
Consequently, the transaction started by Yl can be rolled back without rolling back the entire transaction initiated by Xl. Further, the transaction started by Y1 will be automatically rolled back if the transaction initiated by Xl is rolled back.
For the purpose of illustration, it shall be assumed that YI does not invoke any other entity during the processing of the request. After processing the request, Yl logs a record for rid2 that includes the reply to the request associated with rid2. Y1 then commits the nested transaction, and sends the reply to Xl.
After receiving the reply from Yl, Xl logs a record for ridl that includes the reply to the request associated with ridl. Xl then commits the transaction, and sends its reply to C.
The nested transaction initiated by Yl is committed in response to the commission of the transaction initiated by X1.
FAILURE OF Yl BEFORE Yl COMMITS
Figure 4 is a block diagram that illustrates the sequence of events when a particular type of error occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. The particular error illustrated in Figure 4 is the failure of Yl before Yl commits its transaction.
Consequently, the sequence of events is identical to the no-failure scenario shown in Figure 3 up to the point where Yl is processing the request from Xl.
Because Y1 has crashed, X1 will not receive a reply from Yl within the predetermined time period. However, because the operations performed by Y1 where part of a nested transaction, they are rolled back without rolling back the transaction started by Xl.
The ability to continue without rolling back the transaction started by Xl may be significant, for example, if Xl had perforrned extensive work as part of that transaction prior to calling Yl.
Consequently, X1 selects a new primary replica of Y ("Y2") and resends the request (containing rid2) to Y2. Upon receiving the invocation, Y2 performs a lookup operation to determine whether a log contains a record associated with rid2. In the present example, the log will not contain any record associated with rid2 because any log record generated by Yl for rid2 would have been generated as part of a transaction that never committed. ' Consequently, when that transaction is rolled back, any such record would have been removed from the log. Similarly, the effects of any operations performed by Y1 in response to the request associated with rid2 that were made prior to the failure of Y1 will also have been undone.
Upon determining that the log does not have a record for rid2, Y2 begins a transaction and then processes the request as part of the transaction. For the purpose of illustration, it shall be assumed that Y2 does not invoke any other entity during the processing of the request. After processing the request, Y21ogs a record for rid2 that includes the reply to the request associated with rid2. Y2 then commits the transaction, and sends the reply to Xl.
After receiving the reply from Y1, X1 logs a record for ridl that includes the reply to the request associated with ridl. Xl then commits the transaction, and sends its reply to C.
Figure 5 is a block diagram that illustrates the sequence of events when a particular type of error occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. The particular error illustrated in Figure 5 is the failure of Y1 after YI commits its transaction, but before Y1 sends a reply to Xl. Consequently, the sequence of events is identical to the no-failure scenario shown in Figure 3 up to the point where Yl replies to the request from Xl.
Because Y1 has crashed prior to replying to Xl, Xl will not receive a reply from Y1 within the predetermined time period. Consequently, X1 selects Y2 to be the new primary replica of Y and resends the request (containing rid2) to Y2. Upon receiving the invocation, Y2 performs a lookup operation to determine whether a log contains a record associated with rid2. In the present example, the log will contain a record associated with rid2 because a record associated with rid2 was logged by Y1 in a transaction that committed.
The effects of all operations performed by Yl in response to the request associated with rid2 that were made prior to the failure of Y1 will also have been committed.
Upon determining that the log does have a record for rid2, Y2 reads the reply from the record and transmits the reply to Xl in response to the request from Xl.
Y2 performs no further processing in response to the request from Xl. After receiving the reply from Y2, Xl logs a record for ridl that includes the reply to the request associated with ridl. X1 then commits the transaction, and sends its reply to C.
FAILURE OF Xl BEFORE COMMIT AND AFTER INVOKING Yl, WHERE X IS DETERMINISTIC
Figure 6 is a block diagram that illustrates the sequence of events when a particular type of error occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. The particular error illustrated in Figure 6 is the failure of X1 after invoking Yl but before Xl commits its transaction. Consequently, the sequence of events is similar to the no-failure scenario shown in Figure 3 up to the point where Y1 commits its transaction associated with TX. However, unlike the process shown in Figure 3, none of the transactions that are started are nested.
Because Xl has crashed, C will not receive a reply from Xl within the predetermined time period. Consequently, C selects a new primary replica of X ("X2") and resends the request (containing ridl) to X2. Upon receiving the invocation, X2 performs a lookup operation to determine whether a log contains a record associated with ridl.
In the present example, the log will not contain any record associated with ridl because any log record generated by Xl for i7d1 would have been generated as part of a transaction that never committed. Consequently, when that transaction is rolled back, any such record would have been removed from the log. Similarly, the effects of any operations performed by Xl in response to the request associated with ridl that were made prior to the failure of Xl will also have been undone.
Upon determining that the log does not have a record for ridl, X2 begins a transaction and then processes the request as part of the transaction. During the processing of the request, X2 invokes Y1 with a request that includes the request identifier rid2. Upon receiving the invocation, Yl performs a lookup operation to determine whether a log contains a record associated with rid2. In the present example, the log will contain a record associated with rid2 because a record associated with rid2 was logged by Y1 in a transaction that committed before X1 failed. The effects of all operations performed by Yl in response to the request associated with rid2 that were made prior to the failure of Y1 will also have been committed.
Upon deterrnining that the log does have a record for rid2, Yl reads the reply from the record and transmits the reply to X2 in response to the request from X2.
Yl performs no further processing in response to the request from X2. After receiving the reply from Y1, X2 logs a record for ridl that includes the reply to the request associated with rid1. X1 then commits the transaction, and sends its reply to C.
The sequence of events illustrated in Figure 6 assumes that object X is deterministic.
Because object X is deterministic, object Y is able to commit its changes without regard to whether object X is able to successfully commit. Because object X is deterministic, object Y
can be assured that even if the current primary replica of object X fails to commit, a subsequent primary replica of object X will (1) send the same request to object Y and (2) will commit its transaction. If object X is not deterministic, then a nested transaction is used, as shall now be described with reference to Figure 7.
FAILURE OF Xl BEFORE COMMIT AND AFTER INVOKING Yl, WHERE X IS NOT DETERMINISTIC
Figure 7 is a block diagram that illustrates the sequence of events when a particular type of elTor occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. The particular error illustrated in Figure 7 is the failure of Xl after invoking Y1 but before X1 commits its transaction, where X is a non-deter7ninistic object. The sequence of events is identical to the no-failure scenario shown in Figure 3 prior to the point where Yl commits its transaction.
However, the transaction started by Y1 is a nested transaction within the transaction TX1 executing within Xl, that caused the invocation of Yl. Consequently, the nested transaction started by Yl will be rolled back if TX1 is rolled back.
After completing all processing, Y1 comniits, and sends a reply to X1. Because has crashed, C will not receive a reply from Xl within the predetermined time peiiod.
Consequently, C selects a new primary replica of X ("X2") and resends the request (containing ridl) to X2. Upon receiving the invocation, X2 performs a lookup operation to determine whether a log contains a record associated with ridl. In the present example, the log will not contain any record associated with ridl because any log record generated by X1 for ridl would have been generated as part of transaction TXl that never committed.
Consequently, when TX1 is rolled back, any such record would have been removed from the log. Similarly, the effects of any operations performed by X1 in response to the request associated with ridl that were made prior to the failure of Xl will also have been undone, and the effects of any operations performed by Yl in response to the request associated with rid2 will also have been undone.
Upon determining that the log does not have a record for ridl, X2 begins a new transaction TX2 and then processes the request as part of the transaction.
During the processing of the request, X2 may or may not invoke Y1. Specifically, because object X is non-deterministic, the factors that caused X1 to invoke YI may have changed, thereby causing X2 to process the same request from C without invoking Y. In the scenario illustrated in Figure 7, X2 processes the request from C without invoking Y1.
After performing the operations required to process the request, X2 logs a record for ridl that includes the reply to the request associated with ridl. Xl then commits TX2, and sends its reply to C.
Because the operations performed by Y1 were part of a nested transaction within TX1, and TX1 did not commit, the operations perfoimed by Y1 will be rolled back.
Consequently, even though X2 did not invoke Yl after the crash of Xl, the system was returned to a consistent state.
SAVEPOINTS
In the examples illustrated above, a nested transaction is used to perform operations in objects called by non-detenninisitic objects to guarantee that (1) nested invocations will be rolled back if the transactions in which they are nested fail, and (2) nested invocations can be rolled back if they fail without rolling back the transaction in which they are nested.
However, other mechanisms may be used instead of or in addition to nested transactions to achieve this result. For example, nested transactions can be replaced by mechanisms like savepoints, with the transaction context being propagated in nested invocations.
Savepoints are intermediate markers used to specify locations within transactions. If an error occurs during the processing of a transaction that includes one or more savepoints, then the transaction is rolled back to the last savepoint in the transaction, rather than all the way to the beginning of the transaction.
In an embodiment that uses a savepoint, a calling object may set a savepoint prior to invoking another object, or the called object may set a savepoint prior to performing any work in response to the call. All operations performed by the called object in response to the call are performed as part of that same transaction that is being executed by the calling object. If the primary replica of the called object fails, then the transaction is rolled back to the savepoint prior to the call. The calling object may then re-issue the invocation to a newly selected primary replica of the called object without rolling back all operations performed by the calling object that were made prior to the savepoint.
Embodiments may implement the techniques described herein without employing savepoints or nested transactions. In embodiments that do not use nested transactions or savepoints, operations performed by X and Y may be performed as part of a single transaction. Under these circumstances, a failure of the primary replica of either X or Y
would cause all changes made by both X and Y to roll back. This may result in the need to re-execute some of the operations that were rolled back, where such re-execution may have been avoided through the use of nested transactions or savepoints.
ASSUMPTIONS AND VARIATIONS
The techniques described above assume that all replicas of a replicated object can access the same log. The log may be, for example, a shared database that is used for logging messages. The shared database itself may be replicated for high availability.
In addition, the techniques assume that one of the following is true for each object Y (whether or not replicated) that is invoked by a replica Xthat uses this technique (next object in the invocation path):
Y uses the same technique and supports nested transactions (or similar mechanisms, such as save points).
Yuses the same technique and supports flat transactions. Xmust be deterministic if Y
induces any state change in its invocation sub-tree.
Y is deterministic and idempotent or appears so (e.g., it discards duplicates and always returns the same reply). X must deterministic if Y induces any state change in its invocation sub-tree.
In the description given above, it was stated that the calling object selects a new primary replica of the called object when the current primary replica of the called object fails.
However, selection of a new primary replica may be performed in a variety of ways. For example, in one embodiment, the remaining replicas may negotiate among themselves which replica is to be the primary replica. If a calling object invokes a replica other than the new primary replica, the invoked replica may simply respond by sending the calling object data that identifies the new primary replica. The calling object may then invoke the correct new primary replica.
The techniques described herein may be combined with various other passive replication techniques. For example, any number of well-known techniques may be used to select a new primary replica in response to a failure, and to communicate the state of a failed replica to the new primary replica. Similarly, any number of techniques may be used to generate and log records, and to provide the replicas of a replicated object shared access to the log.
HARDWARE OVERVIEW
Figure 8 is a block diagram that illustrates a computer system 800 upon which an embodiment of the invention may be implemented. Computer system 800 includes a bus 802 or other communication mechanism for communicating information, and a processor 804 coupled with bus 802 for processing information. Computer system 800 also includes a main memory 806, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 802 for storing infoimation and instructions to be executed by processor 804.
Main memory 806 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 804.
Computer system 800 further includes a read only memory (ROM) 808 or other static storage device coupled to bus 802 for storing static information and instructions for processor 804. A-storage device 810, such as a magnetic disk or optical disk, is provided and coupled to bus 802 for storing information and instructions.
Computer system 800 may be coupled via bus 802 to a display 812, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 814, including alphanumeric and other keys, is coupled to bus 802 for communicating information and command selections to processor 804. Another type of user input device is cursor contro1816, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 804 and for controlling cursor movement on display 812. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
The invention is related to the use of computer system 800 for implementing the techniques described herein. According to one embodiment of the invention, those techniques are implemented by computer system 800 in response to processor 804 executing one or more sequences of one or more instructions contained in main memory 806. Such instructions may be read into main memory 806 from another computer-readable medium, such as storage device 810. Execution of the sequences of instructions contained in main memory 806 causes processor.804 to perform the process steps described herein.
In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to processor 804 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 810. Volatile media includes dynamic memory, such as main memory 806. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 802. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 804 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer.
The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 800 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 802. Bus 802 carries the data to main memory 806, from which processor 804 retrieves and executes the instructions. The instructions received by main memory 806 may optionally be stored on storage device 810 either before or after execution by processor 804.
Computer system 800 also includes a communication interface 818 coupled to bus 802. Communication interface 818 provides a two-way data communication coupling to a network link 820 that is connected to a local network 822. For example, communication interface 818 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 818 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 818 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
Network link 820 typically provides data communication through one or more networks to other data devices. For example, network link 820 may provide a connection through local network 822 to a host computer 824 or to data equipment operated by an Internet Service Provider (ISP) 826. ISP 826 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the "Internet" 828. Local network 822 and Internet 828 both use electrical, electromagnetic or optical signals that caiTy digital data streams. The signals through the various networks and the signals on network link 820 and through communication interface 818, which carry the digital data to and from computer system 800, are exemplary forms of carrier waves transporting the information.
Computer system 800 can send messages and receive data, including program code, through the network(s), network link 820 and communication interface 818. In the Internet example, a server 830 might transmit a requested code for an application program through Internet 828, ISP 826, local network 822 and communication interface 818. In accordance with the invention, one such downloaded application implements the techniques described herein.
The received code may be executed by processor 804 as it is received, andlor stored in storage device 810, or other non-volatile storage for later execution. In this manner, computer system 800 may obtain application code in the form of a carrier wave.
In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
FIG. 6 is a block diagram illustrating a scenario in which an object makes a nested invocation and fails after the object invoked in the nested transaction commits the transaction, according to an embodiment of the invention;
FIG. 7 is a block diagram illustrating a scenario in which an object makes a nested invocation and fails after the object invoked in the nested transaction commits the transaction, according to an embodiment of the invention; and FIG. 8 is a block diagram of a computer system upon which embodiments of the invention may be implemented according to an embodiment of the invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
A method and apparatus for executing an operation in which a client invokes a replicated object is described. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuiing the present invention.
FUNCTIONAL OVERVIEW
Techniques are provided for performing passive replication using a "roll-backwards"
approach that does not require determinism from replicated objects. The techniques can be applied, for example, in the context of distributed middleware to provide transparent replication and fail-over. The technique is based on transactions, and has different properties than the traditional approaches that are based on group cornmunication and roll-forward techniques.
TRANSACTIONS
A transaction is a sequence of operations on objects or data items that satisfies the following for properties (ACID properties):
Atomicity: all the operations of the transaction are performed, or none of them is performed.
Consistency: a transaction must be correct, by transforming the data from one consistent state to another.
Isolation: when several transactions are executed concurrently, each of them must appear as if it is executed alone.
Durability: despite crashes, the effect of the performed operations is permanent.
If a failure occurs in the system or the transaction can not be committed for some reason, it will roll back, i.e., all modifications performed up to this point will be undone and the system will revert to its previous state. Some systems support nested transactions, where transactions can be started in the scope of another transaction. If the nested transaction fails, the enclosing transaction does not automatically roll back and the application can try to correct the problem and retry the transaction. If the enclosing transaction rolls back, then all nested transactions also roll back.
TRANSACTION-BASED PASSIVE REPLICATION
Techniques are provided for performing transaction-based passive replication in a manner that achieves accurate results even when a failure occurs in a non-deterministic replicated object. For the purpose of illustration, the techniques shall be described with reference to a system that includes an invocation tree that involves three entities. The first entity in the invocation tree shall be referred to as the client, or "C". The second entity in the -~-invocation tree (the entity called by the client) shall be referred to as object X, or "X". The third entity in the invocation tree (the entity called by X) shall be referred to as object Y, or Y,~
It should be noted that the designations given to these entities are merely for the purpose of explanation, and do not in any way limit the nature of the entities involved.
Further, the techniques are in no way limited to invocation trees that involve three entities.
However, a three-entity invocation tree is sufficient to illustrate how all entities in an invocation tree may respond relative to the entity that invokes them and the entities that they invoke.
Figure 2A is a flowchart that illustrates the steps performed by a client C in a transaction-based passive replication system according to one embodiment of the invention.
Referring to Figure 2A, at step 202, the client C invokes the primary replica of object X. The primary replica of object X shall be referred to herein as X1. After invoking X1, client C
waits for a response from X1. If X1 does not respond within a predetelmined amount of time (step 204), client C assumes that XI has failed, and selects a new primary replica (step 206).
Client C then invokes the new primary replica (202). This sequence is repeated until one of the replicas of object X successfully responds to the invocation within the specified amount of time.
Figure 2B is a flowchart that illustrates the steps performed by the primary replica of object X in a transaction-based passive replication system according to one embodiment of the invention. Referring to Figure 2B, at step 250, the primary replica of object X (i.e. X1) receives the request from client C. At step 252, X1 determines whether the received request is identical to a request that was already processed by object X. According to one embodiment, this determination is made by (1) reading a request identifier from the request, and (2) comparing the request identifier against request identifiers contained in records stored in a log, where the log contains records of all replies that have been committed by all replicas of object X. If the request identifier in the request matches the request identifier in a log record, then Xl responds to the request by returning the reply indicated by the log record (step 258).
If the request identifier in the request does not match the request identifier in any log record, then control passes to step 254. At step 254, Xl begins a transaction TX. After beginning the transaction, X1 performs, as part of the transaction, all of the operations required to service the request. For the purpose of illustration, it shall be assumed that one of those operations involves invoking another replicated object Y. Thus, at step 256, Xl sends a request to the primary replica of object Y(Yl). At step 260, X1 receives a reply from Y1.
At step 262, Xl logs the reply from Yl. At step 264, after all of the operations required by the request have been performed by X1, X1 commits the transaction. After committing the transaction, X1 sends the reply to the request to C.
It should be noted that, relative to Y1, X1 is a client. Consequently, when X1 sends a request to Yl, X1 performs the steps shown in FIG. 2A. Specifically, Xl waits for a response. If the response does not come within a predetermined period of time from the transmission of the request to Yl, then Xl selects a different replica of Y to be the primary replica, and sends the request to the new primary replica of Y. This process is repeated until Xl successfully receives a reply from one of the replicas of Y.
As shall be illustrated by the scenarios described hereafter, in a system where clients and primary replicas are configured to operate as shown in Figures 2A and 2B, respectively, the clients receive correct responses even when failures occur in non-deterministic replicated obj ects.
NO-FAILURE SCENARIO
Figure 3 is a block diagram that illustrates the sequence of events when no error occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. Referring to Figure 3, the client C
invokes the primary replica of X(X 1). The invocation includes a request identifier (rid l).
Upon receiving the invocation, X1 performs a lookup operation to determine whether a log contains a record associated with ridl. In the present example, C has not previously issued the request, so the log will not contain any record associated with ridl.
Upon determining that the log does not have a record for ridl, X1 begins a transaction TX and then processes the request as part of the transaction. As part of processing the request, Xl invokes the primary replica of Y (Yl). Yl The invocation of Yl includes a request identifier (rid2). Upon receiving the invocation, Yl performs a lookup operation to determine whether a log contains a record associated with rid2. In the present example, X1 has not previously issued the request, so the log will not contain any record associated with rid2.
Upon determining that the log does not have a record for rid2, Y1 begins a transaction and then processes the request as part of the transaction. According to one embodiment, the transaction started by Yl is nested within the transaction started by Xl.
Consequently, the transaction started by Yl can be rolled back without rolling back the entire transaction initiated by Xl. Further, the transaction started by Y1 will be automatically rolled back if the transaction initiated by Xl is rolled back.
For the purpose of illustration, it shall be assumed that YI does not invoke any other entity during the processing of the request. After processing the request, Yl logs a record for rid2 that includes the reply to the request associated with rid2. Y1 then commits the nested transaction, and sends the reply to Xl.
After receiving the reply from Yl, Xl logs a record for ridl that includes the reply to the request associated with ridl. Xl then commits the transaction, and sends its reply to C.
The nested transaction initiated by Yl is committed in response to the commission of the transaction initiated by X1.
FAILURE OF Yl BEFORE Yl COMMITS
Figure 4 is a block diagram that illustrates the sequence of events when a particular type of error occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. The particular error illustrated in Figure 4 is the failure of Yl before Yl commits its transaction.
Consequently, the sequence of events is identical to the no-failure scenario shown in Figure 3 up to the point where Yl is processing the request from Xl.
Because Y1 has crashed, X1 will not receive a reply from Yl within the predetermined time period. However, because the operations performed by Y1 where part of a nested transaction, they are rolled back without rolling back the transaction started by Xl.
The ability to continue without rolling back the transaction started by Xl may be significant, for example, if Xl had perforrned extensive work as part of that transaction prior to calling Yl.
Consequently, X1 selects a new primary replica of Y ("Y2") and resends the request (containing rid2) to Y2. Upon receiving the invocation, Y2 performs a lookup operation to determine whether a log contains a record associated with rid2. In the present example, the log will not contain any record associated with rid2 because any log record generated by Yl for rid2 would have been generated as part of a transaction that never committed. ' Consequently, when that transaction is rolled back, any such record would have been removed from the log. Similarly, the effects of any operations performed by Y1 in response to the request associated with rid2 that were made prior to the failure of Y1 will also have been undone.
Upon determining that the log does not have a record for rid2, Y2 begins a transaction and then processes the request as part of the transaction. For the purpose of illustration, it shall be assumed that Y2 does not invoke any other entity during the processing of the request. After processing the request, Y21ogs a record for rid2 that includes the reply to the request associated with rid2. Y2 then commits the transaction, and sends the reply to Xl.
After receiving the reply from Y1, X1 logs a record for ridl that includes the reply to the request associated with ridl. Xl then commits the transaction, and sends its reply to C.
Figure 5 is a block diagram that illustrates the sequence of events when a particular type of error occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. The particular error illustrated in Figure 5 is the failure of Y1 after YI commits its transaction, but before Y1 sends a reply to Xl. Consequently, the sequence of events is identical to the no-failure scenario shown in Figure 3 up to the point where Yl replies to the request from Xl.
Because Y1 has crashed prior to replying to Xl, Xl will not receive a reply from Y1 within the predetermined time period. Consequently, X1 selects Y2 to be the new primary replica of Y and resends the request (containing rid2) to Y2. Upon receiving the invocation, Y2 performs a lookup operation to determine whether a log contains a record associated with rid2. In the present example, the log will contain a record associated with rid2 because a record associated with rid2 was logged by Y1 in a transaction that committed.
The effects of all operations performed by Yl in response to the request associated with rid2 that were made prior to the failure of Y1 will also have been committed.
Upon determining that the log does have a record for rid2, Y2 reads the reply from the record and transmits the reply to Xl in response to the request from Xl.
Y2 performs no further processing in response to the request from Xl. After receiving the reply from Y2, Xl logs a record for ridl that includes the reply to the request associated with ridl. X1 then commits the transaction, and sends its reply to C.
FAILURE OF Xl BEFORE COMMIT AND AFTER INVOKING Yl, WHERE X IS DETERMINISTIC
Figure 6 is a block diagram that illustrates the sequence of events when a particular type of error occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. The particular error illustrated in Figure 6 is the failure of X1 after invoking Yl but before Xl commits its transaction. Consequently, the sequence of events is similar to the no-failure scenario shown in Figure 3 up to the point where Y1 commits its transaction associated with TX. However, unlike the process shown in Figure 3, none of the transactions that are started are nested.
Because Xl has crashed, C will not receive a reply from Xl within the predetermined time period. Consequently, C selects a new primary replica of X ("X2") and resends the request (containing ridl) to X2. Upon receiving the invocation, X2 performs a lookup operation to determine whether a log contains a record associated with ridl.
In the present example, the log will not contain any record associated with ridl because any log record generated by Xl for i7d1 would have been generated as part of a transaction that never committed. Consequently, when that transaction is rolled back, any such record would have been removed from the log. Similarly, the effects of any operations performed by Xl in response to the request associated with ridl that were made prior to the failure of Xl will also have been undone.
Upon determining that the log does not have a record for ridl, X2 begins a transaction and then processes the request as part of the transaction. During the processing of the request, X2 invokes Y1 with a request that includes the request identifier rid2. Upon receiving the invocation, Yl performs a lookup operation to determine whether a log contains a record associated with rid2. In the present example, the log will contain a record associated with rid2 because a record associated with rid2 was logged by Y1 in a transaction that committed before X1 failed. The effects of all operations performed by Yl in response to the request associated with rid2 that were made prior to the failure of Y1 will also have been committed.
Upon deterrnining that the log does have a record for rid2, Yl reads the reply from the record and transmits the reply to X2 in response to the request from X2.
Yl performs no further processing in response to the request from X2. After receiving the reply from Y1, X2 logs a record for ridl that includes the reply to the request associated with rid1. X1 then commits the transaction, and sends its reply to C.
The sequence of events illustrated in Figure 6 assumes that object X is deterministic.
Because object X is deterministic, object Y is able to commit its changes without regard to whether object X is able to successfully commit. Because object X is deterministic, object Y
can be assured that even if the current primary replica of object X fails to commit, a subsequent primary replica of object X will (1) send the same request to object Y and (2) will commit its transaction. If object X is not deterministic, then a nested transaction is used, as shall now be described with reference to Figure 7.
FAILURE OF Xl BEFORE COMMIT AND AFTER INVOKING Yl, WHERE X IS NOT DETERMINISTIC
Figure 7 is a block diagram that illustrates the sequence of events when a particular type of elTor occurs in a three-entity invocation tree, where the clients and primary replicas are configured to operate as described in Figures 2A and 2B, respectively. The particular error illustrated in Figure 7 is the failure of Xl after invoking Y1 but before X1 commits its transaction, where X is a non-deter7ninistic object. The sequence of events is identical to the no-failure scenario shown in Figure 3 prior to the point where Yl commits its transaction.
However, the transaction started by Y1 is a nested transaction within the transaction TX1 executing within Xl, that caused the invocation of Yl. Consequently, the nested transaction started by Yl will be rolled back if TX1 is rolled back.
After completing all processing, Y1 comniits, and sends a reply to X1. Because has crashed, C will not receive a reply from Xl within the predetermined time peiiod.
Consequently, C selects a new primary replica of X ("X2") and resends the request (containing ridl) to X2. Upon receiving the invocation, X2 performs a lookup operation to determine whether a log contains a record associated with ridl. In the present example, the log will not contain any record associated with ridl because any log record generated by X1 for ridl would have been generated as part of transaction TXl that never committed.
Consequently, when TX1 is rolled back, any such record would have been removed from the log. Similarly, the effects of any operations performed by X1 in response to the request associated with ridl that were made prior to the failure of Xl will also have been undone, and the effects of any operations performed by Yl in response to the request associated with rid2 will also have been undone.
Upon determining that the log does not have a record for ridl, X2 begins a new transaction TX2 and then processes the request as part of the transaction.
During the processing of the request, X2 may or may not invoke Y1. Specifically, because object X is non-deterministic, the factors that caused X1 to invoke YI may have changed, thereby causing X2 to process the same request from C without invoking Y. In the scenario illustrated in Figure 7, X2 processes the request from C without invoking Y1.
After performing the operations required to process the request, X2 logs a record for ridl that includes the reply to the request associated with ridl. Xl then commits TX2, and sends its reply to C.
Because the operations performed by Y1 were part of a nested transaction within TX1, and TX1 did not commit, the operations perfoimed by Y1 will be rolled back.
Consequently, even though X2 did not invoke Yl after the crash of Xl, the system was returned to a consistent state.
SAVEPOINTS
In the examples illustrated above, a nested transaction is used to perform operations in objects called by non-detenninisitic objects to guarantee that (1) nested invocations will be rolled back if the transactions in which they are nested fail, and (2) nested invocations can be rolled back if they fail without rolling back the transaction in which they are nested.
However, other mechanisms may be used instead of or in addition to nested transactions to achieve this result. For example, nested transactions can be replaced by mechanisms like savepoints, with the transaction context being propagated in nested invocations.
Savepoints are intermediate markers used to specify locations within transactions. If an error occurs during the processing of a transaction that includes one or more savepoints, then the transaction is rolled back to the last savepoint in the transaction, rather than all the way to the beginning of the transaction.
In an embodiment that uses a savepoint, a calling object may set a savepoint prior to invoking another object, or the called object may set a savepoint prior to performing any work in response to the call. All operations performed by the called object in response to the call are performed as part of that same transaction that is being executed by the calling object. If the primary replica of the called object fails, then the transaction is rolled back to the savepoint prior to the call. The calling object may then re-issue the invocation to a newly selected primary replica of the called object without rolling back all operations performed by the calling object that were made prior to the savepoint.
Embodiments may implement the techniques described herein without employing savepoints or nested transactions. In embodiments that do not use nested transactions or savepoints, operations performed by X and Y may be performed as part of a single transaction. Under these circumstances, a failure of the primary replica of either X or Y
would cause all changes made by both X and Y to roll back. This may result in the need to re-execute some of the operations that were rolled back, where such re-execution may have been avoided through the use of nested transactions or savepoints.
ASSUMPTIONS AND VARIATIONS
The techniques described above assume that all replicas of a replicated object can access the same log. The log may be, for example, a shared database that is used for logging messages. The shared database itself may be replicated for high availability.
In addition, the techniques assume that one of the following is true for each object Y (whether or not replicated) that is invoked by a replica Xthat uses this technique (next object in the invocation path):
Y uses the same technique and supports nested transactions (or similar mechanisms, such as save points).
Yuses the same technique and supports flat transactions. Xmust be deterministic if Y
induces any state change in its invocation sub-tree.
Y is deterministic and idempotent or appears so (e.g., it discards duplicates and always returns the same reply). X must deterministic if Y induces any state change in its invocation sub-tree.
In the description given above, it was stated that the calling object selects a new primary replica of the called object when the current primary replica of the called object fails.
However, selection of a new primary replica may be performed in a variety of ways. For example, in one embodiment, the remaining replicas may negotiate among themselves which replica is to be the primary replica. If a calling object invokes a replica other than the new primary replica, the invoked replica may simply respond by sending the calling object data that identifies the new primary replica. The calling object may then invoke the correct new primary replica.
The techniques described herein may be combined with various other passive replication techniques. For example, any number of well-known techniques may be used to select a new primary replica in response to a failure, and to communicate the state of a failed replica to the new primary replica. Similarly, any number of techniques may be used to generate and log records, and to provide the replicas of a replicated object shared access to the log.
HARDWARE OVERVIEW
Figure 8 is a block diagram that illustrates a computer system 800 upon which an embodiment of the invention may be implemented. Computer system 800 includes a bus 802 or other communication mechanism for communicating information, and a processor 804 coupled with bus 802 for processing information. Computer system 800 also includes a main memory 806, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 802 for storing infoimation and instructions to be executed by processor 804.
Main memory 806 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 804.
Computer system 800 further includes a read only memory (ROM) 808 or other static storage device coupled to bus 802 for storing static information and instructions for processor 804. A-storage device 810, such as a magnetic disk or optical disk, is provided and coupled to bus 802 for storing information and instructions.
Computer system 800 may be coupled via bus 802 to a display 812, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 814, including alphanumeric and other keys, is coupled to bus 802 for communicating information and command selections to processor 804. Another type of user input device is cursor contro1816, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 804 and for controlling cursor movement on display 812. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
The invention is related to the use of computer system 800 for implementing the techniques described herein. According to one embodiment of the invention, those techniques are implemented by computer system 800 in response to processor 804 executing one or more sequences of one or more instructions contained in main memory 806. Such instructions may be read into main memory 806 from another computer-readable medium, such as storage device 810. Execution of the sequences of instructions contained in main memory 806 causes processor.804 to perform the process steps described herein.
In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to processor 804 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 810. Volatile media includes dynamic memory, such as main memory 806. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 802. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 804 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer.
The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 800 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 802. Bus 802 carries the data to main memory 806, from which processor 804 retrieves and executes the instructions. The instructions received by main memory 806 may optionally be stored on storage device 810 either before or after execution by processor 804.
Computer system 800 also includes a communication interface 818 coupled to bus 802. Communication interface 818 provides a two-way data communication coupling to a network link 820 that is connected to a local network 822. For example, communication interface 818 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 818 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 818 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
Network link 820 typically provides data communication through one or more networks to other data devices. For example, network link 820 may provide a connection through local network 822 to a host computer 824 or to data equipment operated by an Internet Service Provider (ISP) 826. ISP 826 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the "Internet" 828. Local network 822 and Internet 828 both use electrical, electromagnetic or optical signals that caiTy digital data streams. The signals through the various networks and the signals on network link 820 and through communication interface 818, which carry the digital data to and from computer system 800, are exemplary forms of carrier waves transporting the information.
Computer system 800 can send messages and receive data, including program code, through the network(s), network link 820 and communication interface 818. In the Internet example, a server 830 might transmit a requested code for an application program through Internet 828, ISP 826, local network 822 and communication interface 818. In accordance with the invention, one such downloaded application implements the techniques described herein.
The received code may be executed by processor 804 as it is received, andlor stored in storage device 810, or other non-volatile storage for later execution. In this manner, computer system 800 may obtain application code in the form of a carrier wave.
In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
Claims (27)
1. A method for executing an operation in which a client invokes a replicated object, the method comprising the steps of:
receiving, at a primary replica of said replicated object, a first request from said client, wherein said first request includes a request identifier;
determining whether a record exists that corresponds to the request identifier;
if a record exists that corresponds to the request identifier, then said primary replica responding to said first request with a reply associated with said record;
if no record exists that corresponds to the request identifier, then the primary replica performing the steps of starting a transaction;
as part of the transaction, processing said first request;
as part of the transaction, storing a record associated with the request identifier and a reply to the first request;
committing said transaction; and delivering said reply to said client.
receiving, at a primary replica of said replicated object, a first request from said client, wherein said first request includes a request identifier;
determining whether a record exists that corresponds to the request identifier;
if a record exists that corresponds to the request identifier, then said primary replica responding to said first request with a reply associated with said record;
if no record exists that corresponds to the request identifier, then the primary replica performing the steps of starting a transaction;
as part of the transaction, processing said first request;
as part of the transaction, storing a record associated with the request identifier and a reply to the first request;
committing said transaction; and delivering said reply to said client.
2. The method of claim 1, wherein the step of processing said first request includes executing logic that is non-deterministic.
3. The method of claim 1 wherein the step of processing said first request includes the steps of:
sending a second request to a server; and receiving a second reply from said server in response to said second request.
sending a second request to a server; and receiving a second reply from said server in response to said second request.
4. The method of claim 3 wherein:
the server is a second replicated object;
the step of sending the second request to the server includes repeatedly performing the following steps until a reply is received from a replica of the second replicated object:
sending the second request to a primary replica of the second replicated object wherein the primary replica of the second replicated object is one of a plurality of replicas of the second replicated object; and if the primary replica of the second replicated object does not respond within a predetermined period, then substituting a replica of the second replicated object other than said primary replica of the second replicated object as a new primary replica.
the server is a second replicated object;
the step of sending the second request to the server includes repeatedly performing the following steps until a reply is received from a replica of the second replicated object:
sending the second request to a primary replica of the second replicated object wherein the primary replica of the second replicated object is one of a plurality of replicas of the second replicated object; and if the primary replica of the second replicated object does not respond within a predetermined period, then substituting a replica of the second replicated object other than said primary replica of the second replicated object as a new primary replica.
5. The method of claim 2 wherein the step of processing said first request includes the step of causing a primary replica of a second replicated object to perform an operation within a nested transaction to said transaction.
6. The method of claim 1 wherein:
the client is deterministic; and the step of starting a transaction is performed by starting a transaction that is not nested within or part of any transaction executing on said client.
the client is deterministic; and the step of starting a transaction is performed by starting a transaction that is not nested within or part of any transaction executing on said client.
7. The method of claim 1 wherein:
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the step of starting a transaction is performed by starting a transaction that is part of the transaction executing on the client.
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the step of starting a transaction is performed by starting a transaction that is part of the transaction executing on the client.
8. The method of claim 7 further comprising the step of setting a savepoint before the primary replica performs the step of processing said first request.
9. The method of claim 1 wherein:
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the step of starting a transaction is performed by starting a transaction that is nested within the transaction executing on the client.
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the step of starting a transaction is performed by starting a transaction that is nested within the transaction executing on the client.
10. A computer-readable medium bearing instructions for executing an operation in which a client invokes a replicated object, the instructions including instructions for performing the steps of:
receiving, at a primary replica of said replicated object, a first request from said client, wherein said first request includes a request identifier;
determining whether a record exists that corresponds to the request identifier;
if a record exists that corresponds to the request identifier, then said primary replica responding to said first request with a reply associated with said record;
if no record exists that corresponds to the request identifier, then the primary replica performing the steps of starting a transaction;
as part of the transaction, processing said first request;
as part of the transaction, storing a record associated with the request identifier and a reply to the first request;
committing said transaction; and delivering said reply to said client.
receiving, at a primary replica of said replicated object, a first request from said client, wherein said first request includes a request identifier;
determining whether a record exists that corresponds to the request identifier;
if a record exists that corresponds to the request identifier, then said primary replica responding to said first request with a reply associated with said record;
if no record exists that corresponds to the request identifier, then the primary replica performing the steps of starting a transaction;
as part of the transaction, processing said first request;
as part of the transaction, storing a record associated with the request identifier and a reply to the first request;
committing said transaction; and delivering said reply to said client.
11. The computer-readable medium of claim 10, wherein the step of processing said first request includes executing logic that is non-deterministic.
12. The computer-readable medium of claim 11 wherein the step of processing said first request includes the step causing a primary replica of a second replicated object to perform an operation within a nested transaction to said transaction.
13. The computer-readable medium of claim 10 wherein the step of processing said first request includes the steps of:
sending a second request to a server; and receiving a second reply from said server in response to said second request.
sending a second request to a server; and receiving a second reply from said server in response to said second request.
14. The computer-readable medium of claim 13 wherein:
the server is a second replicated object;
the step of sending the second request to the server includes repeatedly performing the following steps until a reply is received from a replica of the second replicated object:
sending the second request to a primary replica of the second replicated object wherein the primary replica of the second replicated object is one of a plurality of replicas of the second replicated object; and if the primary replica of the second replicated object does not respond within a predetermined period, then substituting a replica of the second replicated object other than said primary replica of the second replicated object as a new primary replica.
the server is a second replicated object;
the step of sending the second request to the server includes repeatedly performing the following steps until a reply is received from a replica of the second replicated object:
sending the second request to a primary replica of the second replicated object wherein the primary replica of the second replicated object is one of a plurality of replicas of the second replicated object; and if the primary replica of the second replicated object does not respond within a predetermined period, then substituting a replica of the second replicated object other than said primary replica of the second replicated object as a new primary replica.
15. The computer-readable medium of claim 10 wherein:
the client is deterministic; and the step of starting a transaction is performed by starting a transaction that is not nested within or part of any transaction executing on said client.
the client is deterministic; and the step of starting a transaction is performed by starting a transaction that is not nested within or part of any transaction executing on said client.
16. The computer-readable medium of claim 10 wherein:
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the step of starting a transaction is performed by starting a transaction that is part of the transaction executing on the client.
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the step of starting a transaction is performed by starting a transaction that is part of the transaction executing on the client.
17. The computer-readable medium of claim 16 further comprising instructions for performing the step of setting a savepoint before the primary replica performs the step of processing said first request.
18. The computer-readable medium of claim 10 wherein:
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the step of starting a transaction is performed by starting a transaction that is nested within the transaction executing on the client.
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the step of starting a transaction is performed by starting a transaction that is nested within the transaction executing on the client.
19. A system for executing an operation, the system comprising:
a client configured to send a first request to a replicated object, wherein said first request includes a request identifier;
a primary replica of said replicated object that is configured to respond to the first request by performing the following steps:
determining whether a record exists that corresponds to the request identifier;
if a record exists that corresponds to the request identifier, then said primary replica responding to said first request with a reply associated with said record;
if no record exists that corresponds to the request identifier, then the primary replica performing the steps of starting a transaction;
as part of the transaction, processing said first request;
as part of the transaction, storing a record associated with the request identifier and a reply to the first request;
committing said transaction; and delivering said reply to said client.
a client configured to send a first request to a replicated object, wherein said first request includes a request identifier;
a primary replica of said replicated object that is configured to respond to the first request by performing the following steps:
determining whether a record exists that corresponds to the request identifier;
if a record exists that corresponds to the request identifier, then said primary replica responding to said first request with a reply associated with said record;
if no record exists that corresponds to the request identifier, then the primary replica performing the steps of starting a transaction;
as part of the transaction, processing said first request;
as part of the transaction, storing a record associated with the request identifier and a reply to the first request;
committing said transaction; and delivering said reply to said client.
20. The system of claim 19, wherein the primary replica is configured to process said first request by executing logic that is non-deterministic.
21. The system of claim 20 wherein a primary replica of a second replicated object is configured to respond to a second request by performing an operation within a nested transaction to said transaction.
22. The system of claim 19 further including:
a server;
wherein the primary replica is configured to process the first request by performing actions that include:
sending a second request to said server; and receiving a second reply from said server in response to said second request.
a server;
wherein the primary replica is configured to process the first request by performing actions that include:
sending a second request to said server; and receiving a second reply from said server in response to said second request.
23. The system of claim 22 wherein:
the server is a second replicated object;
the primary replica is configured to repeatedly perform the following steps until a reply is received from a replica of the second replicated object:
sending the second request to the primary replica of the second replicated object wherein the primary replica of the second replicated object is one of a plurality of replicas of the second replicated object; and if the primary replica of the second replicated object does not respond within a predetermined period, then substituting a replica of the second replicated object other than said primary replica of the second replicated object as a new primary replica of the second replicated object.
the server is a second replicated object;
the primary replica is configured to repeatedly perform the following steps until a reply is received from a replica of the second replicated object:
sending the second request to the primary replica of the second replicated object wherein the primary replica of the second replicated object is one of a plurality of replicas of the second replicated object; and if the primary replica of the second replicated object does not respond within a predetermined period, then substituting a replica of the second replicated object other than said primary replica of the second replicated object as a new primary replica of the second replicated object.
24. The system of claim 19 wherein:
the client is deterministic; and the primary replica is configured to start said transaction by starting a transaction that is not nested within or part of any transaction executing on said client.
the client is deterministic; and the primary replica is configured to start said transaction by starting a transaction that is not nested within or part of any transaction executing on said client.
25. The system of claim 19 wherein:
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the primary replica is configured to start said transaction by starting a transaction that is part of the transaction executing on the client.
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the primary replica is configured to start said transaction by starting a transaction that is part of the transaction executing on the client.
26. The system of claim 25 wherein a savepoint is set before the primary replica processes said first request.
27. The system of claim 19 wherein:
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the primary replica is configured to start the transaction by starting a transaction that is nested within the transaction executing on the client.
the client is non-deterministic;
the first request is sent to the primary replica as part of a transaction executing on the client; and the primary replica is configured to start the transaction by starting a transaction that is nested within the transaction executing on the client.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US09/478,946 | 2000-01-06 | ||
| US09/478,946 US6574750B1 (en) | 2000-01-06 | 2000-01-06 | Preserving consistency of passively-replicated non-deterministic objects |
| PCT/US2000/035230 WO2001050264A1 (en) | 2000-01-06 | 2000-12-21 | Preserving consistency of passively-replicated non-deterministic objects |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2395282A1 CA2395282A1 (en) | 2001-07-12 |
| CA2395282C true CA2395282C (en) | 2009-05-05 |
Family
ID=23902036
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002395282A Expired - Lifetime CA2395282C (en) | 2000-01-06 | 2000-12-21 | Preserving consistency of passively-replicated non-deterministic objects |
Country Status (8)
| Country | Link |
|---|---|
| US (1) | US6574750B1 (en) |
| EP (1) | EP1247182B1 (en) |
| AT (1) | ATE242896T1 (en) |
| AU (1) | AU771514B2 (en) |
| CA (1) | CA2395282C (en) |
| DE (1) | DE60003339T2 (en) |
| HK (1) | HK1046455B (en) |
| WO (1) | WO2001050264A1 (en) |
Families Citing this family (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7363325B2 (en) * | 2000-08-10 | 2008-04-22 | Nec Laboratories America, Inc. | Synchronizable transactional database method and system |
| US6922792B2 (en) * | 2000-10-27 | 2005-07-26 | Eternal Systems, Inc. | Fault tolerance for computer programs that operate over a communication network |
| US9659292B1 (en) * | 2001-08-30 | 2017-05-23 | EMC IP Holding Company LLC | Storage-based replication of e-commerce transactions in real time |
| EP1495414B1 (en) * | 2002-03-25 | 2019-06-12 | Open Invention Network LLC | Transparent consistent active replication of multithreaded application programs |
| GB0324961D0 (en) * | 2003-10-25 | 2003-11-26 | Ibm | Invocation of a follow on unit of work |
| US7188273B2 (en) | 2003-11-24 | 2007-03-06 | Tsx Inc. | System and method for failover |
| US7634566B2 (en) * | 2004-06-03 | 2009-12-15 | Cisco Technology, Inc. | Arrangement in a network for passing control of distributed data between network nodes for optimized client access based on locality |
| US7203871B2 (en) | 2004-06-03 | 2007-04-10 | Cisco Technology, Inc. | Arrangement in a network node for secure storage and retrieval of encoded data distributed among multiple network nodes |
| WO2006079623A1 (en) * | 2005-01-28 | 2006-08-03 | International Business Machines Corporation | Method for counting instructions for logging and replay of a deterministic sequence of events |
| US7730286B2 (en) * | 2005-12-30 | 2010-06-01 | Intel Corporation | Software assisted nested hardware transactions |
| US7725764B2 (en) * | 2006-08-04 | 2010-05-25 | Tsx Inc. | Failover system and method |
| EP2350876A2 (en) * | 2008-10-03 | 2011-08-03 | Telefonaktiebolaget LM Ericsson (publ) | Monitoring mechanism for a distributed database |
| US20120072604A1 (en) * | 2009-05-29 | 2012-03-22 | France Telecom | technique for delivering content to a user |
| WO2012050562A1 (en) * | 2010-10-12 | 2012-04-19 | Hewlett-Packard Development Company, L.P. | Transaction processing system and method |
| US9600315B2 (en) * | 2010-10-22 | 2017-03-21 | Netapp, Inc. | Seamless takeover of a stateful protocol session in a virtual machine environment |
| FR3030805B1 (en) * | 2014-12-19 | 2016-12-23 | Thales Sa | QUALITY OF SERVICE OF A FLIGHT MANAGEMENT SYSTEM |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5864854A (en) | 1996-01-05 | 1999-01-26 | Lsi Logic Corporation | System and method for maintaining a shared cache look-up table |
| US5963915A (en) * | 1996-02-21 | 1999-10-05 | Infoseek Corporation | Secure, convenient and efficient system and method of performing trans-internet purchase transactions |
| US6286011B1 (en) * | 1997-04-30 | 2001-09-04 | Bellsouth Corporation | System and method for recording transactions using a chronological list superimposed on an indexed list |
| US6490610B1 (en) * | 1997-05-30 | 2002-12-03 | Oracle Corporation | Automatic failover for clients accessing a resource through a server |
| US5958004A (en) * | 1997-10-28 | 1999-09-28 | Microsoft Corporation | Disabling and enabling transaction committal in transactional application components |
| US6223215B1 (en) * | 1998-09-22 | 2001-04-24 | Sony Corporation | Tracking a user's purchases on the internet by associating the user with an inbound source and a session identifier |
| US6457065B1 (en) * | 1999-01-05 | 2002-09-24 | International Business Machines Corporation | Transaction-scoped replication for distributed object systems |
| JP2000207266A (en) * | 1999-01-13 | 2000-07-28 | Mitsubishi Electric Corp | Replica system and replica method |
| US6446048B1 (en) * | 1999-09-03 | 2002-09-03 | Intuit, Inc. | Web-based entry of financial transaction information and subsequent download of such information |
-
2000
- 2000-01-06 US US09/478,946 patent/US6574750B1/en not_active Expired - Lifetime
- 2000-12-21 WO PCT/US2000/035230 patent/WO2001050264A1/en active IP Right Grant
- 2000-12-21 AT AT00986735T patent/ATE242896T1/en not_active IP Right Cessation
- 2000-12-21 DE DE60003339T patent/DE60003339T2/en not_active Expired - Lifetime
- 2000-12-21 AU AU22917/01A patent/AU771514B2/en not_active Expired
- 2000-12-21 CA CA002395282A patent/CA2395282C/en not_active Expired - Lifetime
- 2000-12-21 EP EP00986735A patent/EP1247182B1/en not_active Expired - Lifetime
-
2002
- 2002-10-26 HK HK02107773.6A patent/HK1046455B/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| DE60003339T2 (en) | 2004-04-29 |
| ATE242896T1 (en) | 2003-06-15 |
| EP1247182B1 (en) | 2003-06-11 |
| US6574750B1 (en) | 2003-06-03 |
| AU771514B2 (en) | 2004-03-25 |
| HK1046455B (en) | 2003-09-11 |
| DE60003339D1 (en) | 2003-07-17 |
| CA2395282A1 (en) | 2001-07-12 |
| EP1247182A1 (en) | 2002-10-09 |
| WO2001050264A1 (en) | 2001-07-12 |
| AU2291701A (en) | 2001-07-16 |
| HK1046455A1 (en) | 2003-01-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2395282C (en) | Preserving consistency of passively-replicated non-deterministic objects | |
| EP1649397B1 (en) | One-phase commit in a shared-nothing database system | |
| Vogels | Eventually consistent | |
| US5884327A (en) | System, method and program for performing two-phase commit with a coordinator that performs no logging | |
| US6845384B2 (en) | One-phase commit in a shared-nothing database system | |
| US7640249B2 (en) | System and method for transactional session management | |
| Felber et al. | Experiences, strategies, and challenges in building fault-tolerant CORBA systems | |
| US6510421B1 (en) | Performing 2-phase commit with presumed prepare | |
| Levandoski et al. | Deuteronomy: Transaction support for cloud data | |
| CN107070919B (en) | Idempotency for database transactions | |
| US20020035590A1 (en) | Guaranteed end-to-end transaction execution in a client/server environment | |
| US7089244B2 (en) | Multiversion read-commit order concurrency control | |
| CA2189307C (en) | Method of commitment in a distributed database transaction | |
| US6442552B1 (en) | Method and apparatus for implementing three tier client asynchronous transparency | |
| US6493726B1 (en) | Performing 2-phase commit with delayed forget | |
| US20070288555A1 (en) | System and method for transaction processing with delegated commit feature | |
| US6381617B1 (en) | Multiple database client transparency system and method therefor | |
| US20070078911A1 (en) | Replicating data across the nodes in a cluster environment | |
| US11681683B2 (en) | Transaction compensation for single phase resources | |
| US7051028B2 (en) | Concurrency control in high performance database systems | |
| US6202079B1 (en) | Synchronization procedure in a routing node | |
| Felber et al. | Reconciling replication and transactions for the end-to-end reliability of CORBA applications | |
| CN112579620A (en) | Message queue-based distributed system data final consistency method | |
| US20030191984A1 (en) | Data processing arrangement and method | |
| US7716523B2 (en) | End-to-end transactional protection for requests in a web application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKEX | Expiry |
Effective date: 20201221 |