WO1996017300A1

WO1996017300A1 - Method for transforming a hash bucket number to a control interval to identify the physical location of information in a mass memory

Info

Publication number: WO1996017300A1
Application number: PCT/US1995/013254
Authority: WO
Inventors: Arnold S. Lipton; Mariam P. Sanford; David A. Egolf; David W. Wagner; Todd B. Kneisel; Michael L. Giroux
Original assignee: Bull Hn Information Systems Inc.
Priority date: 1994-11-30
Filing date: 1995-10-19
Publication date: 1996-06-06
Also published as: DE69536076D1; EP0799451A4; EP0799451A1; EP0799451B1; US5579501A

Abstract

A method for addressing mass memory (4) in which information is stored in control intervals of physically contiguous disk segments subject to irregularities in the mapping is disclosed. Such irregularities may include discontinuities at some regular interval, which may or may not be 2**n, and/or offset from zero with respect to a virtual address employed by a user. Within the method, a unique hashing algorithm is employed to convert a virtual address to a physical address taking into account such irregularities in the mapping. This algorithm is particularly characterized by its use of integer binary arithmetic which results in high speed and complete accuracy. For the special and common condition in which discontinuities appearing at some regular interval of 2**n, a similar disclosed algorithm may be employed to achieve even greater speed of address transformation.

Description

METHOD FOR TRANSFORMING A HASH BUCKET NUMBER TO A CONTROL INTERVAL TO IDENTIFY THE PHYSICAL LOCATION OF INFORMATION IN A MASS MEMORY

Field of the Invention This invention relates to the art of data storage and retrieval and, more particularly, to the transformation of a hash bucket number to a physical address on a storage medium during such storage and retrieval. Background of the Invention As is well known in the art, the fundamental concept of a hash is to provide a repeatable mathematical algorithm that will accept a key input value which is processed by a special algoritlim into a numeric output value in a known range of numbers. The numeric output value is known as a hash bucket number and, in the data processing art, is used as an identifier to describe the memory physical location(s) in which data records are to be stored or retrieved from. Thus, within a process for storing and retrieving data records, once a data record has been placed in memory (typically, a disk memory) using a hashing algoritlim as a step in the storage process, then access to the data record can be obtained by reuse of the hashing algoritlim to locate the record in the counterpart retrieval process. For the algorithms, hash bucket numbers are logically contiguous numbers which can range from 1 to n. Control interval (hereinafter "CI") numbers are physically contiguous disk segments also numbered from 1 to n. Irregularities in the mapping may exist as follows: 1) The first bucket (bucket number 0) may not start at CI number 0. 2) There may be multiple buckets per CI. 3)There may be multiple CIs per bucket. 4) Discontinuities may exist in the CI range at some regular interval. For example, a disk organization scheme may place special CIs at regular intervals to efficiently manage disk space usage. Therefore, when mapping a hash bucket to a CI, these discontinuities must be dealt with by a hashing algorithm. Further, those skilled in the art will appreciate that a hashing algoritlim must by absolutely accurate and also higlily efficient and fast in order that the transformation process can proceed very quickly and not significantly lower system performance. Objects of the Invention It is therefore a broad object of this invention to provide an improved virtual-to-real address translation capability in a computer system. It is a more specific object of this invention to provide such address translation capability which employs a unique hashing process. Still more particularly, it is an object of this invention to provide such address translation capability by employing a unique hashing process that requires only the use of integer aritlur-etic. In another aspect, it is an object of this invention to provide such address translation capability that is highly efficient and absolutely accurate. Summary of the Invention Briefly, these and other objects of the invention are achieved, in a computer system including a user interface, a processor and a mass memory which stores information in physically contiguous memory location segments which are interrupted by discontinuities at a regular interval, by a hashing process for identifying the physical locations in the mass memory to be accessed by the user interface. For each set of information to be stored in the mass memory, a set of characteristics are established which include at least the regular interval at which discontinuities appear and any offset from the first physical location in the relevant subdivision of the mass memory at which the first record of the information set is to be stored. A virtual identification specified from the user interface is either converted to, or already is, a hash bucket# which can be transformed by the processor to a physical address in the mass memory using the special hashing algorithms disclosed. These algorithms can operate in integer binary arithmetic and are thus very fast and accurate. A first algoritlim, which can handle any interval, includes the steps: 1) obtaining a value "MIN" by employing the equation: MIN = BUCKET . /(INTER VAL - 1 ); 2) obtaining a value "CLOSE" by employing the equation: CLOSE = (MIN * INTERVAL) + START_CI; 3) obtaining a value "DIST" by employing the equation: DIST = INTERVAL - (CLOSE MOD INTERVAL) in which the term "CLOSE MOD INTERVAL" means the remainder of CLOSE divided by INTERVAL; 4) making a YES/NO decision by making the determination: (BUCKET# - (MIN * (INTERVAL - 1))) < DIST?; 5) if the decision in sub-step 4) is YES, obtaining the CI# by employing the equation: CI# = START_CI + BUCKETS + MIN; 6) if the decision in sub-step 4) is NO, obtaining the CI# by employing the equation: CI# = START_CI + BUCKET# + MIN + 1. Once the physical location in the mass memory been developed by the hashing algorithm, the communications path between the user interface and the physical location can be established. For the common configuration in which discontinuities in the mass memory occur at regular intervals of 2ⁿ, a similar, but faster hashing algorithm may be employed. This version includes the steps: 1) obtaining a value "BCI#" by employing the equation: BCI# = BUCKET#; 2) obtaining a trial value "CI# " by employing the equation: CI# = BCI# + START_CI; 3) obtaining a revised interval value "INTERVAL" by employing the equation: INTERVAL = INTERVAL - 1 ; 4) obtaining a revised value "BCI# " by employing the equation: BCI# = BCI# - ((START_CI + INTERVAL) & -INTERVAL) - START_CI where the operator "&" is a logical AND and the operator "~" is a logical NOT; 5) making a YES/NO decision by making the determination: BCI# >= 0?; 6) if the decision in sub-step 5) is YES, obtaining the CI# by employing the equation: CI = CI# + (BCI#/INTER VAL) + 1 ; 7) if the decision in sub-step 5) is NO, accepting the CI# obtained in sub-step 2) as the final CI# For environments in which there may be multiple CIs per bucket# or multiple bucket#s per CI, a PER-IND value may be assigned and relevant preprocessing performed at the beginning of the algorithm used. Description of the Drawings The subject matter of the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, may best be understood by reference to the following description taken in conjunction with the subjoined claims and the accompanying drawing of which: FIG. 1 is a generalized view of a typical computer system environment in which the invention may be practiced; FIG. 2 is a high level block diagram of a server component of the computer system; 1 FIG. 3 is a high level flow chart illustrating the process steps of

2 storing information into a mass memory and accessing previously

3 stored information, both utilizing hashing techniques;

4 FIG. 4 is a flow chart of a hashing process according to the

5 present invention; and

6 FIG. 5 is a flow chart of a special condition of the process

7 shown in FIG. 4.

8 Description of the Preferred Embodiment(s)

9 Attention is first directed to FIG. 1 which illustrates an lo exemplary environment in which the invention may be practiced. A l i work station 1 is coupled by communications channel 2 (which may,

12 for example, be a local area network) to a server 3. The server 3 may

13 be a mainframe computer, a minicomputer or even a powerful

14 workstation. The server 3 is also coupled to a mass memory 4 (which

15 may, for example, be a large disk unit) by communications channel 5.

16 During normal operation, a user at the workstation 1 may wish

17 to access a large database stored in the memory 4. Using the database

18 interface appearing on the workstation screen and the workstation

19 keyboard and/or a pointing device such as a mouse, the user can 0 virtually specify the information it is desired to access, and this virtual 1 address and an access request will be transmitted to the server 3. 2 Referring also to the high level flow chart shown in FIG. 3, 3 assume now that hashing techniques are used in the exemplary system 4 and that access to an information record already stored in the mass 5 memory 4 is sought. Either in the workstation 1 or in the server 3, 6 depending upon the system software and other factors, the virtual 1 specification of the location of the desired information will be

2 converted, if necessary, to one of a series of sequentially contiguous

3 hash bucket numbers or virtual addresses in preparation for accessing

4 the memory. However, for the reasons discussed above, this virtual

5 address (the hash bucket niunber) may still not be, and usually is not,

6 the physical address of the desired information as it is actually stored in

7 the memory 4. The hash bucket number, therefore, must be

8 transformed to a physical address before the actual memory access can

9 be performed. According to the present invention, this transformation 10 is effected by a unique hashing algorithm which runs in the server 3 in l i the example.

12 Similarly, if it is desired to store a new block of information into

13 mass memory, a hash bucket number will be selected and used to

14 develop a CI number using the same hashing algoritlim to develop a

15 physical address in which the information is stored. Thereafter, the

16 selected hash bucket number is used to virtually identify that specific

17 block of information. Typically, the selected hash bucket number is the

18 next sequentially available; i.e., previously unused.

19 FIG. 2 is a high level block diagram of an exemplary server 3 0 which may include one or more central processing units 6 interfacing to i a system bus 7 via individual caches 8. An I/O unit 9 also interfaces to 2 the system bus 7 and includes one or more ports 10. The workstation 1 3 is coupled to the server 3 at one of the ports 10, and those skilled in the 4 art will understand that other, even many other, workstations may be 5 similarly coupled to the server 3. The mass memory 4, in the example, 6 communicates with the server 3 via a system control unit 1 1 which also 1 interfaces to the system bus 7. Other server units which may be

2 present in a given system are represented by the block 12.

3 Thus, a request for access to the mass memory 4 received by the

4 server 3 from the workstation 1 proceeds through a port 10 of the I/O

5 unit 9 to the system bus 7 and to a CPU 6 via a cache 8. In the CPU,

6 the access request is interpreted and the hashing process is instituted to

7 transform the specified hash bucket number to the physical address on

8 the memory 4. The program routine for running the hashing algorithm

9 may either be resident in the cache 8 or itself called by the CPU from 10 memory 4. When the address transformation has been completed, the l i CPU issues a request to the memory 4, via the system bus 7 and SCU

12 11, which includes the physical address of the desired block of

13 information. The memory sends the requested information to the

14 server 3 which directs it to the requesting workstation 1.

15 Consider now, with reference to FIG. 4, the basic algorithm

16 which is constitutes a key aspect of the present invention. Hash bucket

17 numbers are logically contiguous numbers which can range from 1 to n. is CI numbers are physically contiguous disk segments also numbered

19 from 1 to n. Example 1 below illustrates an exemplary CI layout with

20 discontinuities every 512 CIs and the corresponding bucket-to-CI

21 mapping. For this example, it is assumed that there is one bucket per

22 CI, that there are 512 buckets in the hash and that bucket 0 starts at CI

23 4. Although an algoritlim to efficiently and quickly map a bucket

24 number to a CI number may appear quite simple upon original

25 consideration, it is in fact not at all obvious because of the need to treat the discontinuities and because of the fact that Bucket 0 may start at any given CI. EXAMPLE 1 CI O CI 1 CI 2 CI 3 CI 4 = bucket 0 CI 5 = bucket 1 CI 6 = bucket 2

CI 511 = bucket 507 CI 512 [reserved] CI 513 = bucket 508 CI 514 = bucket 509 CI 515 = bucket 510 CI 516 = bucket 511 CI 517 = bucket 512 CI 518 The process charted in FIG. 4 is a general purpose algoritlim which will map a bucket number to a CI number where, as in the present example, discontinuities may exist at any regular interval and bucket 0 may be placed at any CI number. It will be noted that all calculations are perfoπned using integer arithmetic which contributes very significantly to the efficiency and speed of the address transformation process.

INPUTS:

START_CI the CI number of bucket number 0. INTERVAL the interval of intervening discontinuities. BUCKET# the bucket number to be mapped to a

CI. PER IND - an indicator set as follows:

0 = 1 bucket per CI

1 = multiple buckets per CI

2 = multiple CIs per bucket.

PER -if PER ND = 0 this field is not used.

- if PER_IND = 1 this field is the number of buckets per CI.

- if PERJND = 2 this field is the number of CIs per bucket.

OUTPUTS:

CI# the CI number where the desired bucket is located. INTERNAL VARIABLES:

MIN - used to hold intermediate values in the calculation related to the number of intervening discontinuities. CLOSE - used to hold intermediate values in the calculation related to the output CI#. 1 DIST - used to hold intermediate values in the

2 calculation related to the number of CIs to the next

3 discontinuity.

4 For the example, then:

5 START_CI = 4

6 INTERVAL = 512

7 PERJND = 0

8 PER = [not used in the example]

9 Assume, as an example only, that information already stored in 10 the memory location virtually specified by bucket# 1000 is to be n accessed. Therefore, progressing through the steps in the algorithm:

12 Equation 1 MIN = 1000/51 1 = 1

13 Equation 2 CLOSE = 512 + 4 = 516

14 Equation 3 DIST = 512 - 4 = 508

15 Equation 4 DECISION ANSWER IS YES; i.e., (1000 - 511) <

16 508

17 Equation 5 CI# = 4 + 1000 + 1 =1005

18 Because integer arithmetic is used, more particularly, binary

19 integer arithmetic, the result of Equation 1 is "1 ". In Equation 3, those 0 skilled in the teπninology of mathematics will understand that the teπΗ i "CLOSE MOD INTERVAL" means the remainder of CLOSE divided 2 by INTERVAL which is "4" in the example. 3 EXAMPLE 2 4 The algoritlim shown in FIG. 5 is a special case of the algoritlim 5 shown in FIG. 4 in which the discontinuity interval is at every 2ⁿ CIs., 6 where n >= 1. It has been found that when the interval is limited to some power of two, the algorithm can be made even more efficient. Again, it should be noted that all calculations are performed using integer arithmetic. The inputs and outputs are the same as discussed above for the algorithm of FIG. 4. However, the only internal variable used is BCI# which is used to hold intermediate values in the calculation. Assume the same starting conditions as previously set for Example 1. Thus: Equation 6 BCI# = 1000 Equation 7 CI# = 1000 + 4 =1004 Equation 8 INTERVAL = 512 -1 = 511 Equation 9 BCI# = 1000 - (515 AND NOT511) - 4 = 484 Equation 10 DECISION ANSWER IS YES Equation 11 CI# = 1004 + (0 ) + 1 = 1005 The parenthetical component I Equation 9 may require brief explanation. The symbol "&" means a logical AND, and the symbol "~" means a logical NOT. Assuming for example only, sixteen-bit integer binary arithmetic: 515 = 0000001000000011 511 = 0000000111111111 NOT511 = 1111111000000000 0000001000000011 AND 1111111000000000 0000001000000000 =512 Thus, 1000 - 512 -4 = 484 For each of the algorithms discussed above and shown in FIGs. 4 and 4, similar exercises can be run in which PERJND = 1 and 2. While the algorithms have been discussed in the context of a hash bucket-to-CI transformation, it will be understood that these same algorithms may also be used in other domains where a similar type mapping is required. In addition, while the environment of the invention has been disclosed as that of a workstation in communication with a server which is in further communication with a mass memory, those skilled in the art will understand that numerous system configurations exist in which the invention may be practiced with equal advantage. Thus, while the principles of the invention have now been made clear in an illustrative embodiment, there will be immediately obvious to those skilled in the art many modifications used in the practice of the invention which are particularly adapted for specific environments and operating requirements without departing from those principles.

Claims

WHAT IS CLAIMED IS: 1. In a computer system including a user interface, a processor and a mass memory, which mass memory stores infoπnation in physically contiguous memory location segments which are interrupted by discontinuities at a regular interval, a process for communicating between the user interface and die mass memory comprising the steps of: A) establishing, for each set of information to be stored in the mass memory with which the process will be used, a set of characteristics which include at least: 1) the regular interval at which discontinuities appear, the "INTERVAL"; and 2) any offset from the first physical location in the relevant subdivision of the mass memory at which the first record of the infoπnation set is to be stored, the "START_CI"; B) employing the user interface to specify a virtual identification for the physical location to be accessed in the mass memory; C) relating the virtual identification to a hash bucket number, the "BUCKETS", in a series of sequentially contiguous hash bucket numbers; E) in the processor, transfoπning the hash bucket number into a physical location "CI#" in the mass memory employing an integer arithmetic hashing process comprising the following sub- steps: 1) obtaining a value "MIN" by employing the equation: MIN = BUCKET#/(INTER VAL - 1 ); 2) obtaining a value "CLOSE" by employing die equation: CLOSE = (MIN * INTERVAL) + START_CI; 3) obtaining a value "DIST" by employing the equation: DIST = INTERVAL - (CLOSE MOD INTERVAL) in which the term "CLOSE MOD INTERVAL" means the remainder of CLOSE divided by INTERVAL; 4) making a YES/NO decision by making the determination: (BUCKET# - (MIN * (INTERVAL - 1))) < DIST?; 5) if the decision in sub-step 4) is YES, obtaining the CI# by employing the equation: CI# = START_CI + BUCKET# + MIN; 6) if the decision in sub-step 4) is NO, obtaining the CI# by employing the equation: CI# = START_CI + BUCKET# + MIN + 1 ; and F) using the CI# to establish a communications path between the user interface and the physical location to be accessed in the mass memory.

2. In a computer system including a user interface, a processor and a mass memory, which mass memory stores infoπnation in physically contiguous memory location segments which are interrupted by discontinuities at a regular interval, a process for communicating between the user interface and the mass memory by specifying a virtual identification value comprising the steps of: A) establishing, for each set of information to be stored in the mass memory with which the process will be used, a set of characteristics which include: 1) the regular interval at which discontinuities appear, the "INTERVAL"; 2) any offset from the first physical location in the relevant subdivision of the mass memory at which the first record of the information set is to be stored, the "START_CI"; and 3) a "PERJND" value determined as follows: a) if a virtual identification value has a single counterpart segment in the mass memory, set the PERJND value to a first number; b) if a plurality of virtual identification values map to a single counterpart segment in the mass memory, set the PERJND value to a second number; and c) if a virtual identification value has a plurality of counterpart segments in the mass memory, set the PERJND value to a third number; B) employing the user interface to specify a virtual identification value for the physical location to be accessed in the mass memory; C) relating the virtual identification value to a hash bucket number, the "original BUCKETS", in a series of sequentially contiguous hash bucket munbers; E) in the processor, transfoπning the hash bucket number into a physical location "CI#" in the mass memory employing an integer arithmetic hashing process comprising the following sub- steps: 1) examining the PERJND value; 2) if the PERJND value is the first number, go to sub- step 5); 3) if the PERJND value is the second number, obtain a revised BUCKET# to be used following this sub-step by dividing the original BUCKETS by the second number and then going to sub-step 5; 4) if the PERJND value is the third niunber, obtain a revised BUCKET// to be used following this sub-step by multiplying the original BUCKETS by the third number; 5) obtaining a value "MIN" by employing the equation: MIN = BUCKET . /(INTER VAL - 1 ); 6) obtaining a value "CLOSE" by employing the equation: CLOSE = (MIN * INTERVAL) + START_CI; 7) obtaining a value "DIST" by employing the equation: DIST = INTERVAL - (CLOSE MOD INTERVAL) in which the teπn "CLOSE MOD INTERVAL" means the remainder of CLOSE divided by INTERVAL; 8) making a YES/NO decision by making the determination: (BUCKET/ - (MIN * (INTERVAL - 1))) < DIST?; 9) if the decision in sub-step 8) is YES, obtaining the CI// by employing the equation: CI# = START_CI + BUCKET// + MIN; 10) if the decision in sub-step 8) is NO, obtaining the CI# by employing the equation: CI = START_CI + BUCKET// + MIN + 1 ; and F) using the CI/ to establish a communications path between the user interface and the physical location to be accessed in the mass memory.

3. The process of Claim 1 in which the virtual identification is the BUCKET//.

4. The process of Claim 2 in which the virtual identification is the original BUCKET//.

5. In a computer system including a user interface, a processor and a mass memory, which mass memory stores infoπnation in physically contiguous memory location segments which are interrupted by discontinuities at a regular interval of 2ⁿ, a process for communicating between the user interface and the mass memory comprising the steps of: A) establishing, for each set of infoπnation to be stored in the mass memory with which the process will be used, a set of characteristics which include at least: 10 1) the regular interval of 2ⁿ at which discontinuities n appear, the "INTERVAL"; and

12 2) any offset from the first physical location in the

13 relevant subdivision of the mass memory at which the first

14 record of the infoπnation set is to be stored, the is "START_CI";

16 B) employing the user interface to specify a virtual identification

17 for the physical location to be accessed in die mass memory; is C) relating the virtual identification to a hash bucket number, the

19 "BUCKET//", in a series of sequentially contiguous hash bucket 0 numbers; i E) in the processor, transfoπning the hash bucket number into a 2 physical location "CI//" in the mass memory employing an 3 integer arithmetic hashing process comprising the following sub- 4 steps: 5 1) obtaining a value "BCI//" by employing the equation: 6 BCI// = BUCKET//; 7 2) obtaining a trial value "CI#" by employing the 8 equation: 9 CIU = BCI// + START_CI; 0 3) obtaining a revised interval value "INTERVAL" by 1 employing the equation: 2 INTERVAL ^•= INTERVAL - 1 ; 3 4) obtaining a revised value "BCI//" by employing the 4 equation: 5 BCI# = BCI# - ((START_CI + INTERVAL) & -INTERVAL) - START_CI where the operator "&" is a logical AND and the operator "~" is a logical NOT; 5) making a YES/NO decision by making the determination: BCI// >= 0?; 6) if the decision in sub-step 5) is YES, obtaining the CI# by employing the equation: C = C + (BCI///INTER VAL) + 1 ; 7) if the decision in sub-step 5) is NO, accepting die CI// obtained in sub-step 2) as the final CI#; and F) using the CI// to establish a communications path between the user interface and the physical location to be accessed in die mass memory.

6. In a computer system including a user interface, a processor and a mass memory, wliich mass memory stores infoπnation in physically contiguous memory location segments which are interrupted by discontinuities at a regular interval of 2ⁿ, a process for communicating between the user interface and the mass memory by specifying a virtual identification value comprising the steps of: A) establishing, for each set of infoπnation to be stored in the mass memory with which the process will be used, a set of characteristics which include: 1) the regular interval of 2" at which discontinuities appear, the "INTERVAL"; 2) any offset from the first physical location in the relevant subdivision of the mass memory at wliich the first record of the information set is to be stored, the "START_CI"; and 3) a "PERJND" value determined as follows: a) if a virtual identification value has a single counterpart segment in the mass memory, set the PERJND value to a first number; b) if a plurality of virtual identification values map to a single counterpart segment in the mass memory, set the PERJND value to a second number; and c) if a virtual identification value has a plurality of counterpart segments in the mass memory, set the PERJND value to a third number; B) employing the user interface to specify a virtual identification value for the physical location to be accessed in the mass memory; C) relating the virtual identification value to a hash bucket number, the "BUCKET//", in a series of sequentially contiguous hash bucket numbers; E) in the processor, transfoπning the hash bucket number into a physical location "CI/ " in the mass memory employing an integer arithmetic hashing process comprising the following sub- steps: 1) examining the PERJND value; 38 2) if the PERJND value is die first number, obtaining a

39 value "BCI#" by employing the equation:

40 BCI// = BUCKET//

41 and then going to sub-step 5);

42 3) if the PERJND value is the second number, obtaining

43 a value "BCI" by employing the equation:

44 BCI# = BUCKET///PER;

45 and then going to sub-step 5;

46 4) if the PERJND value is the third number, obtaining a

47 value "BCI//" by employing the equation:

48 BCI// = BUCKET// * PER;

49 5) obtaining a trial value "CI//" by employing the

50 equation:

51 CI# = BCI# + START_CI;

52 6) obtaining a revised interval value "INTERVAL" by

53 employing the equation:

54 INTERVAL = INTERVAL - 1 ;

55 7) obtaining a revised value "BCI//" by employing the

56 equation:

57 BCI// = BCI// - ((START_CI + INTERVAL) &

58 -INTERVAL) - START_CI

59 where the operator "&" is a logical AND and the operator

60 "~" is a logical NOT;

61 8) making a YES/NO decision by making the

62 determination:

63 BCI// >= 0?; 9) if die decision in sub-step 8) is YES, obtaining the CI# by employing the equation: Cm = CI# + (BCI NTER VAL;) + 1 ; 10) if the decision in sub-step 8) is NO, accepting the CI# obtained in sub-step 2) as die final CI#; and F) using die CI# to establish a communications path between the user interface and die physical location to be accessed in the mass memory.

7. The process of Claim 5 in which the virtual identification is the BUCKET//.

8. The process of Claim 6 in which the virtual identification is the BUCKET//.