CA2125606A1 - Efficient storage of objects in a file system - Google Patents

Efficient storage of objects in a file system

Info

Publication number
CA2125606A1
CA2125606A1 CA002125606A CA2125606A CA2125606A1 CA 2125606 A1 CA2125606 A1 CA 2125606A1 CA 002125606 A CA002125606 A CA 002125606A CA 2125606 A CA2125606 A CA 2125606A CA 2125606 A1 CA2125606 A1 CA 2125606A1
Authority
CA
Canada
Prior art keywords
sized
variable
file system
stored
storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CA002125606A
Other languages
French (fr)
Other versions
CA2125606C (en
Inventor
Mark Zbikowski
Brian T. Berkowitz
Robert I. Ferguson
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corp filed Critical Microsoft Corp
Publication of CA2125606A1 publication Critical patent/CA2125606A1/en
Application granted granted Critical
Publication of CA2125606C publication Critical patent/CA2125606C/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
    • G11B20/10—Digital recording or reproducing
    • G11B20/12—Formatting, e.g. arrangement of data block or words on the record carriers
    • G11B20/1217—Formatting, e.g. arrangement of data block or words on the record carriers on discs
    • G11B20/1252—Formatting, e.g. arrangement of data block or words on the record carriers on discs for discontinuous data, e.g. digital information signals, computer programme data
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • G11B27/32—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier
    • G11B27/327—Table of contents
    • G11B27/329—Table of contents on a disc [VTOC]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F2003/0697—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers device management, e.g. handlers, drivers, I/O schedulers
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B2220/00—Record carriers by type
    • G11B2220/20—Disc-shaped record carriers

Abstract

A file system provides a hierarchical approach to storage of file data. Logically contiguous bytes of file data are stored in stream data structures, and streams which hold related file data are stored in a variable sized onode data structures on disk. The variable-sized onode data structures are stored in fixed sized buckets, and the buckets are organized into a variable sized array. Thus, the file system provides a combination of variable-sized storage with fixed-sized storage to provide a very efficient approach to storing file data. In addition, indexes may be provided within the structures to allow quick access to onodes and streams.
CA002125606A 1993-06-30 1994-06-10 Efficient storage of objects in a file system Expired - Lifetime CA2125606C (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US08/086,344 1993-06-30
US08/086,344 US5613105A (en) 1993-06-30 1993-06-30 Efficient storage of objects in a file system

Publications (2)

Publication Number Publication Date
CA2125606A1 true CA2125606A1 (en) 1994-12-31
CA2125606C CA2125606C (en) 2002-05-21

Family

ID=22197972

Family Applications (1)

Application Number Title Priority Date Filing Date
CA002125606A Expired - Lifetime CA2125606C (en) 1993-06-30 1994-06-10 Efficient storage of objects in a file system

Country Status (5)

Country Link
US (1) US5613105A (en)
EP (1) EP0632364B1 (en)
JP (3) JPH0728675A (en)
CA (1) CA2125606C (en)
DE (1) DE69430027T2 (en)

Families Citing this family (95)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5819283A (en) * 1993-05-11 1998-10-06 Apple Computer, Inc. Method and system for the extensibility of objects
CA2124752C (en) * 1993-06-30 2005-04-12 Mark Zbikowski Meta-data structure and handling
CA2124754C (en) * 1993-06-30 2005-06-28 Mark Zbikowski Storage of file data on disk in multiple representations
CA2117846C (en) * 1993-10-20 2001-02-20 Allen Reiter Computer method and storage structure for storing and accessing multidimensional data
US6247105B1 (en) * 1996-06-20 2001-06-12 Sun Microsystems, Inc. Externally identifiable descriptor for standard memory allocation interface
US5960431A (en) * 1996-12-19 1999-09-28 International Business Machines Corporation Method and apparatus for adding data storage bins to a stored computer database while minimizing movement of data and balancing data distribution
US6182060B1 (en) * 1997-04-15 2001-01-30 Robert Hedgcock Method and apparatus for storing, retrieving, and processing multi-dimensional customer-oriented data sets
US6282605B1 (en) * 1999-04-26 2001-08-28 Moore Computer Consultants, Inc. File system for non-volatile computer memory
US7197491B1 (en) 1999-09-21 2007-03-27 International Business Machines Corporation Architecture and implementation of a dynamic RMI server configuration hierarchy to support federated search and update across heterogeneous datastores
US6370541B1 (en) 1999-09-21 2002-04-09 International Business Machines Corporation Design and implementation of a client/server framework for federated multi-search and update across heterogeneous datastores
US6792416B2 (en) 1999-09-21 2004-09-14 International Business Machines Corporation Managing results of federated searches across heterogeneous datastores with a federated result set cursor object
US7113939B2 (en) 1999-09-21 2006-09-26 International Business Machines Corporation Architecture to enable search gateways as part of federated search
US6466933B1 (en) 1999-09-21 2002-10-15 International Business Machines Corporation Delayed delivery of query results or other data from a federated server to a federated client until such information is needed
JP5220974B2 (en) * 1999-10-14 2013-06-26 ブルアーク ユーケー リミテッド Apparatus and method for acceleration of hardware execution or operating system functions
US7331058B1 (en) * 1999-12-16 2008-02-12 International Business Machines Corporation Distributed data structures for authorization and access control for computing resources
AU2001293509A1 (en) * 2000-10-04 2002-04-15 Bullant Technology Pty Ltd Data processing structure
US7088910B2 (en) * 2001-02-09 2006-08-08 Microsoft Corporation Optimizing use of storage space in a video data recording system
US6766413B2 (en) 2001-03-01 2004-07-20 Stratus Technologies Bermuda Ltd. Systems and methods for caching with file-level granularity
EP1417800B1 (en) * 2001-08-15 2017-10-04 Bentley Systems, Incorporated Method and system for storing large data files
US7428548B2 (en) 2001-08-15 2008-09-23 Bentley Systems, Inc. Computer readable medium for storing large data files
JP4225740B2 (en) * 2002-04-26 2009-02-18 三洋電機株式会社 File management device
GB2390181A (en) * 2002-06-20 2003-12-31 Achiezer Racov Method for storing data received over a telephone network
US8041735B1 (en) 2002-11-01 2011-10-18 Bluearc Uk Limited Distributed file system and method
US7457822B1 (en) 2002-11-01 2008-11-25 Bluearc Uk Limited Apparatus and method for hardware-based file system
US7120637B2 (en) * 2003-05-30 2006-10-10 Microsoft Corporation Positional access using a b-tree
US7633886B2 (en) * 2003-12-31 2009-12-15 University Of Florida Research Foundation, Inc. System and methods for packet filtering
US7904906B2 (en) * 2004-11-23 2011-03-08 Stratus Technologies Bermuda Ltd. Tracking modified pages on a computer system
US7908246B2 (en) * 2008-03-06 2011-03-15 International Business Machines Corporation Separating file data streams to enhance progressive incremental processing
US8984390B2 (en) 2008-09-15 2015-03-17 Palantir Technologies, Inc. One-click sharing for screenshots and related documents
DE102009031923A1 (en) * 2009-07-07 2011-01-13 Sones Gmbh Method for managing data objects
US9092482B2 (en) 2013-03-14 2015-07-28 Palantir Technologies, Inc. Fair scheduling for mixed-query loads
US8799228B2 (en) 2011-06-29 2014-08-05 Nokia Corporation Method and apparatus for providing a list-based interface to key-value stores
US8732574B2 (en) 2011-08-25 2014-05-20 Palantir Technologies, Inc. System and method for parameterizing documents for automatic workflow generation
US8504542B2 (en) 2011-09-02 2013-08-06 Palantir Technologies, Inc. Multi-row transactions
US8433702B1 (en) * 2011-09-28 2013-04-30 Palantir Technologies, Inc. Horizon histogram optimizations
US8560494B1 (en) 2011-09-30 2013-10-15 Palantir Technologies, Inc. Visual data importer
CN103186617B (en) * 2011-12-30 2016-04-06 北京新媒传信科技有限公司 A kind of method and apparatus storing data
US9378526B2 (en) 2012-03-02 2016-06-28 Palantir Technologies, Inc. System and method for accessing data objects via remote references
US9798768B2 (en) 2012-09-10 2017-10-24 Palantir Technologies, Inc. Search around visual queries
US9348677B2 (en) 2012-10-22 2016-05-24 Palantir Technologies Inc. System and method for batch evaluation programs
US9471370B2 (en) 2012-10-22 2016-10-18 Palantir Technologies, Inc. System and method for stack-based batch evaluation of program instructions
US9367463B2 (en) 2013-03-14 2016-06-14 Palantir Technologies, Inc. System and method utilizing a shared cache to provide zero copy memory mapped database
US8903717B2 (en) 2013-03-15 2014-12-02 Palantir Technologies Inc. Method and system for generating a parser and parsing complex data
US9898167B2 (en) 2013-03-15 2018-02-20 Palantir Technologies Inc. Systems and methods for providing a tagging interface for external content
US8868486B2 (en) 2013-03-15 2014-10-21 Palantir Technologies Inc. Time-sensitive cube
US9740369B2 (en) 2013-03-15 2017-08-22 Palantir Technologies Inc. Systems and methods for providing a tagging interface for external content
US8930897B2 (en) 2013-03-15 2015-01-06 Palantir Technologies Inc. Data integration tool
US8909656B2 (en) 2013-03-15 2014-12-09 Palantir Technologies Inc. Filter chains with associated multipath views for exploring large data sets
US8855999B1 (en) 2013-03-15 2014-10-07 Palantir Technologies Inc. Method and system for generating a parser and parsing complex data
US9317213B1 (en) * 2013-05-10 2016-04-19 Amazon Technologies, Inc. Efficient storage of variably-sized data objects in a data store
US9105000B1 (en) 2013-12-10 2015-08-11 Palantir Technologies Inc. Aggregating data from a plurality of data sources
US8935201B1 (en) 2014-03-18 2015-01-13 Palantir Technologies Inc. Determining and extracting changed data from a data source
US20160026923A1 (en) 2014-07-22 2016-01-28 Palantir Technologies Inc. System and method for determining a propensity of entity to take a specified action
US10282874B2 (en) 2014-09-17 2019-05-07 Circonus, Inc. Efficient time-series histograms
US9697227B2 (en) * 2014-10-27 2017-07-04 Cohesity, Inc. Concurrent access and transactions in a distributed file system
US9348880B1 (en) 2015-04-01 2016-05-24 Palantir Technologies, Inc. Federated search of multiple sources with conflict resolution
US9996302B2 (en) * 2015-04-03 2018-06-12 Toshiba Memory Corporation Storage device writing data on the basis of stream
US9514205B1 (en) 2015-09-04 2016-12-06 Palantir Technologies Inc. Systems and methods for importing data from electronic data files
US10558339B1 (en) 2015-09-11 2020-02-11 Palantir Technologies Inc. System and method for analyzing electronic communications and a collaborative electronic communications user interface
US9772934B2 (en) 2015-09-14 2017-09-26 Palantir Technologies Inc. Pluggable fault detection tests for data pipelines
US9652510B1 (en) 2015-12-29 2017-05-16 Palantir Technologies Inc. Systems and user interfaces for data analysis including artificial intelligence algorithms for generating optimized packages of data items
US10579680B2 (en) 2016-05-13 2020-03-03 Tibco Software Inc. Using a B-tree to store graph information in a database
US10554516B1 (en) 2016-06-09 2020-02-04 Palantir Technologies Inc. System to collect and visualize software usage metrics
US9678850B1 (en) 2016-06-10 2017-06-13 Palantir Technologies Inc. Data pipeline monitoring
US10621314B2 (en) 2016-08-01 2020-04-14 Palantir Technologies Inc. Secure deployment of a software package
US10133782B2 (en) 2016-08-01 2018-11-20 Palantir Technologies Inc. Techniques for data extraction
US11256762B1 (en) 2016-08-04 2022-02-22 Palantir Technologies Inc. System and method for efficiently determining and displaying optimal packages of data items
US10552531B2 (en) 2016-08-11 2020-02-04 Palantir Technologies Inc. Collaborative spreadsheet data validation and integration
US10373078B1 (en) 2016-08-15 2019-08-06 Palantir Technologies Inc. Vector generation for distributed data sets
EP3282374A1 (en) 2016-08-17 2018-02-14 Palantir Technologies Inc. User interface data sample transformer
US10650086B1 (en) 2016-09-27 2020-05-12 Palantir Technologies Inc. Systems, methods, and framework for associating supporting data in word processing
US10152306B2 (en) 2016-11-07 2018-12-11 Palantir Technologies Inc. Framework for developing and deploying applications
US10261763B2 (en) 2016-12-13 2019-04-16 Palantir Technologies Inc. Extensible data transformation authoring and validation system
US11157951B1 (en) 2016-12-16 2021-10-26 Palantir Technologies Inc. System and method for determining and displaying an optimal assignment of data items
US10509844B1 (en) 2017-01-19 2019-12-17 Palantir Technologies Inc. Network graph parser
US10180934B2 (en) 2017-03-02 2019-01-15 Palantir Technologies Inc. Automatic translation of spreadsheets into scripts
US10572576B1 (en) 2017-04-06 2020-02-25 Palantir Technologies Inc. Systems and methods for facilitating data object extraction from unstructured documents
US10824604B1 (en) 2017-05-17 2020-11-03 Palantir Technologies Inc. Systems and methods for data entry
US10534595B1 (en) 2017-06-30 2020-01-14 Palantir Technologies Inc. Techniques for configuring and validating a data pipeline deployment
US10204119B1 (en) 2017-07-20 2019-02-12 Palantir Technologies, Inc. Inferring a dataset schema from input files
US10754820B2 (en) 2017-08-14 2020-08-25 Palantir Technologies Inc. Customizable pipeline for integrating data
US11016936B1 (en) 2017-09-05 2021-05-25 Palantir Technologies Inc. Validating data for integration
US11379525B1 (en) 2017-11-22 2022-07-05 Palantir Technologies Inc. Continuous builds of derived datasets in response to other dataset updates
US10783162B1 (en) 2017-12-07 2020-09-22 Palantir Technologies Inc. Workflow assistant
US10552524B1 (en) 2017-12-07 2020-02-04 Palantir Technolgies Inc. Systems and methods for in-line document tagging and object based data synchronization
US10360252B1 (en) 2017-12-08 2019-07-23 Palantir Technologies Inc. Detection and enrichment of missing data or metadata for large data sets
US11176116B2 (en) 2017-12-13 2021-11-16 Palantir Technologies Inc. Systems and methods for annotating datasets
US10853352B1 (en) 2017-12-21 2020-12-01 Palantir Technologies Inc. Structured data collection, presentation, validation and workflow management
GB201800595D0 (en) 2018-01-15 2018-02-28 Palantir Technologies Inc Management of software bugs in a data processing system
CN110109914A (en) * 2018-01-16 2019-08-09 恒为科技（上海）股份有限公司 A kind of data storage of application drive and indexing means
US10599762B1 (en) 2018-01-16 2020-03-24 Palantir Technologies Inc. Systems and methods for creating a dynamic electronic form
US10885021B1 (en) 2018-05-02 2021-01-05 Palantir Technologies Inc. Interactive interpreter and graphical user interface
US11263263B2 (en) 2018-05-30 2022-03-01 Palantir Technologies Inc. Data propagation and mapping system
US11061542B1 (en) 2018-06-01 2021-07-13 Palantir Technologies Inc. Systems and methods for determining and displaying optimal associations of data items
US10795909B1 (en) 2018-06-14 2020-10-06 Palantir Technologies Inc. Minimized and collapsed resource dependency path

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4611272A (en) * 1983-02-03 1986-09-09 International Business Machines Corporation Key-accessed file organization
US4606002A (en) * 1983-05-02 1986-08-12 Wang Laboratories, Inc. B-tree structured data base using sparse array bit maps to store inverted lists
US4677550A (en) * 1983-09-30 1987-06-30 Amalgamated Software Of North America, Inc. Method of compacting and searching a data index
US4761737A (en) * 1986-01-16 1988-08-02 International Business Machines Corporation Method to automatically increase the segment size of unix files in a page segmented virtual memory data processing system
US5265244A (en) * 1986-02-14 1993-11-23 International Business Machines Corporation Method and system for facilitating processing of statistical inquires on stored data accessible through a data access structure
US5034914A (en) * 1986-05-15 1991-07-23 Aquidneck Systems International, Inc. Optical disk data storage method and apparatus with buffered interface
US4953122A (en) * 1986-10-31 1990-08-28 Laserdrive Ltd. Pseudo-erasable and rewritable write-once optical disk memory system
US5058002A (en) * 1987-06-23 1991-10-15 Mitsubishi Denki Kabushiki Kaisha Page splitting method and apparatus for a database stored in a plurality of memory storage units
US5040110A (en) * 1987-10-30 1991-08-13 Matsushita Electric Industrial Co., Ltd. Write once read many optical disc storage system having directory for storing virtual address and corresponding up-to-date sector address
US5129088A (en) * 1987-11-30 1992-07-07 International Business Machines Corporation Data processing method to create virtual disks from non-contiguous groups of logically contiguous addressable blocks of direct access storage device
US5081608A (en) * 1988-04-18 1992-01-14 Matsushita Electric Industrial Co., Ltd. Apparatus for processing record-structured data by inserting replacement data of arbitrary length into selected data fields
US4935825A (en) * 1988-12-16 1990-06-19 Emulex Corporation Cylinder defect management system for data storage system
US5029125A (en) * 1989-03-07 1991-07-02 Drexler Technology Corporation Method of reading and writing files on nonerasable storage media
US5125086A (en) * 1989-06-29 1992-06-23 Digital Equipment Corporation Virtual memory paging apparatus with variable size in-page clusters
US5218696A (en) * 1989-07-24 1993-06-08 International Business Machines Corporation Method for dynamically expanding and rapidly accessing file directories
US5371885A (en) * 1989-08-29 1994-12-06 Microsoft Corporation High performance file system
US5202986A (en) * 1989-09-28 1993-04-13 Bull Hn Information Systems Inc. Prefix search tree partial key branching
US5247658A (en) * 1989-10-31 1993-09-21 Microsoft Corporation Method and system for traversing linked list record based upon write-once predetermined bit value of secondary pointers
US5261088A (en) * 1990-04-26 1993-11-09 International Business Machines Corporation Managing locality in space reuse in a shadow written B-tree via interior node free space list
US5237675A (en) * 1990-06-04 1993-08-17 Maxtor Corporation Apparatus and method for efficient organization of compressed data on a hard disk utilizing an estimated compression factor
US5043967A (en) * 1990-08-20 1991-08-27 International Business Machines Corporation Structured data storage method and medium
US5206939A (en) * 1990-09-24 1993-04-27 Emc Corporation System and method for disk mapping and data retrieval
JP2761289B2 (en) * 1990-11-30 1998-06-04 富士通株式会社 Disk track emulation method
US5204958A (en) * 1991-06-27 1993-04-20 Digital Equipment Corporation System and method for efficiently indexing and storing a large database with high data insertion frequency

Also Published As

Publication number Publication date
CA2125606C (en) 2002-05-21
JP4250190B2 (en) 2009-04-08
EP0632364A1 (en) 1995-01-04
JPH0728675A (en) 1995-01-31
DE69430027T2 (en) 2002-07-18
JP2008041108A (en) 2008-02-21
JP2005100451A (en) 2005-04-14
EP0632364B1 (en) 2002-03-06
DE69430027D1 (en) 2002-04-11
US5613105A (en) 1997-03-18

Similar Documents

Publication Publication Date Title
CA2125606A1 (en) Efficient storage of objects in a file system
CA2124752A1 (en) Meta-data structure and handling
EP0640966A3 (en) Storage and retrieval system.
AU5597898A (en) File locking within a clustered storage environment
EP0466389A3 (en) File system with read/write and read only storage
AU7477298A (en) Method of increasing the storage capacity of a level five raid disk array, and expanded array produced by such method
CA2157167A1 (en) Launching computer applications
AU7013894A (en) Sleeve and storage device for compact disks
EP0465818A3 (en) Mass document storage and retrieval system
CA2176213A1 (en) Mass data storage library
AU628724B2 (en) Cryogenic storage container
AU3601593A (en) Self-powered portable telephone with cdi disk storage medium
CA2278642A1 (en) A method and device for storing audio-centered information by a table-of-contents (toc) mechanism and also by a file-based access mechanism through a root directory that contains a highest level toc directory, and a unitary storage medium containing such information
CA2124754A1 (en) Storage of file data on disk in multiple representations
AU1403092A (en) Information storage and retrieval systems
GB2274881B (en) A multi-mode engine integrating the following modes:ejector using liquefied or cooled turbocompressed air;ramjet,and super ramjet
EP0624877A3 (en) A cd storage and display system.
GB2281726B (en) Fuel storage vessels
CA2123445A1 (en) Storage system
USD382009S (en) Pencil box calculator
EP0708442A3 (en) Disk storage and retrieval
USD351080S (en) Storage case for compact discs
WO1996024139A3 (en) Trapping and storage of free thermal neutrons in fullerene molecules
MEGERDIGIAN et al. Carbon fiber/resin matrix interphase- Effect of carbon fiber surface treatment and environmental conditioning on composite performance
AU2460895A (en) Data storage and retrieval system with protection against premature cartridge ejection

Legal Events

Date Code Title Description
EEER Examination request
MKEX Expiry

Effective date: 20140610