US20100290364A1 - Packet Compression for Network Packet Traffic Analysis - Google Patents
Packet Compression for Network Packet Traffic Analysis Download PDFInfo
- Publication number
- US20100290364A1 US20100290364A1 US12/848,391 US84839110A US2010290364A1 US 20100290364 A1 US20100290364 A1 US 20100290364A1 US 84839110 A US84839110 A US 84839110A US 2010290364 A1 US2010290364 A1 US 2010290364A1
- Authority
- US
- United States
- Prior art keywords
- packet
- representation
- file
- record
- address
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/02—Capturing of monitoring data
- H04L43/026—Capturing of monitoring data using flow identification
Abstract
Methods of capturing and compressing trace data for use in network packet traffic analysis are described. In an embodiment, when a packet is received, two records of the packet are created and stored. One record is stored in a file associated with the source address of the packet and the other record is stored in a file associated with the destination address of the packet. Various packet compression techniques are described and one example compares a newly received packet to the previous packet which has been stored in the same file and sets bits in the record which denote whether fields in the newly received packet are the same as the corresponding fields in the previous packet.
Description
- This application is a continuation of U.S. patent application Ser. No. 12/118,595, filed on May 9, 2008, the entirety of which is incorporated by reference.
- Network packet traffic analysis may be performed in a number of different ways: the analysis may be performed in real time (on-line) or from stored data (off-line) and the data analyzed may represent a substantially complete record of packet activity or the data may be sampled from the network and therefore represent only a small fraction of the packets in the network. A substantially complete record of packet activity is known as a trace.
- A capturing agent may be used to capture a trace by capturing data and storing it on disk. Where the speed at which data arrives exceeds the speed that the data can be written to a disk, packets may be truncated and truncated packets stored. Each truncated packet is shorter than a complete packet and comprises the front portion of a packet without the end portion of the packet. The length of the truncated packet is known as the snap-length. Having captured a trace, the data may be analyzed in many different ways and many different aspects of the data maybe investigated.
- Typically a trace comprises complete or truncated packets captured over a short period of time or the trace comprises a statistical sampling of the number and temporal distribution of packets sent between machines.
- The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known methods of capturing trace data.
- The following presents a simplified summary of the disclosure in order to provide a basic understanding to the reader. This summary is not an extensive overview of the disclosure and it does not identify key/critical elements of the invention or delineate the scope of the invention. Its sole purpose is to present some concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later.
- Methods of capturing and compressing trace data for use in network packet traffic analysis are described. In an embodiment, when a packet is received, two records of the packet are created and stored. One record is stored in a file associated with the source address of the packet and the other record is stored in a file associated with the destination address of the packet. Various packet compression techniques are described and one example compares a newly received packet to the previous packet which has been stored in the same file and sets bits in the record which denote whether fields in the newly received packet are the same as the corresponding fields in the previous packet.
- Many of the attendant features will be more readily appreciated as the same becomes better understood by reference to the following detailed description considered in connection with the accompanying drawings.
- The present description will be better understood from the following detailed description read in light of the accompanying drawings, wherein:
-
FIG. 1 is a schematic diagram of a network; -
FIG. 2 is a schematic diagram of a packet in a computer network; -
FIG. 3 is a flow diagram of an example method of capturing trace data; -
FIGS. 4-8 are flow diagrams of example methods of compressing trace data; -
FIG. 9 shows example implementations of two method blocks fromFIG. 8 in more detail; -
FIG. 10 shows a comparison between two formats of a packet record; -
FIG. 11 shows a flow diagram of a method of converting a stream of packets into multiple files using a multiplicity of instances of the method shown inFIG. 8 ; -
FIG. 12 illustrates the process of decompressing a compressed trace file; and -
FIG. 13 illustrates an exemplary computing-based device in which embodiments of the methods described herein may be implemented. Like reference numerals are used to designate like parts in the accompanying drawings. - The detailed description provided below in connection with the appended drawings is intended as a description of the present examples and is not intended to represent the only forms in which the present example may be constructed or utilized. The description sets forth the functions of the example and the sequence of steps for constructing and operating the example. However, the same or equivalent functions and sequences may be accomplished by different examples.
- Trace captures are often taken at a SPAN (switched port analyzer) port on a router or switch which provides a stream of packets representing a copy of packets between one or more computers on each of one or more other ports.
FIG. 1 is a schematic diagram of a network comprising a router/switch 101 and a number ofcomputers 102 which may be interconnected in any way. Port mirroring may be used on the router/switch 101 to send a copy of all packets that pass through the router/switch 101 to aport 103 which is connected to acomputer 104 which runs a capturing agent that captures the trace. Thecomputer 104 receives the packets on anetwork card 105. - A
typical packet 200 of interest in a computer network, as shown inFIG. 2 , comprises an Ethernetheader 201 of fourteen bytes, followed by an Internet Protocol (IP)header 202 which may be variable in length but which is at least twenty bytes and usually but not always twenty bytes. This IP header indicates the protocol of the packet which is usually either the unreliable datagram protocol (UDP) or the transport control protocol (TCP). If the UDP is indicated then the IP header is followed by a twentybyte UDP header 203. If the TCP is indicated then the IP header is followed by a variablelength TCP header 203 which is at least twenty bytes and usually but not always twenty bytes. These headers 201-203 are followed by thedata 204 which might be the user's data and/or a header belonging to some higher level protocol. The amount of data is variable, but the total length of the packet is less than the frame limit which is usually fifteen hundred and fourteen bytes. -
FIG. 3 is a flow diagram of an example method of capturing trace data. A packet is received (block 301) and data is appended to the packet (block 302). A first version of the packet with the appended data is then stored in a file associated with the source of the packet (block 303) and a second version of the packet with the appended data is stored in a file associated with the destination of the packet (block 304). The first and second versions of the packet with appended data may be identical or they may be different (as described in more detail below). - When data is appended to the packet (in block 302), data which is in the packet, such as some or all of the user's data, may also be removed e.g. packets may be truncated (or snapped) to conserve disk space, or disk bandwidth, or to reduce the privacy impact of storing the portion of the packet comprising data rather than headers. In an example, the computer records the current time when the packet arrives, the length of the packet and the amount of the data which will be snapped (i.e. captured) and not discarded. The computer then appends the time (usually requiring eight bytes) and the two lengths (usually requiring four bytes each) to the packet (snapped to at least 14+20+20 but frequently a value such as 128 bytes) and this is output to the storage device, e.g. a disk.
- The versions of the packets stored (in
blocks 303 and 304) may comprise packet records instead of the packets themselves (either full or truncated packets). Examples of packet records are described in more detail below. - Using the method shown in
FIG. 3 , multiple files are used to record packet information. A multiplicity of files is used, with in many cases one file for each network address, though a file may be used for several addresses. Each packet record is written to two files, the file for the source address of the packet (in block 303), and the file for the destination address of the packet (in block 304). For broadcast packets, the file for the destination address may comprise a file for the broadcast address. Alternatively, broadcast packets may be stored in the files for multiple destination addresses; however this is less efficient. - By dividing the trace data into multiple files, the packets relevant to a computation for a single computer (or a group of computers) can be processed more efficiently since only the file for that computer must be read; or, if broadcast packets are relevant, the files for the address of that computer and the file for the broadcast address of the portion of the network where the computer is present. Additionally the files are more easily processed in parallel on a cluster, since the files required for any particular processing component are much reduced.
- When a packet is stored as part of a captured trace, either using the method of
FIG. 3 or another method of capturing trace data, the packet may be compressed. As described above, the packet may be snapped (or truncated) to a particular length (the snap length) but in other examples, further compression techniques may be used. The following description describes a number of different compression techniques which may be used together (in any combination) or independently. -
FIG. 4 is a flow diagram of an example method of compressing trace data. A packet is received (block 401) and a packet record is created for the packet which comprises specified fields associated with the packet (block 402). The record may encode these specified fields. Having created the packet record (in block 402), the packet is discarded (block 403). The specified fields which are encoded (or otherwise stored) in the packet record may be those fields associated with the original received packet which are generally used in the analysis of network data. These fields may, for example, comprise one or more of: the time of the frame, the source and destination addresses, whether the packet is UDP or TCP, the port numbers, and whether the packet was empty or not (e.g. whether thepacket 200, as shown inFIG. 2 , comprises any data 204). In some implementations some encoding of TCP's protocol flags may also be present in the packet record. In other examples, different fields may be included within the packet record which is generated. -
FIG. 5 is a flow diagram of another example method of compressing trace data. A packet is received (block 501) and if it is the first packet to be stored in a trace file (‘Yes’ in block 502), a packet record is created and stored for the packet (block 503). This packet record may, as described above, comprise specified fields which are to be used in analysis of the trace. The received packet is then discarded (block 504). If however, the packet is not the first to be stored in a trace file (‘No’ in block 502), the packet is compared to the previously stored packet record (block 505) and a packet record for the newly received packet is created which comprises flags which are set to denote whether each of the specified fields have changed from the previously stored packet record (block 506). These flags which denote whether fields have changed between packets are encoded using bits grouped together in one of more bytes which may be referred to as a ‘flag byte’ or ‘flag bytes’. Where fields have changed, these changed fields are appended to the packet record (block 507) and the received packet is then discarded (block 504). The method may then be repeated for each packet received. - This method provides a compression format for the fields (which are themselves larger than one bit) from one packet to the next within a file in which a single bit is used to indicate whether a field would have the same value as the field in the previous packet. This provides significant compression of the size of the trace.
- In addition to compressing the fields, as described above, the time of the packet can also be compressed. Instead of storing the absolute time at which the packet was observed, the difference in time from one packet to the next may be stored. Such a difference is likely to be a smaller value than the absolute time of arrival. For example, two bits in the flag byte may be used to encode whether the time is represented by a one byte, two byte or four byte difference, or by an eight byte absolute time. This significantly reduces the size of the stored file and improves the performance of analysis through the much reduced data volumes.
-
FIG. 6 is a flow diagram of a further example method of compressing trace data. A packet is received (block 601) and two packet records are created (block 602)—one associated with the source address and one associated with the destination address. Within each packet record a direction bit is set (block 603) which indicates whether the packet is being sent to or from the address with which the packet record is associated, i.e. in the packet record associated with the source address, the direction bit will indicate that the packet was being sent from the associated address and in the packet record associated with the destination address, the direction bit will indicate that the packet was being sent to the associated address. The two packet records created are therefore not the same. The fields in the packet record are then mapped according to the direction bit (block 604). For example, instead of having source port and destination port fields, the record encodes local port and remote port fields. If the packet is an input at the address represented in the current file then the destination port is represented in the local port field; if the packet is an output at the address then the destination port would be represented in the remote port field. The created packet records are then stored (block 605) and the received packet discarded (block 606). The packets may be stored in two separate files, one associated with the source address and one associated with the destination address (e.g. as described above with reference toFIG. 3 ). - Another optimization for collecting trace data is shown in
FIG. 7 , in which the address of the peer host for a packet is not stored directly in the file, instead an indirect identifier is stored (and potentially compressed using the method and system described above). For a packet or packet record associated with a source address, the peer host address is the destination address or vice versa. When a packet is received (block 701), the source and/or destination address is replaced by an identifier for each replaced address (block 702). The mapping between the actual address and the identifier is stored (block 703), for example in a library or dictionary, and the amended packet or a packet record for the packet (as described above) is stored (block 704). Where a packet record is stored, the original received packet may be discarded (not shown inFIG. 7 ). - The use of an identifier instead of an actual address, as shown in
FIG. 7 , enables additional compression because the number of addresses present on the network of interest is likely to be smaller than the actual size of an address. Furthermore, the dictionary consulted to find the address represented by the identifiers need not be made available to the persons (or machines) processing the packet data; or alternatively a different dictionary to which a prefix conserving anonymization technique has been applied can be provided instead. Thus the private details of the original addresses in the trace can be completely and easily separated from the main trace data on which the computations will be carried out, thereby improving privacy. Previously two copies of the trace were stored—one containing the actual addresses and one containing anonymized addresses (e.g. addresses to which a prefix conserving anonymization technique has been applied) and therefore use of this method also reduces storage requirements. -
FIG. 8 illustrates the process of creating a compressed trace file for a single address (or group of addresses) from a source of original packets and the packet records stored are referred to herein as being in ‘Reduced Packet Format’ (RPF). It will be appreciated that the method will be implemented in parallel for both the source and destination addresses; howeverFIG. 8 only a single process flow is shown for purposes of clarity. The method shown inFIG. 8 uses all of the compression techniques and optimizations described above, although in other examples only some of those techniques and/or optimizations may be employed.FIG. 9 shows flow diagrams of example implementations ofblocks 808 and 810 of the method shown inFIG. 8 in more detail. - When a packet is received (block 801), the method checks if this is the first packet in the file (block 802), i.e. whether this is the first packet received having this particular address as either the source or the destination address. If the packet received is the first packet in the file (‘Yes’ in block 802), a flag byte indicating an escape of type absolute time is written (block 803) and the absolute time is also written (block 804). A flag byte is written indicating a minimum sized time delta and that all other fields are present (block 805) followed by each of the fields (block 806) and these fields may be written in a predefined order.
- If, however, the packet is not the first in the file (‘No’ in block 802), the time of the packet received (in block 801) is compared to the time of the previous packet, a packet record for which has been previously stored in the file, to determine the magnitude of the time difference between the two packets (block 807). If the time difference (also referred to as the time delta) does not exceed four bytes (‘No’ in block 808), then a flag byte is constructed for the specified fields (block 809). As described above with reference to
FIG. 5 and shown in more detail inFIG. 9 , this flag byte is constructed by converting the source and destination addresses and port numbers into local and remote values (block 901) and setting the direction bit (block 902). The values of the specified fields (e.g. peer address, local port and remote port) in the current packet (received in block 801) are compared to the corresponding values in the previous packet and bits in the flag byte are set to denote whether the values have changed (block 903). In order to perform this comparison, the previous uncompressed packet record may be stored. In addition the protocol and empty bits are set (block 904). - The flag byte, including the time difference between the current and previous packets, is then written to the file (block 810) followed by the values of any of the fields which have changed (block 811), as shown in more detail in
FIG. 9 . If the peer changed bit is set (‘Yes’ in block 910) then an identifier for the peer is written to the file (block 911). This may require consulting the dictionary including adding a new identifier to the dictionary if the address has not previously been seen (not shown inFIG. 9 ). If the local port changed bit is set (‘Yes’ in block 912), the local port is written to the file (block 913) and if the remote port changed bit is set (‘Yes’ in block 914), the remote port is written to the file (block 91 5). - If the time difference is larger than will fit in four bytes (‘Yes’ in block 808), then a flag byte is written indicating an escape of type absolute time (block 812) followed by an eight byte absolute time (block 813). The time difference is then set to zero (block 814) and a second flag byte is constructed for the specified fields (block 809 and as shown in more detail in
FIG. 9 ). This second flag byte, including the time difference between the current and previous packets (which was set to zero in block 814) is then written to the file (block 810) followed by the values of any of the fields which have changed (block 811 and as shown in more detail inFIG. 9 ). The method may then be repeated for subsequent packets received. -
FIG. 10 shows a comparison between theReduced Packet Format 1010 of a packet record and an expandedformat 1000 of a packet record. In the example shown, the expandedformat 1000 comprises values for each of the following: the time of the frame (or packet) 1001, theremote port number 1002, thelocal port number 1003, a protocol field 1004 (e.g. indicating whether TCP or UDP is used), apeer address identifier 1005, adirection bit 1006 and a field indicating whether the packet comprised any user data or not 1007. The data elements may be arranged in any order. TheRPF 1010 comprises theflag byte 1011 and values for any fields that have changed 1019 (e.g. as written in block 810). The flag byte comprises adirection bit 1012, aprotocol bit 1013, anempty bit 1014, a time stamp 1018 (which may comprise two bits, as described above) and bits 1015-1017 indicating whether the peer address, remote port number and local port number have changed since the previous packet. - The
RPF 1010 is the format in which the trace is stored and the expandedformat 1000 may be the format which is used when the trace is processed. The decompression process (i.e. conversion from expandedformat 1000 to RPF 1010) is described below with reference toFIG. 12 . - Since many communication patterns in real networks involve bursts bidirectional communication between computers, and since the encoding from source and destination has been transformed using the method shown in
FIG. 8 to a direction and local and remote it can be seen that adjacent packets in the file are very likely to have identical values of local and remote port numbers and peer address and so these will compress very densely to a single flag each (bits 1015-1017 in the example inFIG. 10 ) indicating that they are the same as the previous packet. - Moreover, even on a busy server computer with many simultaneous connections the local port indicating the service on the computer which is receiving and transmitting many packets will also be identical from packet to packet and hence can also be condensed to a
single bit 1017. Not only does this save very large amounts of storage space in the file, and save on the time to read and write the file to disk, but it can also be used to optimize actual processing of the file since several forms of processing can be enhanced if they can more rapidly identify that certain aspects of a packet are the same as previously. For example, packet processing may begin by determining the flow or channel associated with a packet and this step can be elided if can rapidly be determined that the flow or channel of some packet will be the same as the flow or channel determined for the previous packet. - As described above, and shown in
FIG. 3 , a stream of packets may be stored in multiple files, with each file being associated with an address (or group of addresses).FIG. 11 shows a flow diagram of a method of converting a stream of packets into multiple files using a multiplicity of instances of the method shown inFIG. 8 . When a packet is received (block 1101), the method checks to see if an instance of the method ofFIG. 8 exists for the source address of the packet (block 1102). If such an instance does not exist (‘No’ in block 1102) then one is created and the dictionary is updated (block 1 103) to include a new mapping between an indirect identifier (as described above) and the actual address. The packet is then given to the instance of the method for the source address (block 1 104). The method then checks to see if an instance of the method ofFIG. 8 exists for the destination address of the packet (block 1105). If such an instance does not exist (‘No’ in block 1105) then one is created and the dictionary is updated (block 1106) to include a new mapping between an indirect identifier (as described above) and the actual destination address. The packet is then given to the instance of the method for the destination address (block 1107). The method is then repeated for subsequent packets received. - Whilst the above description in relation to
FIG. 11 describes generating packets associated with each address, in many examples, files may only be created for a subset of addresses. In such an example, the method shown inFIG. 11 may be modified such that for each new source/destination address the dictionary is updated (e.g. inblocks 1103 and 1106) but that new instances are generated only for the subset of addresses where files are required. - The method and system described above enables the processing of packets into files to be performed with an exceptionally high degree of parallelism using all the available processing cores or computers, since the only time that coordination is required is when the dictionary is to be updated (in
blocks FIG. 11 ) and that happens only rarely on the first time that an address is seen in the network. In an example, packets may be received in blocks (e.g. in 10 GB slices) and each core may deal with a separate block of packets. Typically the packets are received in blocks which are divided in time by the capture device such that each block is not bigger than a disk. Each core may run an instance of the method ofFIG. 8 (the RPF writer) for each address such that each core generates a file for each address which is a source or destination address for one of the packets it processes. This results in multiple files for a single address (e.g. where files are generated by different cores) and these may subsequently be combined into a single file per address. - Whilst this provides one example parallelization technique, in another example, one machine or core may perform the method shown in
FIG. 11 and different machines/cores may perform different instances of the method ofFIG. 8 . As described above, each machine/core involved refers back to and updates a central dictionary. - Where a trace file is used for multiple addresses, the methods used may be the same as described above. Alternatively, an extra field or flag may be used to indicate which of the multiple addresses is the source/destination address for a particular packet record. Use of such a field or flag enables the trace file to be separately divided into individual files for each of the addresses. A trace file for multiple addresses may, for example, be compiled where a single machine has multiple addresses on a single interface (e.g. IPv4 and IPv6) or on multiple interfaces.
- Experimental results have demonstrated the compression which is achievable using the methods described above. When the methods were applied to a 4500 GB dataset it was reduced it to a mere 70 GB. This represents a compression ratio in excess of 5000%, well beyond what is achievable with general purpose compression techniques. In addition the resulting dataset was easier to process and parallelize on a cluster of computers.
- The methods described herein and the resulting large compression ratio enable a trace to be captured which provides data on all packets captured over a large period of time and/or for a large number of machines.
- In the methods described above, all the packets received result in a packet or packet record being stored. In other examples, the packets received may be filtered e.g. so that only packets of interest are captured and stored in the trace. In an example, the filtering may be performed based on round trip time (RTT).
-
FIG. 12 illustrates the process of decompressing a compressed trace file (which may be referred to as a Reduced Packet Format file), which may have been generated using the method shown inFIG. 8 . This decompression may be performed for the purpose of processing the file e.g. to perform network packet analysis. - The method starts by reading a flag byte from the file (block 1201). If the byte is determined (in block 1202) to be an escape, as indicated by the two timestamp bits being both set, then the method checks if it is an absolute timestamp escape (block 1203). If it is (‘Yes’ block 1203) then an eight byte absolute time is read, the current time is set to this value (block 1204) and the method returns to the start and reads the next flag byte (block 1201). If it is some other type of escape (‘No’ in block 1203) then additional escape specific processing is performed (block 1205). If it was not an escape then the two bits which encode the size of the timestamp are examined (in block 1202) and if they are 00 then a single byte is read and added to the current time (block 1206). If the two bits are 01 then a two-byte value is read and added it to the current time (block 1207) and if they are 10 then a four-byte value is read and added to the current time (block 1208).
- Where the two bits which encode the size of the timestamp are 00, 01 or 10, the method continues (following blocks 1206-1208) by checking to see if the peer present bit is set (block 1210) and if it is, the current peer is updated by reading a peer address identifier (block 1211). The method then checks to see if the local port present bit is set (block 1220) and if it is the current local port is updated by reading a port value (block 1221). The method then checks to see if the remote port present bit is set (block 1230) and if it is, the current remote port is updated by reading a port value (block 1231).
- Having updated the values of the peer address identifier, local port and remote port if required (
blocks format 1000, as shown inFIG. 10 ) is made available for processing (block 1240) and then if the end of file has not been reached (‘No’ in block 1250) the method is repeated. The method stops (block 1260) when the end of the file is reached (‘Yes’ in block 1250). - The output comprises expanded format records and each expanded format record comprises a value for each field and each record stands independently (unlike RPF which is a comparison with a previous packet). The method of
FIG. 12 (referred to as an RPF reader) hides the compression form the processing engine which performs the network packet traffic analysis. - Whilst the method of decompressing a compressed trace file shown in
FIG. 12 demonstrates decompression of a compressed packet which has been created using all the compression techniques described above, it will be appreciated that in some examples the compressed packet may have been created using only a subset of the techniques described above. In such an example, a corresponding decompression method may comprise only a subset of the steps shown inFIG. 12 . -
FIG. 13 illustrates various components of an exemplary computing-baseddevice 1300 which may be implemented as any form of a computing and/or electronic device, and in which embodiments of the methods described above may be implemented. - Computing-based
device 1300 comprises one ormore processors 1301 which may be microprocessors, controllers or any other suitable type of processors for processing computing executable instructions to control the operation of the device in order to generate and/or process packet traces. Platform software comprising anoperating system 1302 or any other suitable platform software may be provided at the computing-based device to enable application software 1303-1305 to be executed on the device. The application software may comprise a RPF writer 1304 (e.g. which performs one or more of the methods shown inFIGS. 3-9 and 11) and/or a RPF reader (e.g. which performs the method shown inFIG. 12 ). - The computer executable instructions may be provided using any computer-readable media, such as
memory 1306. The memory is of any suitable type such as random access memory (RAM), a disk storage device of any type such as a magnetic or optical storage device, a hard disk drive, or a CD, DVD or other disc drive. Flash memory, EPROM or EEPROM may also be used. The memory may also be used to provide adata store 1307 which may, for example, be used to store the compressed trace files and/or the decompressed trace files. - The computing-based device 1200 also comprises a
network interface 1308 for receiving packets and may also comprise additional inputs and outputs (not shown inFIG. 13 ). - Although the present examples are described and illustrated herein as being implemented in a system as shown in
FIG. 1 , the system described is provided as an example and not a limitation. As those skilled in the art will appreciate, the present examples are suitable for application in a variety of different types of systems which comprise more than one computing device and which may be interconnected in any way. - The term ‘computer’ is used herein to refer to any device with processing capability such that it can execute instructions. Those skilled in the art will realize that such processing capabilities are incorporated into many different devices and therefore the term ‘computer’ includes PCs, servers, mobile telephones, personal digital assistants and many other devices.
- The methods described herein may be performed by software in machine readable form on a tangible storage medium. The software can be suitable for execution on a parallel processor or a serial processor such that the method steps may be carried out in any suitable order, or simultaneously.
- This acknowledges that software can be a valuable, separately tradable commodity. It is intended to encompass software, which runs on or controls “dumb” or standard hardware, to carry out the desired functions. It is also intended to encompass software which “describes” or defines the configuration of hardware, such as HDL (hardware description language) software, as is used for designing silicon chips, or for configuring universal programmable chips, to carry out desired functions.
- Those skilled in the art will realize that storage devices utilized to store program instructions can be distributed across a network. For example, a remote computer may store an example of the process described as software. A local or terminal computer may access the remote computer and download a part or all of the software to run the program. Alternatively, the local computer may download pieces of the software as needed, or execute some software instructions at the local terminal and some at the remote computer (or computer network). Those skilled in the art will also realize that by utilizing conventional techniques known to those skilled in the art that all, or a portion of the software instructions may be carried out by a dedicated circuit, such as a DSP, programmable logic array, or the like.
- Any range or device value given herein may be extended or altered without losing the effect sought, as will be apparent to the skilled person.
- It will be understood that the benefits and advantages described above may relate to one embodiment or may relate to several embodiments. The embodiments are not limited to those that solve any or all of the stated problems or those that have any or all of the stated benefits and advantages. It will further be understood that reference to ‘an’ item refers to one or more of those items.
- The steps of the methods described herein may be carried out in any suitable order, or simultaneously where appropriate. Additionally, individual blocks may be deleted from any of the methods without departing from the spirit and scope of the subject matter described herein. Aspects of any of the examples described above may be combined with aspects of any of the other examples described to form further examples without losing the effect sought.
- The term ‘comprising’ is used herein to mean including the method blocks or elements identified, but that such blocks or elements do not comprise an exclusive list and a method or apparatus may contain additional blocks or elements.
- It will be understood that the above description of a preferred embodiment is given by way of example only and that various modifications may be made by those skilled in the art. The above specification, examples and data provide a complete description of the structure and use of exemplary embodiments of the invention. Although various embodiments of the invention have been described above with a certain degree of particularity, or with reference to one or more individual embodiments, those skilled in the art could make numerous alterations to the disclosed embodiments without departing from the spirit or scope of this invention.
Claims (20)
1. A method of capturing trace data for use in network packet traffic analysis, the method comprising:
under control of one or more processors configured with executable instructions:
receiving a packet; and
storing a representation of the packet in a file, said representation and said file being associated with one of a source address of the packet and a destination address of the packet.
2. A method according to claim 1 , further comprising:
storing a second representation of the packet in a second file, said second representation and said second file being associated with another of said source address of the packet and said destination address of the packet.
3. A method according to claim 2 , wherein the first representation and the second representation are the same.
4. A method according to claim 1 , wherein a representation of a packet comprises a truncated packet.
5. A method according to claim 1 , wherein a representation of a packet comprises values of specified fields.
6. A method according to claim 1 , wherein storing a representation of the packet in a file comprises:
if the file associated with an address is empty, creating and storing the associated representation;
if the file associated with an address comprises a representation of a previous packet, creating the associated representation based on a comparison of said packet and said previous packet and storing the associated representation.
7. A method according to claim 6 , wherein creating the associated representation based on a comparison of said packet and said previous packet comprises:
comparing a value of a first field in said packet with a value of said first field in said previous packet;
setting a bit in the representation identifying if the values are the same; and
repeating the comparing and setting steps for each field in a set of specified fields; and
appending the value of each field where the value of the field in said packet is not the same as the value of the field in the previous packet.
8. A method according to claim 1 , wherein storing a representation of the packet in a file comprises:
creating a representation of the packet;
setting a flag in the representation indicating whether the address associated with the representation is the source address or the destination address;
mapping a value of a field in the representation based on said flag; and
storing the representation.
9. A method according to claim 1 , wherein each representation comprises at least one of a source address and a destination address and wherein storing two representations of the packet in separate files comprises, for each representation:
replacing an address in the representation with an identifier;
storing a mapping between the address and the identifier in a separate file; and
storing the representation.
10. A method according to claim 1 , further comprising comparing the representation of the packet to a representation of a previously received packet which has been stored in the file and which sets bits which denote whether fields in the packet are the same as corresponding fields in the previously received packet.
11. One or more tangible device-readable media with device-executable instructions for performing acts comprising:
on receipt of a packet, creating a packet record for use in network packet traffic analysis, the packet record comprising a plurality of fields; and
discarding the packet.
12. One or more tangible device-readable media according to claim 11 , further comprising device-executable instructions for performing acts comprising:
creating a first and second version of the packet record; and
storing the first version in a file associated with a source address of the packet; and
storing the second version of the packet record in a file associated with a destination address of the packet.
13. One or more tangible device-readable media according to claim 12 , wherein creating a first and second version of the packet record further comprises, for each version:
setting a flag in the version of the packet record according to a direction of travel of the packet); and
mapping at least one of the plurality of fields based on said flag.
14. One or more tangible device-readable media according to claim 11 , further comprising device-executable instructions for performing act comprising:
compressing each packet record based on a comparison of the packet and a packet previously stored in the file.
15. One or more tangible device-readable media according to claim 14 , wherein compressing each packet record based on a comparison of the packet and a packet previously stored in the file comprises:
comparing the packet to a previous packet;
setting a flag in the packet record for each of the plurality of fields, the flag indicating whether a value of the field in the packet is different to a value of the field in the previous packet; and
for each field where the value of the field in the packet is different to a value of the field in the previous packet, appending the value of the field to the packet record.
16. One or more tangible device-readable media according to claim 14 , further comprising device-executable instructions for performing acts comprising:
creating an instance of a record creation and compression method for each file;
passing a received packet to at least one of: an instance of the method corresponding to a source address of the packet and an instance of the method corresponding to a source address of the packet.
17. One or more tangible device-readable media according to claim 11 , further comprising device-executable instructions for performing acts comprising:
replacing an address in the packet record with an identifier; and
storing a mapping between the address and the identifier in a dictionary.
18. One or more tangible device-readable media with device-executable instructions for performing acts comprising:
accessing a file comprising a plurality of compressed packet records, each compressed packet record comprising a flag byte;
reading a flag byte from the file; and
generating an uncompressed packet record comprising a packet time and a plurality of fields by:
determining the packet time based on a timestamp within said flag byte; and
determining the plurality of fields based on a plurality of flags in the flag byte, each flag corresponding to one of the plurality of fields.
19. One or more tangible device-readable media according to claim 18 , wherein determining the plurality of fields based on a plurality of flags in the flag byte comprises:
reading a first flag from the flag byte;
if said first flag is set, reading a value of a first field from the file;
if said first flag is not set, setting the value of the first field to a value of a first field in a previous uncompressed packet record; and
repeating the steps for each of the plurality of flags.
20. One or more tangible device-readable media according to claim 18 , wherein determining the packet time based on a timestamp within said flag byte comprises:
examining the timestamp; and
if the timestamp is equal to a first value, reading a time from the file and setting the packet time to said time; and
if the timestamp is equal to one of a set of values, reading one or more bytes from the file and setting the packet time based on said one or more bytes and a packet time of a previous uncompressed packet record.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/848,391 US20100290364A1 (en) | 2008-05-09 | 2010-08-02 | Packet Compression for Network Packet Traffic Analysis |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/118,595 US7793001B2 (en) | 2008-05-09 | 2008-05-09 | Packet compression for network packet traffic analysis |
US12/848,391 US20100290364A1 (en) | 2008-05-09 | 2010-08-02 | Packet Compression for Network Packet Traffic Analysis |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/118,595 Continuation US7793001B2 (en) | 2008-05-09 | 2008-05-09 | Packet compression for network packet traffic analysis |
Publications (1)
Publication Number | Publication Date |
---|---|
US20100290364A1 true US20100290364A1 (en) | 2010-11-18 |
Family
ID=41267687
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/118,595 Active 2029-04-28 US7793001B2 (en) | 2008-05-09 | 2008-05-09 | Packet compression for network packet traffic analysis |
US12/848,391 Abandoned US20100290364A1 (en) | 2008-05-09 | 2010-08-02 | Packet Compression for Network Packet Traffic Analysis |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/118,595 Active 2029-04-28 US7793001B2 (en) | 2008-05-09 | 2008-05-09 | Packet compression for network packet traffic analysis |
Country Status (1)
Country | Link |
---|---|
US (2) | US7793001B2 (en) |
Cited By (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8095774B1 (en) | 2007-07-05 | 2012-01-10 | Silver Peak Systems, Inc. | Pre-fetching data into a memory |
US8171238B1 (en) | 2007-07-05 | 2012-05-01 | Silver Peak Systems, Inc. | Identification of data stored in memory |
US8307115B1 (en) | 2007-11-30 | 2012-11-06 | Silver Peak Systems, Inc. | Network memory mirroring |
US8312226B2 (en) | 2005-08-12 | 2012-11-13 | Silver Peak Systems, Inc. | Network memory appliance for providing data based on local accessibility |
US8392684B2 (en) | 2005-08-12 | 2013-03-05 | Silver Peak Systems, Inc. | Data encryption in a network memory architecture for providing data based on local accessibility |
US8442052B1 (en) | 2008-02-20 | 2013-05-14 | Silver Peak Systems, Inc. | Forward packet recovery |
US8489562B1 (en) | 2007-11-30 | 2013-07-16 | Silver Peak Systems, Inc. | Deferred data storage |
JP2013187685A (en) * | 2012-03-07 | 2013-09-19 | Fujitsu Telecom Networks Ltd | Fault information collection analysis system |
US8743683B1 (en) | 2008-07-03 | 2014-06-03 | Silver Peak Systems, Inc. | Quality of service using multiple flows |
US8755381B2 (en) | 2006-08-02 | 2014-06-17 | Silver Peak Systems, Inc. | Data matching using flow based packet data storage |
US8811431B2 (en) * | 2008-11-20 | 2014-08-19 | Silver Peak Systems, Inc. | Systems and methods for compressing packet data |
US8885632B2 (en) | 2006-08-02 | 2014-11-11 | Silver Peak Systems, Inc. | Communications scheduler |
US8929402B1 (en) | 2005-09-29 | 2015-01-06 | Silver Peak Systems, Inc. | Systems and methods for compressing packet data by predicting subsequent data |
CN104333476A (en) * | 2014-11-26 | 2015-02-04 | 中国电子科技集团公司第三十八研究所 | Ethernet data capturing and analyzing equipment and method, and Ethernet data capturing and analyzing device and method for Ethernet data capturing and analyzing equipment |
US9130991B2 (en) | 2011-10-14 | 2015-09-08 | Silver Peak Systems, Inc. | Processing data packets in performance enhancing proxy (PEP) environment |
CN106374934A (en) * | 2016-08-19 | 2017-02-01 | 中国地质大学(武汉) | Controllable parallel trajectory data compression method |
US9626224B2 (en) | 2011-11-03 | 2017-04-18 | Silver Peak Systems, Inc. | Optimizing available computing resources within a virtual environment |
US9717021B2 (en) | 2008-07-03 | 2017-07-25 | Silver Peak Systems, Inc. | Virtual network overlay |
US9875344B1 (en) | 2014-09-05 | 2018-01-23 | Silver Peak Systems, Inc. | Dynamic monitoring and authorization of an optimization device |
US9948496B1 (en) | 2014-07-30 | 2018-04-17 | Silver Peak Systems, Inc. | Determining a transit appliance for data traffic to a software service |
US9967056B1 (en) | 2016-08-19 | 2018-05-08 | Silver Peak Systems, Inc. | Forward packet recovery with constrained overhead |
US10164861B2 (en) | 2015-12-28 | 2018-12-25 | Silver Peak Systems, Inc. | Dynamic monitoring and visualization for network health characteristics |
US10257082B2 (en) | 2017-02-06 | 2019-04-09 | Silver Peak Systems, Inc. | Multi-level learning for classifying traffic flows |
US10367656B2 (en) | 2015-02-25 | 2019-07-30 | Quantea Corporation | Network traffic system and method of operation thereof |
US10432484B2 (en) | 2016-06-13 | 2019-10-01 | Silver Peak Systems, Inc. | Aggregating select network traffic statistics |
US10637721B2 (en) | 2018-03-12 | 2020-04-28 | Silver Peak Systems, Inc. | Detecting path break conditions while minimizing network overhead |
US10771394B2 (en) | 2017-02-06 | 2020-09-08 | Silver Peak Systems, Inc. | Multi-level learning for classifying traffic flows on a first packet from DNS data |
US10805840B2 (en) | 2008-07-03 | 2020-10-13 | Silver Peak Systems, Inc. | Data transmission via a virtual wide area network overlay |
US10892978B2 (en) | 2017-02-06 | 2021-01-12 | Silver Peak Systems, Inc. | Multi-level learning for classifying traffic flows from first packet data |
US11044202B2 (en) | 2017-02-06 | 2021-06-22 | Silver Peak Systems, Inc. | Multi-level learning for predicting and classifying traffic flows from first packet data |
US11212210B2 (en) | 2017-09-21 | 2021-12-28 | Silver Peak Systems, Inc. | Selective route exporting using source type |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7793001B2 (en) * | 2008-05-09 | 2010-09-07 | Microsoft Corporation | Packet compression for network packet traffic analysis |
US20100103183A1 (en) * | 2008-10-23 | 2010-04-29 | Hung-Ming Lin | Remote multiple image processing apparatus |
US8111704B2 (en) * | 2009-06-26 | 2012-02-07 | Intel Corporation | Multiple compression techniques for packetized information |
US8472449B2 (en) * | 2010-03-02 | 2013-06-25 | Intrusion, Inc. | Packet file system |
US8751450B1 (en) * | 2011-04-27 | 2014-06-10 | Netapp, Inc. | Method and system for securely capturing workloads at a live network for replaying at a test network |
US9973473B2 (en) * | 2012-03-30 | 2018-05-15 | The University Of North Carolina At Chapel Hill | Methods, systems, and computer readable media for rapid filtering of opaque data traffic |
JP6021498B2 (en) * | 2012-08-01 | 2016-11-09 | 任天堂株式会社 | Data compression apparatus, data compression program, data compression system, data compression method, data decompression apparatus, data compression / decompression system, and data structure of compressed data |
US9524227B2 (en) * | 2014-07-09 | 2016-12-20 | Intel Corporation | Apparatuses and methods for generating a suppressed address trace |
GB2568165B (en) * | 2017-10-18 | 2022-08-31 | Frank Donnelly Stephen | Entropy and value based packet truncation |
GB2575246A (en) * | 2018-06-25 | 2020-01-08 | British Telecomm | Processing local area network diagnostic data |
WO2020002158A1 (en) | 2018-06-25 | 2020-01-02 | British Telecommunications Public Limited Company | Processing local area network diagnostic data |
CN109710611B (en) * | 2018-12-25 | 2019-09-17 | 北京三快在线科技有限公司 | The method of storage table data, the method, apparatus of lookup table data and storage medium |
GB2587427A (en) | 2019-09-30 | 2021-03-31 | British Telecomm | Processing local area network diagnostic data |
WO2021063671A1 (en) | 2019-09-30 | 2021-04-08 | British Telecommunications Public Limited Company | Processing local area network diagnostic data |
US11665262B2 (en) * | 2020-10-28 | 2023-05-30 | Viavi Solutions Inc. | Analyzing network data for debugging, performance, and identifying protocol violations using parallel multi-threaded processing |
Citations (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4771391A (en) * | 1986-07-21 | 1988-09-13 | International Business Machines Corporation | Adaptive packet length traffic control in a local area network |
US5978386A (en) * | 1995-01-10 | 1999-11-02 | Nokia Telecommunications Oy | Packet radio system, and a terminal equipment for a packet radio system |
US20010050903A1 (en) * | 2000-01-28 | 2001-12-13 | Paul Vanlint | Method and system to calculate network latency, and to display the same field of the invention |
US6337863B1 (en) * | 1997-01-17 | 2002-01-08 | Alcatel Interworking, Inc. | Seamless communication service with intelligent edge devices |
US20030074476A1 (en) * | 2001-10-05 | 2003-04-17 | Samsung Electronics Co., Ltd | Apparatus and method for transmitting and receiving TBS information in an HSDPA communication system |
US6560206B1 (en) * | 1999-06-21 | 2003-05-06 | Nortel Networks Limited | Cell based data transmission method |
US20040085906A1 (en) * | 2001-04-27 | 2004-05-06 | Hisamichi Ohtani | Packet tracing system |
US20040090989A1 (en) * | 2002-11-08 | 2004-05-13 | Nec Infrontia Corporation | Packet compression system, packet restoration system, packet compression method, and packet restoration method |
US20050270982A1 (en) * | 2004-06-07 | 2005-12-08 | Mcbeath Tom | Method and apparatus for monitoring latency, jitter, packet throughput and packet loss ratio between two points on a network |
US20060053455A1 (en) * | 2002-12-04 | 2006-03-09 | Murali Mani | Synchronization of signals |
US7016351B1 (en) * | 2000-02-29 | 2006-03-21 | Cisco Technology, Inc. | Small group multicast in a computer network |
US20060083180A1 (en) * | 2004-10-19 | 2006-04-20 | Yokogawa Electric Corporation | Packet analysis system |
US20060129689A1 (en) * | 2004-12-10 | 2006-06-15 | Ricky Ho | Reducing the sizes of application layer messages in a network element |
US20070091359A1 (en) * | 2005-10-04 | 2007-04-26 | Sony Corporation | Content transmission device, content transmission method, and computer program used therewith |
US20070186130A1 (en) * | 2004-03-19 | 2007-08-09 | Novo Nordisk A/S | Reduced size transmission data packet header format for a medical device |
US20070211724A1 (en) * | 2006-03-07 | 2007-09-13 | Ingemar Johansson | Communication station and method providing flexible compression of data packets |
US20080025312A1 (en) * | 2006-07-28 | 2008-01-31 | Qualcomm Incorporated | Zero-header compression for improved communications |
US20080089238A1 (en) * | 2006-10-13 | 2008-04-17 | Safe Media, Corp. | Network monitoring and intellectual property protection device, system and method |
US20090232094A1 (en) * | 2008-03-12 | 2009-09-17 | Toshiba Research America, Inc. | Method and function for maintaining mapping between media independent handover functions and transport addresses |
US20090281984A1 (en) * | 2008-05-09 | 2009-11-12 | Microsoft Corporation | Packet Compression for Network Packet Traffic Analysis |
US20090327751A1 (en) * | 2006-05-31 | 2009-12-31 | Haim Koifman | Method and system for transformation of logical data objects for storage |
US7827272B2 (en) * | 2002-11-04 | 2010-11-02 | Riverbed Technology, Inc. | Connection table for intrusion detection |
US7899323B2 (en) * | 2007-09-28 | 2011-03-01 | Verizon Patent And Licensing Inc. | Multi-interface protocol analysis system |
-
2008
- 2008-05-09 US US12/118,595 patent/US7793001B2/en active Active
-
2010
- 2010-08-02 US US12/848,391 patent/US20100290364A1/en not_active Abandoned
Patent Citations (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4771391A (en) * | 1986-07-21 | 1988-09-13 | International Business Machines Corporation | Adaptive packet length traffic control in a local area network |
US5978386A (en) * | 1995-01-10 | 1999-11-02 | Nokia Telecommunications Oy | Packet radio system, and a terminal equipment for a packet radio system |
US6337863B1 (en) * | 1997-01-17 | 2002-01-08 | Alcatel Interworking, Inc. | Seamless communication service with intelligent edge devices |
US6560206B1 (en) * | 1999-06-21 | 2003-05-06 | Nortel Networks Limited | Cell based data transmission method |
US20010050903A1 (en) * | 2000-01-28 | 2001-12-13 | Paul Vanlint | Method and system to calculate network latency, and to display the same field of the invention |
US7016351B1 (en) * | 2000-02-29 | 2006-03-21 | Cisco Technology, Inc. | Small group multicast in a computer network |
US20040085906A1 (en) * | 2001-04-27 | 2004-05-06 | Hisamichi Ohtani | Packet tracing system |
US20030074476A1 (en) * | 2001-10-05 | 2003-04-17 | Samsung Electronics Co., Ltd | Apparatus and method for transmitting and receiving TBS information in an HSDPA communication system |
US7827272B2 (en) * | 2002-11-04 | 2010-11-02 | Riverbed Technology, Inc. | Connection table for intrusion detection |
US20040090989A1 (en) * | 2002-11-08 | 2004-05-13 | Nec Infrontia Corporation | Packet compression system, packet restoration system, packet compression method, and packet restoration method |
US20060053455A1 (en) * | 2002-12-04 | 2006-03-09 | Murali Mani | Synchronization of signals |
US20070186130A1 (en) * | 2004-03-19 | 2007-08-09 | Novo Nordisk A/S | Reduced size transmission data packet header format for a medical device |
US20050270982A1 (en) * | 2004-06-07 | 2005-12-08 | Mcbeath Tom | Method and apparatus for monitoring latency, jitter, packet throughput and packet loss ratio between two points on a network |
US20060083180A1 (en) * | 2004-10-19 | 2006-04-20 | Yokogawa Electric Corporation | Packet analysis system |
US20060129689A1 (en) * | 2004-12-10 | 2006-06-15 | Ricky Ho | Reducing the sizes of application layer messages in a network element |
US20070091359A1 (en) * | 2005-10-04 | 2007-04-26 | Sony Corporation | Content transmission device, content transmission method, and computer program used therewith |
US20070211724A1 (en) * | 2006-03-07 | 2007-09-13 | Ingemar Johansson | Communication station and method providing flexible compression of data packets |
US20090327751A1 (en) * | 2006-05-31 | 2009-12-31 | Haim Koifman | Method and system for transformation of logical data objects for storage |
US20080025312A1 (en) * | 2006-07-28 | 2008-01-31 | Qualcomm Incorporated | Zero-header compression for improved communications |
US20080089238A1 (en) * | 2006-10-13 | 2008-04-17 | Safe Media, Corp. | Network monitoring and intellectual property protection device, system and method |
US7899323B2 (en) * | 2007-09-28 | 2011-03-01 | Verizon Patent And Licensing Inc. | Multi-interface protocol analysis system |
US20090232094A1 (en) * | 2008-03-12 | 2009-09-17 | Toshiba Research America, Inc. | Method and function for maintaining mapping between media independent handover functions and transport addresses |
US20090281984A1 (en) * | 2008-05-09 | 2009-11-12 | Microsoft Corporation | Packet Compression for Network Packet Traffic Analysis |
Cited By (79)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8732423B1 (en) | 2005-08-12 | 2014-05-20 | Silver Peak Systems, Inc. | Data encryption in a network memory architecture for providing data based on local accessibility |
US9363248B1 (en) | 2005-08-12 | 2016-06-07 | Silver Peak Systems, Inc. | Data encryption in a network memory architecture for providing data based on local accessibility |
US8312226B2 (en) | 2005-08-12 | 2012-11-13 | Silver Peak Systems, Inc. | Network memory appliance for providing data based on local accessibility |
US8370583B2 (en) | 2005-08-12 | 2013-02-05 | Silver Peak Systems, Inc. | Network memory architecture for providing data based on local accessibility |
US8392684B2 (en) | 2005-08-12 | 2013-03-05 | Silver Peak Systems, Inc. | Data encryption in a network memory architecture for providing data based on local accessibility |
US10091172B1 (en) | 2005-08-12 | 2018-10-02 | Silver Peak Systems, Inc. | Data encryption in a network memory architecture for providing data based on local accessibility |
US9549048B1 (en) * | 2005-09-29 | 2017-01-17 | Silver Peak Systems, Inc. | Transferring compressed packet data over a network |
US9363309B2 (en) | 2005-09-29 | 2016-06-07 | Silver Peak Systems, Inc. | Systems and methods for compressing packet data by predicting subsequent data |
US9036662B1 (en) * | 2005-09-29 | 2015-05-19 | Silver Peak Systems, Inc. | Compressing packet data |
US9712463B1 (en) | 2005-09-29 | 2017-07-18 | Silver Peak Systems, Inc. | Workload optimization in a wide area network utilizing virtual switches |
US8929402B1 (en) | 2005-09-29 | 2015-01-06 | Silver Peak Systems, Inc. | Systems and methods for compressing packet data by predicting subsequent data |
US9961010B2 (en) | 2006-08-02 | 2018-05-01 | Silver Peak Systems, Inc. | Communications scheduler |
US8929380B1 (en) | 2006-08-02 | 2015-01-06 | Silver Peak Systems, Inc. | Data matching using flow based packet data storage |
US9191342B2 (en) | 2006-08-02 | 2015-11-17 | Silver Peak Systems, Inc. | Data matching using flow based packet data storage |
US9438538B2 (en) | 2006-08-02 | 2016-09-06 | Silver Peak Systems, Inc. | Data matching using flow based packet data storage |
US8755381B2 (en) | 2006-08-02 | 2014-06-17 | Silver Peak Systems, Inc. | Data matching using flow based packet data storage |
US9584403B2 (en) | 2006-08-02 | 2017-02-28 | Silver Peak Systems, Inc. | Communications scheduler |
US8885632B2 (en) | 2006-08-02 | 2014-11-11 | Silver Peak Systems, Inc. | Communications scheduler |
US8095774B1 (en) | 2007-07-05 | 2012-01-10 | Silver Peak Systems, Inc. | Pre-fetching data into a memory |
US8473714B2 (en) | 2007-07-05 | 2013-06-25 | Silver Peak Systems, Inc. | Pre-fetching data into a memory |
US9092342B2 (en) | 2007-07-05 | 2015-07-28 | Silver Peak Systems, Inc. | Pre-fetching data into a memory |
US9152574B2 (en) | 2007-07-05 | 2015-10-06 | Silver Peak Systems, Inc. | Identification of non-sequential data stored in memory |
US8738865B1 (en) | 2007-07-05 | 2014-05-27 | Silver Peak Systems, Inc. | Identification of data stored in memory |
US9253277B2 (en) | 2007-07-05 | 2016-02-02 | Silver Peak Systems, Inc. | Pre-fetching stored data from a memory |
US8225072B2 (en) | 2007-07-05 | 2012-07-17 | Silver Peak Systems, Inc. | Pre-fetching data into a memory |
US8171238B1 (en) | 2007-07-05 | 2012-05-01 | Silver Peak Systems, Inc. | Identification of data stored in memory |
US8489562B1 (en) | 2007-11-30 | 2013-07-16 | Silver Peak Systems, Inc. | Deferred data storage |
US9613071B1 (en) | 2007-11-30 | 2017-04-04 | Silver Peak Systems, Inc. | Deferred data storage |
US8595314B1 (en) | 2007-11-30 | 2013-11-26 | Silver Peak Systems, Inc. | Deferred data storage |
US8307115B1 (en) | 2007-11-30 | 2012-11-06 | Silver Peak Systems, Inc. | Network memory mirroring |
US8442052B1 (en) | 2008-02-20 | 2013-05-14 | Silver Peak Systems, Inc. | Forward packet recovery |
US9717021B2 (en) | 2008-07-03 | 2017-07-25 | Silver Peak Systems, Inc. | Virtual network overlay |
US11419011B2 (en) | 2008-07-03 | 2022-08-16 | Hewlett Packard Enterprise Development Lp | Data transmission via bonded tunnels of a virtual wide area network overlay with error correction |
US10805840B2 (en) | 2008-07-03 | 2020-10-13 | Silver Peak Systems, Inc. | Data transmission via a virtual wide area network overlay |
US9143455B1 (en) | 2008-07-03 | 2015-09-22 | Silver Peak Systems, Inc. | Quality of service using multiple flows |
US9397951B1 (en) | 2008-07-03 | 2016-07-19 | Silver Peak Systems, Inc. | Quality of service using multiple flows |
US11412416B2 (en) | 2008-07-03 | 2022-08-09 | Hewlett Packard Enterprise Development Lp | Data transmission via bonded tunnels of a virtual wide area network overlay |
US10313930B2 (en) | 2008-07-03 | 2019-06-04 | Silver Peak Systems, Inc. | Virtual wide area network overlays |
US8743683B1 (en) | 2008-07-03 | 2014-06-03 | Silver Peak Systems, Inc. | Quality of service using multiple flows |
US8811431B2 (en) * | 2008-11-20 | 2014-08-19 | Silver Peak Systems, Inc. | Systems and methods for compressing packet data |
US9130991B2 (en) | 2011-10-14 | 2015-09-08 | Silver Peak Systems, Inc. | Processing data packets in performance enhancing proxy (PEP) environment |
US9906630B2 (en) | 2011-10-14 | 2018-02-27 | Silver Peak Systems, Inc. | Processing data packets in performance enhancing proxy (PEP) environment |
US9626224B2 (en) | 2011-11-03 | 2017-04-18 | Silver Peak Systems, Inc. | Optimizing available computing resources within a virtual environment |
JP2013187685A (en) * | 2012-03-07 | 2013-09-19 | Fujitsu Telecom Networks Ltd | Fault information collection analysis system |
US10812361B2 (en) | 2014-07-30 | 2020-10-20 | Silver Peak Systems, Inc. | Determining a transit appliance for data traffic to a software service |
US9948496B1 (en) | 2014-07-30 | 2018-04-17 | Silver Peak Systems, Inc. | Determining a transit appliance for data traffic to a software service |
US11381493B2 (en) | 2014-07-30 | 2022-07-05 | Hewlett Packard Enterprise Development Lp | Determining a transit appliance for data traffic to a software service |
US11374845B2 (en) | 2014-07-30 | 2022-06-28 | Hewlett Packard Enterprise Development Lp | Determining a transit appliance for data traffic to a software service |
US11868449B2 (en) | 2014-09-05 | 2024-01-09 | Hewlett Packard Enterprise Development Lp | Dynamic monitoring and authorization of an optimization device |
US11921827B2 (en) | 2014-09-05 | 2024-03-05 | Hewlett Packard Enterprise Development Lp | Dynamic monitoring and authorization of an optimization device |
US9875344B1 (en) | 2014-09-05 | 2018-01-23 | Silver Peak Systems, Inc. | Dynamic monitoring and authorization of an optimization device |
US10719588B2 (en) | 2014-09-05 | 2020-07-21 | Silver Peak Systems, Inc. | Dynamic monitoring and authorization of an optimization device |
US10885156B2 (en) | 2014-09-05 | 2021-01-05 | Silver Peak Systems, Inc. | Dynamic monitoring and authorization of an optimization device |
US11954184B2 (en) | 2014-09-05 | 2024-04-09 | Hewlett Packard Enterprise Development Lp | Dynamic monitoring and authorization of an optimization device |
CN104333476A (en) * | 2014-11-26 | 2015-02-04 | 中国电子科技集团公司第三十八研究所 | Ethernet data capturing and analyzing equipment and method, and Ethernet data capturing and analyzing device and method for Ethernet data capturing and analyzing equipment |
US10367656B2 (en) | 2015-02-25 | 2019-07-30 | Quantea Corporation | Network traffic system and method of operation thereof |
US10771370B2 (en) | 2015-12-28 | 2020-09-08 | Silver Peak Systems, Inc. | Dynamic monitoring and visualization for network health characteristics |
US10164861B2 (en) | 2015-12-28 | 2018-12-25 | Silver Peak Systems, Inc. | Dynamic monitoring and visualization for network health characteristics |
US11336553B2 (en) | 2015-12-28 | 2022-05-17 | Hewlett Packard Enterprise Development Lp | Dynamic monitoring and visualization for network health characteristics of network device pairs |
US11757739B2 (en) | 2016-06-13 | 2023-09-12 | Hewlett Packard Enterprise Development Lp | Aggregation of select network traffic statistics |
US11757740B2 (en) | 2016-06-13 | 2023-09-12 | Hewlett Packard Enterprise Development Lp | Aggregation of select network traffic statistics |
US10432484B2 (en) | 2016-06-13 | 2019-10-01 | Silver Peak Systems, Inc. | Aggregating select network traffic statistics |
US11601351B2 (en) | 2016-06-13 | 2023-03-07 | Hewlett Packard Enterprise Development Lp | Aggregation of select network traffic statistics |
US11424857B2 (en) | 2016-08-19 | 2022-08-23 | Hewlett Packard Enterprise Development Lp | Forward packet recovery with constrained network overhead |
CN106374934A (en) * | 2016-08-19 | 2017-02-01 | 中国地质大学(武汉) | Controllable parallel trajectory data compression method |
US9967056B1 (en) | 2016-08-19 | 2018-05-08 | Silver Peak Systems, Inc. | Forward packet recovery with constrained overhead |
US10848268B2 (en) | 2016-08-19 | 2020-11-24 | Silver Peak Systems, Inc. | Forward packet recovery with constrained network overhead |
US10326551B2 (en) | 2016-08-19 | 2019-06-18 | Silver Peak Systems, Inc. | Forward packet recovery with constrained network overhead |
US11729090B2 (en) | 2017-02-06 | 2023-08-15 | Hewlett Packard Enterprise Development Lp | Multi-level learning for classifying network traffic flows from first packet data |
US10257082B2 (en) | 2017-02-06 | 2019-04-09 | Silver Peak Systems, Inc. | Multi-level learning for classifying traffic flows |
US11582157B2 (en) | 2017-02-06 | 2023-02-14 | Hewlett Packard Enterprise Development Lp | Multi-level learning for classifying traffic flows on a first packet from DNS response data |
US11044202B2 (en) | 2017-02-06 | 2021-06-22 | Silver Peak Systems, Inc. | Multi-level learning for predicting and classifying traffic flows from first packet data |
US10892978B2 (en) | 2017-02-06 | 2021-01-12 | Silver Peak Systems, Inc. | Multi-level learning for classifying traffic flows from first packet data |
US10771394B2 (en) | 2017-02-06 | 2020-09-08 | Silver Peak Systems, Inc. | Multi-level learning for classifying traffic flows on a first packet from DNS data |
US11212210B2 (en) | 2017-09-21 | 2021-12-28 | Silver Peak Systems, Inc. | Selective route exporting using source type |
US11805045B2 (en) | 2017-09-21 | 2023-10-31 | Hewlett Packard Enterprise Development Lp | Selective routing |
US11405265B2 (en) | 2018-03-12 | 2022-08-02 | Hewlett Packard Enterprise Development Lp | Methods and systems for detecting path break conditions while minimizing network overhead |
US10637721B2 (en) | 2018-03-12 | 2020-04-28 | Silver Peak Systems, Inc. | Detecting path break conditions while minimizing network overhead |
US10887159B2 (en) | 2018-03-12 | 2021-01-05 | Silver Peak Systems, Inc. | Methods and systems for detecting path break conditions while minimizing network overhead |
Also Published As
Publication number | Publication date |
---|---|
US7793001B2 (en) | 2010-09-07 |
US20090281984A1 (en) | 2009-11-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7793001B2 (en) | Packet compression for network packet traffic analysis | |
EP2608096B1 (en) | Compression of genomic data file | |
JP5566888B2 (en) | Optimized data stream compression using data-dependent chunking | |
US7643505B1 (en) | Method and system for real time compression and decompression | |
US8497788B1 (en) | Efficient techniques for aligned fixed-length compression | |
US20120182891A1 (en) | Packet analysis system and method using hadoop based parallel computation | |
US8666985B2 (en) | Hardware accelerated application-based pattern matching for real time classification and recording of network traffic | |
US20190052553A1 (en) | Architectures and methods for deep packet inspection using alphabet and bitmap-based compression | |
KR101969848B1 (en) | Method and apparatus for compressing genetic data | |
CN113162818A (en) | Method and system for realizing distributed flow acquisition and analysis | |
US11762557B2 (en) | System and method for data compaction and encryption of anonymized datasets | |
US20230216520A1 (en) | System and method for data compression with encryption | |
CN111585832A (en) | Industrial control protocol reverse analysis method based on semantic pre-mining | |
WO2018151786A1 (en) | Method and systems for the reconstruction of genomic reference sequences from compressed genomic sequence reads | |
Kim et al. | PcapWT: An efficient packet extraction tool for large volume network traces | |
Cánovas et al. | Practical compression for multi-alignment genomic files | |
KR101218087B1 (en) | Method for Extracting InputFormat for Binary Format Data in Hadoop MapReduce and Binary Data Analysis Using the Same | |
KR101200773B1 (en) | Method for Extracting InputFormat for Handling Network Packet Data on Hadoop MapReduce | |
JP7239016B2 (en) | Sorting device, sorting method, sorting program | |
US8593310B1 (en) | Data-driven variable length encoding of fixed-length data | |
US11734012B2 (en) | Systems and methods for efficient transfer of log data | |
US11811428B2 (en) | System and method for data compression using genomic encryption techniques | |
US20240072825A1 (en) | System and method for encrypted data compression | |
WO2022176035A1 (en) | Conversion device, conversion method, and conversion program | |
Konow et al. | PcapWT: An Efficient Packet Extraction Tool for Large Volume Network Traces |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MICROSOFT CORPORATION, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BLACK, RICHARD JOHN;REEL/FRAME:024782/0087 Effective date: 20080507 |
|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034544/0001 Effective date: 20141014 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |