CN104199790A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN104199790A CN104199790A CN201410415527.0A CN201410415527A CN104199790A CN 104199790 A CN104199790 A CN 104199790A CN 201410415527 A CN201410415527 A CN 201410415527A CN 104199790 A CN104199790 A CN 104199790A
- Authority
- CN
- China
- Prior art keywords
- queue
- data
- pending data
- level cache
- priority
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The embodiment of the invention discloses a data processing method and device. The data processing method comprises the steps that a data production module produces at least one datum to be processed; the priorities are set for the data to be processed; the data to be processed are written in one-level buffer queues, corresponding to the priorities, of preset one-level buffer queue clusters according to the priorities of the data to be processed; according to the queue polling mode, a preset number of data which are to be processed and correspond to corresponding one-level buffer queues are extracted from the one-level buffer queues in all the one-level buffer queue clusters till all the data to be processed are extracted; the to-be-processed data extracted through polling every time are written in a preset two-level buffer queue in sequence. Obviously, by means of the scheme, the data with the high propriety can be processed as fast as possible rather than be blocked in the first-in first-out queues to wait for the data in front to be processed.
Description
Technical field
The present invention relates to data processing method, particularly a kind of data processing method and device.
Background technology
In the process of systemic software development, often can encounter following scene: certain module is responsible for producing data, and another module is responsible for deal with data.Wherein, the module of generation data is commonly called data production module (or producer); And the module of deal with data is commonly called data processing module (or consumer).In practical application, the type of data production module and data processing module is specifically as follows operation system, class, function, thread or process etc.
Produce the data processing of action and data processing module for the data of equilibrium criterion production module and move, most critical be to there is a buffer queue between data producer and data processor.Concrete, generated pending data are write buffer queue by data production module, and data processing module extracts data and processes from buffer queue.Wherein, buffer queue can be single obstruction queue or Circular buffer queue, and buffer queue is all the queue form of FIFO first in first out conventionally.
Wherein, in the time that deal with data has priority demand, normally the data in buffer zone are done to the rearrangement based on priority, to make the high priority data of high priority processed in the data of low priority.But, under some scene, not strict to the requirement of priority, do not need to force the data of high priority necessarily processed prior to the data of low priority, just require the data of high priority processed as soon as possible, instead of be blocked in the queue of first in first out and wait for that data are above processed complete.
Summary of the invention
Based on the problems referred to above, the embodiment of the invention discloses a kind of data processing method and device, be not blocked in to ensure the data of high priority to be processed as early as possible in the queue of first in first out and wait for that data are above processed complete.Technical scheme is as follows:
First aspect, the embodiment of the present invention provides a kind of data processing method, is applied to data handling system, and wherein, described data handling system comprises for the production of the data production module of data with for the treatment of the data processing module of data; Described method comprises:
Described data production module is produced at least one pending data;
For described at least one pending data arrange priority;
According to the priority of described at least one pending data, described at least one pending data are write to the level cache queue in the default level cache collection of queues corresponding with priority; Wherein, the pending data of described level cache collection of queues for placing same priority;
According to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until described at least one pending data are all extracted out;
Successively the pending data that poll extracts are each time write to default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
Optionally, described level cache queue is single queue of stopping up;
The described priority according to described at least one pending data, writes the level cache queue in the default level cache collection of queues corresponding with priority by described at least one pending data, comprising:
According to the priority of described at least one pending data, described at least one pending data are write to the afterbody of level cache queue in the default level cache collection of queues corresponding with priority;
Described according to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, comprising:
According to queue polling mode, the head in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively.
Optionally, described level cache queue is Circular buffer queue;
The described priority according to described at least one pending data, writes the level cache queue in the default level cache collection of queues corresponding with priority by described at least one pending data, comprising:
According to the priority of described at least one pending data, described at least one pending data are write to the head of level cache queue in the default level cache collection of queues corresponding with priority;
Described according to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, comprising:
According to queue polling mode, the afterbody in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively.
Optionally, described L2 cache queue is single queue of stopping up;
Describedly successively the pending data that poll extracts are each time write to default L2 cache queue, comprising:
Successively the pending data that poll extracts are each time write to the afterbody of default L2 cache queue.
Optionally, described L2 cache queue is Circular buffer queue;
Describedly successively the pending data that poll extracts are each time write to default L2 cache queue, comprising:
Successively the pending data that poll extracts are each time write to the head of default L2 cache queue.
Optionally, the described pending data order of successively poll each time being extracted writes default L2 cache queue, comprising:
Successively the pending data that poll extracts are each time write to default L2 cache queue according to priority order from high to low.
Second aspect, the embodiment of the present invention also provides a kind of data handling system, and described data handling system comprises that described data production module comprises for the production of the data production module of data with for the data processing module of consumption data:
Productive unit, for the production of at least one pending data;
Priority setting unit, is used to described at least one pending data that priority is set;
One-level queue writing unit, for the priority according to described at least one pending data, writes the level cache queue in the default level cache collection of queues corresponding with priority by described at least one pending data; Wherein, the pending data of described level cache collection of queues for placing same priority;
Data extracting unit, be used for according to queue polling mode, in level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until described at least one pending data are all extracted out;
Secondary queue writing unit, for successively the pending data that poll extracts each time being write to default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
Optionally, described level cache queue is single queue of stopping up;
Described one-level queue writing unit, specifically for:
According to the priority of described at least one pending data, described at least one pending data are write to the afterbody of level cache queue in the default level cache collection of queues corresponding with priority;
Described data extracting unit, specifically for:
According to queue polling mode, the head in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively, until described at least one pending data are all extracted out.
Optionally, described level cache queue is Circular buffer queue;
Described one-level queue writing unit, specifically for:
According to the priority of described at least one pending data, described at least one pending data are write to the head of level cache queue in the default level cache collection of queues corresponding with priority;
Described data extracting unit, specifically for:
According to queue polling mode, the afterbody in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively, until described at least one pending data are all extracted out.
Optionally, described L2 cache queue is single queue of stopping up;
Described secondary queue writing unit, specifically for:
Successively the pending data that poll extracts are each time write to the afterbody of default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
Optionally, described L2 cache queue is Circular buffer queue;
Described secondary queue writing unit, specifically for:
Successively the pending data that poll extracts are each time write to the head of default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
Optionally, described secondary queue writing unit, specifically for:
Successively the pending data that poll extracts are each time write to default L2 cache queue according to priority order from high to low so that data processing module extracts pending data and processes from described L2 cache queue.
In the embodiment of the present invention, data production module is produced at least one pending data; For at least one pending data arranges priority; According to the priority of at least one pending data, at least one pending data is write to the level cache queue in the level cache collection of queues corresponding with priority; According to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until at least one pending data is all extracted out; Successively the pending data that poll extracts are each time write to default L2 cache queue so that data processing module extracts pending data and processes from L2 cache queue.Visible, in this programme, the pending data that are at every turn polled to from level cache queue corresponding to different priorities are write to L2 cache queue, the pending data of different priorities are all processed, the data that therefore, can ensure high priority can be processed as early as possible and not be blocked in the queue of first in first out and wait for that data are above processed complete.
Brief description of the drawings
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, to the accompanying drawing of required use in embodiment or description of the Prior Art be briefly described below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, do not paying under the prerequisite of creative work, can also obtain according to these accompanying drawings other accompanying drawing.
The process flow diagram of a kind of data processing method that Fig. 1 provides for the embodiment of the present invention;
The structural representation of a kind of data handling system that Fig. 2 provides for the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiment.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtaining under creative work prerequisite, belong to the scope of protection of the invention.
The embodiment of the present invention provides a kind of data processing method and device, is not blocked in the queue of first in first out waits for that data are above processed complete to ensure the data of high priority to be processed as early as possible.
First a kind of data processing method embodiment of the present invention being provided is below introduced.
It should be noted that, the data processing method that the embodiment of the present invention provides is applied to data handling system, and wherein, this data handling system comprises for the production of the data production module of data with for the treatment of the data processing module of data; In practical application, the type of data production module and data processing module is specifically as follows operation system, class, function, thread or process etc.
And, set in advance two-stage queue: for place same priority pending data level cache collection of queues and for placing the L2 cache queue of pending data of different priorities, wherein, this level cache collection of queues comprises at least one level cache queue.In actual applications, level cache queue and L2 cache queue can be all single stop up queue or Circular buffer queue, are certainly not limited to this.
Be understandable that, the quantity of level cache collection of queues can be identical with the quantity of priority; And, for single obstruction queue, from afterbody data writing, extract from the head data; And for Circular buffer queue, data writing from the head, extracts data from afterbody; And, no matter be the principle that single obstruction queue or Circular buffer queue all meet first in first out.
As shown in Figure 1, a kind of data processing method, can comprise:
S101, this data production module is produced at least one pending data;
S102, at least one pending data arranges priority;
Wherein, produce after at least one pending data in data production module, can priority be set for these at least one pending data, to distinguish the processing urgency of different pending data.
S103, according to the priority of at least one pending data, writes the level cache queue in the default level cache collection of queues corresponding with priority by least one pending data;
Wherein, be equipped with after priority in pending data, can, according to the priority of at least one pending data, at least one pending data be write to the level cache queue in the default level cache collection of queues corresponding with priority.
It should be noted that, level cache collection of queues can comprise a level cache queue, i.e. a corresponding level cache queue of priority; Or level cache collection of queues can comprise at least two level cache queues, a priority is to there being at least two level cache queues.For the situation of the corresponding level cache queue of a priority, directly the pending data with same priority are written in the level cache queue corresponding with this priority.And for the situation of at least two level cache queues of a priority correspondence, the pending data with same priority can be write in the different level cache queues corresponding from this priority successively according to the default rule that writes, for example: in the mode according to write pending data at every turn, the pending data polling with same priority is write in the different level cache queues corresponding from this priority, or, be divided at random after multiple groups the pending data in every group are write to the corresponding corresponding level cache queue of this priority thering are the pending data of same priority, etc..
For example, the priority of pending data comprises: p1, p2, p3 ... pn, the pending data of priority p1 can write the level cache queue Q1 in corresponding level cache collection of queues, the pending data of priority p2 can write the level cache queue Q2 in corresponding level cache collection of queues, by that analogy, the pending data of priority pn write the level cache queue Qn in corresponding level cache collection of queues.If the data of certain priority px do not exist, do not write the level cache queue Qx in corresponding level cache collection of queues.
S104, according to queue polling mode, extracts the pending data of the predetermined number corresponding with corresponding level cache queue, successively until at least one pending data is all extracted out in the level cache queue from each level cache collection of queues;
At least one pending data is write after the level cache queue in level cache collection of queues, can be according to queue polling mode, in level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until described at least one pending data are all extracted out, the priority that ensures the pending data that each poll extracted with this comprises the priority that in level cache queue, current all pending data have.Be understandable that, by the mode of queue poll, constantly take out the pending data in each level cache queue, and due to poll each time, the pending data of some are taken out in capital, therefore,, if there are pending data in the level cache queue of high priority, must in current poll, be got.
Concrete, for level cache queue is single situation of stopping up queue, can, according to the priority of at least one pending data, at least one pending data be write to the afterbody of level cache queue in the default level cache collection of queues corresponding with priority; Accordingly, can be according to queue polling mode, the head in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively.
Concrete, for level cache queue is Circular buffer queue, can, according to the priority of at least one pending data, at least one pending data be write to the head of level cache queue in the level cache collection of queues corresponding with priority; Accordingly, according to queue polling mode, the afterbody in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively.
And, it should be noted that, under the prerequisite of once not exhausted in the data of low priority that ensure level cache queue:
When each level cache collection of queues comprises a level cache queue, predetermined number can be one, two or three etc.;
Or,
When each level cache collection of queues comprises at least one level cache queue, predetermined number can be one, two or three etc.;
Or,
When each level cache collection of queues comprises a level cache queue, predetermined number can be the quantity of determining according to empirical value method or dynamic computation method.
Wherein, the situation that comprises at least one level cache queue for each level cache collection of queues can be for example: the level cache queue Q1 in level cache collection of queues corresponding to priority p1 comprises n1 queue, level cache queue Q2 in level cache collection of queues corresponding to priority p2 comprises n2 queue, level cache queue Q2 in level cache collection of queues corresponding to priority p3 comprises n3 queue, now, can extract the data of n1 priority p1 at every turn, the data of n2 priority p2, the data of n3 priority p3, can be processed to the data that ensure each priority, and can not be blocked and clog.
And, it should be noted that, empirical value method estimates described predetermined number according to the priority of pending data and corresponding data amount size; The response speed that dynamic computation require according to a certain priority data and actual data bulk size, calculate dynamically, will inevitably be processed in the data of data processing module processing some with the data that ensure high priority.
S105, writes default L2 cache queue by the pending data that poll extracts each time successively so that data processing module extracts pending data and processes from L2 cache queue.
After end of polling(EOP) each time, or, after all pending data are extracted out, not be blocked in order to ensure the data of high priority to be processed as early as possible in the queue of first in first out and wait for that data are above processed complete, successively the pending data that poll extracts are each time write to default L2 cache queue so that data processing module extracts pending data and processes from this L2 cache queue.Concrete, the pending data that poll extracts each time can be write to default L2 cache queue according to priority order from high to low; Certainly, also can will after the pending data random alignment that poll extracts each time, write successively in default L2 cache queue, this is all rational.
Concrete, can, for for single situation of stopping up queue, can successively the pending data that poll extracts each time be write to the afterbody of default L2 cache queue for L2 cache queue; Accordingly, data processing module can extract the pending data L2 cache queue from the head of L2 cache queue.
Concrete, for the situation that is Circular buffer queue for L2 cache queue, can successively the pending data that poll extracts each time be write to the head of default L2 cache queue; Accordingly, data processing module can extract the pending data L2 cache queue from the afterbody of L2 cache queue.
In the embodiment of the present invention, data production module is produced at least one pending data; For at least one pending data arranges priority; According to the priority of at least one pending data, at least one pending data is write to the level cache queue in the level cache collection of queues corresponding with priority; According to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until at least one pending data is all extracted out; Successively the pending data that poll extracts are each time write to default L2 cache queue so that data processing module extracts pending data and processes from L2 cache queue.Visible, in this programme, the pending data that are at every turn polled to from level cache queue corresponding to different priorities are write to L2 cache queue, the pending data of different priorities are all processed, the data that therefore, can ensure high priority can be processed as early as possible and not be blocked in the queue of first in first out and wait for that data are above processed complete.
Below in conjunction with concrete application example, the data processing method that the embodiment of the present invention is provided is introduced.
Suppose that pending data exist two-stage priority: P1 and P2, wherein, P2 is higher than P1; Be pre-created and belong to single level cache queue Q1 and level cache queue Q2 that stops up queue, wherein, Q1 is for placing the pending data with priority P 1, and Q2 is for placing the pending data with priority P 2; And, be pre-created and belong to single L2 cache queue of stopping up queue.
(1) the pending data of data production module production low priority P1 in section sometime: A1, A2, A3, A4, A5, A6, A7, A8, the pending data of high priority P2: B1, B2, B3, B4.
(2) data production module is first successively by A1, A2, and A3, A4, A5, A6, A7, A8 writes the afterbody of Q1, and, successively by B1, B2, B3, B4 can be written into the afterbody of Q2:
Wherein, Q1:[A1, A2, A3, A4, A5, A6, A7, A8];
Q2:[B1,B2,B3,B4]。
(3) by the mode of queue poll, data production module is extracted pending data successively from the head of Q1 and Q2, and after the pending data that extract are rearranged, writes the afterbody of L2 cache queue.Concrete:
The 1st poll, extracts [A1, B1], L2 cache queue: [A1, B1];
The 2nd poll, extracts [A2, B2], L2 cache queue: [A1, B1, A2, B2];
The 3rd poll, extracts [A3, B3], L2 cache queue: [A1, B1, A2, B2, A3, B3];
The 4th poll, extracts [A4, B4], L2 cache queue: [A1, B1, A2, B2, A3, B3, A4, B4];
The 5th poll, extracts [A5], L2 cache queue: [A1, B1, A2, B2, A3, B3, A4, B4, A5];
The 6th poll, extracts [A6], L2 cache queue: [A1, B1, A2, B2, A3, B3, A4, B4, A5, A6];
The 7th poll, extracts [A7], L2 cache queue: [A1, B1, A2, B2, A3, B3, A4, B4, A5, A6, A7];
The 8th poll, extracts [A8], L2 cache queue: [A1, B1, A2, B2, A3, B3, A4, B4, A5, A6, A7, A8].
Further, data processing module can take out pending data at the head of L2 cache queue, and according to A1, B1, A2, B2, A3, B3, A4, B4, A5, A6, A7, the order of A8 deal with data successively.
Visible, can ensure that by this programme the data of high priority can be processed as early as possible and are not blocked in the queue of first in first out and wait for that data are above processed complete.
Corresponding to said method embodiment, as shown in Figure 2, the embodiment of the present invention provides a kind of data handling system, and described data handling system comprises that described data production module can comprise for the production of the data production module 210 of data with for the treatment of the data processing module 220 of data:
Productive unit 211, for the production of at least one pending data;
Priority setting unit 212, is used to described at least one pending data that priority is set;
One-level queue writing unit 213, for the priority according to described at least one pending data, writes the level cache queue in the default level cache collection of queues corresponding with priority by described at least one pending data; Wherein, the pending data of described level cache collection of queues for placing same priority;
Data extracting unit 214, be used for according to queue polling mode, in level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until described at least one pending data are all extracted out;
Secondary queue writing unit 215, for successively the pending data that poll extracts each time being write to default L2 cache queue so that data processing module 220 extracts pending data and processes from described L2 cache queue.
In the embodiment of the present invention, data production module is produced at least one pending data; For at least one pending data arranges priority; According to the priority of at least one pending data, at least one pending data is write to the level cache queue in the level cache collection of queues corresponding with priority; According to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until at least one pending data is all extracted out; Successively the pending data that poll extracts are each time write to default L2 cache queue so that data processing module extracts pending data and processes from L2 cache queue.Visible, in this programme, the pending data that are at every turn polled to from level cache queue corresponding to different priorities are write to L2 cache queue, the pending data of different priorities are all processed, the data that therefore, can ensure high priority can be processed as early as possible and not be blocked in the queue of first in first out and wait for that data are above processed complete.
Concrete, described level cache queue is single queue of stopping up;
Described one-level queue writing unit 213, specifically for:
According to the priority of described at least one pending data, described at least one pending data are write to the afterbody of level cache queue in the default level cache collection of queues corresponding with priority;
Described data extracting unit 214, specifically for:
According to queue polling mode, the head in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively, until described at least one pending data are all extracted out.
Concrete, described level cache queue is Circular buffer queue;
Described one-level queue writing unit 213, specifically for:
According to the priority of described at least one pending data, described at least one pending data are write to the head of level cache queue in the default level cache collection of queues corresponding with priority;
Described data extracting unit 214, specifically for:
According to queue polling mode, the afterbody in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively, until described at least one pending data are all extracted out.
Concrete, described L2 cache queue is single queue of stopping up;
Described secondary queue writing unit 215, specifically for:
Successively the pending data that poll extracts are each time write to the afterbody of default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
Concrete, described L2 cache queue is Circular buffer queue;
Described secondary queue writing unit 215, specifically for:
Successively the pending data that poll extracts are each time write to the head of default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
Concrete, described secondary queue writing unit 215, specifically for:
Successively the pending data that poll extracts are each time write to default L2 cache queue according to priority order from high to low so that data processing module extracts pending data and processes from described L2 cache queue.
For system or device embodiment, because it is substantially similar in appearance to embodiment of the method, so description is fairly simple, relevant part is referring to the part explanation of embodiment of the method.
It should be noted that, in this article, relational terms such as the first and second grades is only used for an entity or operation to separate with another entity or operational zone, and not necessarily requires or imply and between these entities or operation, have the relation of any this reality or sequentially.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thereby the process, method, article or the equipment that make to comprise a series of key elements not only comprise those key elements, but also comprise other key elements of clearly not listing, or be also included as the intrinsic key element of this process, method, article or equipment.The in the situation that of more restrictions not, the key element being limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment that comprises described key element and also have other identical element.
One of ordinary skill in the art will appreciate that all or part of step realizing in said method embodiment is can carry out the hardware that instruction is relevant by program to complete, described program can be stored in computer read/write memory medium, here the alleged storage medium obtaining, as: ROM/RAM, magnetic disc, CD etc.
The foregoing is only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.All any amendments of doing within the spirit and principles in the present invention, be equal to replacement, improvement etc., be all included in protection scope of the present invention.
Claims (12)
1. a data processing method, is characterized in that, is applied to data handling system, and wherein, described data handling system comprises for the production of the data production module of data with for the treatment of the data processing module of data; Described method comprises:
Described data production module is produced at least one pending data;
For described at least one pending data arrange priority;
According to the priority of described at least one pending data, described at least one pending data are write to the level cache queue in the default level cache collection of queues corresponding with priority; Wherein, the pending data of described level cache collection of queues for placing same priority;
According to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until described at least one pending data are all extracted out;
Successively the pending data that poll extracts are each time write to default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
2. method according to claim 1, is characterized in that, described level cache queue is single queue of stopping up;
The described priority according to described at least one pending data, writes the level cache queue in the default level cache collection of queues corresponding with priority by described at least one pending data, comprising:
According to the priority of described at least one pending data, described at least one pending data are write to the afterbody of level cache queue in the default level cache collection of queues corresponding with priority;
Described according to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, comprising:
According to queue polling mode, the head in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively.
3. method according to claim 1, is characterized in that, described level cache queue is Circular buffer queue;
The described priority according to described at least one pending data, writes the level cache queue in the default level cache collection of queues corresponding with priority by described at least one pending data, comprising:
According to the priority of described at least one pending data, described at least one pending data are write to the head of level cache queue in the default level cache collection of queues corresponding with priority;
Described according to queue polling mode, in the level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, comprising:
According to queue polling mode, the afterbody in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively.
4. according to the method described in claim 1-3 any one, it is characterized in that, described L2 cache queue is single queue of stopping up;
Describedly successively the pending data that poll extracts are each time write to default L2 cache queue, comprising:
Successively the pending data that poll extracts are each time write to the afterbody of default L2 cache queue.
5. according to the method described in claim 1-3 any one, it is characterized in that, described L2 cache queue is Circular buffer queue;
Describedly successively the pending data that poll extracts are each time write to default L2 cache queue, comprising:
Successively the pending data that poll extracts are each time write to the head of default L2 cache queue.
6. according to the method described in claim 1-3 any one, it is characterized in that, the described pending data order of successively poll each time being extracted writes default L2 cache queue, comprising:
Successively the pending data that poll extracts are each time write to default L2 cache queue according to priority order from high to low.
7. a data handling system, is characterized in that, described data handling system comprises that described data production module comprises for the production of the data production module of data with for the data processing module of consumption data:
Productive unit, for the production of at least one pending data;
Priority setting unit, is used to described at least one pending data that priority is set;
One-level queue writing unit, for the priority according to described at least one pending data, writes the level cache queue in the default level cache collection of queues corresponding with priority by described at least one pending data; Wherein, the pending data of described level cache collection of queues for placing same priority;
Data extracting unit, be used for according to queue polling mode, in level cache queue from each level cache collection of queues, extract successively the pending data of the predetermined number corresponding with corresponding level cache queue, until described at least one pending data are all extracted out;
Secondary queue writing unit, for successively the pending data that poll extracts each time being write to default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
8. system according to claim 7, is characterized in that, described level cache queue is single queue of stopping up;
Described one-level queue writing unit, specifically for:
According to the priority of described at least one pending data, described at least one pending data are write to the afterbody of level cache queue in the default level cache collection of queues corresponding with priority;
Described data extracting unit, specifically for:
According to queue polling mode, the head in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively, until described at least one pending data are all extracted out.
9. system according to claim 7, is characterized in that, described level cache queue is Circular buffer queue;
Described one-level queue writing unit, specifically for:
According to the priority of described at least one pending data, described at least one pending data are write to the head of level cache queue in the default level cache collection of queues corresponding with priority;
Described data extracting unit, specifically for:
According to queue polling mode, the afterbody in the level cache queue from each level cache collection of queues extracts the pending data of the predetermined number corresponding with corresponding level cache queue successively, until described at least one pending data are all extracted out.
10. want the system described in 7-9 any one according to right, it is characterized in that, described L2 cache queue is single queue of stopping up;
Described secondary queue writing unit, specifically for:
Successively the pending data that poll extracts are each time write to the afterbody of default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
11. according to the system described in claim 7-9 any one, it is characterized in that, described L2 cache queue is Circular buffer queue;
Described secondary queue writing unit, specifically for:
Successively the pending data that poll extracts are each time write to the head of default L2 cache queue so that data processing module extracts pending data and processes from described L2 cache queue.
12. according to the system described in claim 7-9 any one, it is characterized in that, and described secondary queue writing unit, specifically for:
Successively the pending data that poll extracts are each time write to default L2 cache queue according to priority order from high to low so that data processing module extracts pending data and processes from described L2 cache queue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410415527.0A CN104199790B (en) | 2014-08-21 | 2014-08-21 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410415527.0A CN104199790B (en) | 2014-08-21 | 2014-08-21 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104199790A true CN104199790A (en) | 2014-12-10 |
| CN104199790B CN104199790B (en) | 2017-10-24 |
Family
ID=52085086
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410415527.0A Active CN104199790B (en) | 2014-08-21 | 2014-08-21 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104199790B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105760236A (en) * | 2016-05-13 | 2016-07-13 | 中国工商银行股份有限公司 | Data collection method and system of distributed computer cluster |
| CN106569746A (en) * | 2016-11-01 | 2017-04-19 | 北京信安世纪科技有限公司 | Data writing method and device |
| CN107171918A (en) * | 2017-04-26 | 2017-09-15 | 成都成电光信科技股份有限公司 | Support the messaging method in the GJB289A bus modules of priority |
| CN109471731A (en) * | 2018-11-21 | 2019-03-15 | 阿里巴巴集团控股有限公司 | A kind of data processing, EMS memory management process, device, equipment and medium |
| CN110019358A (en) * | 2017-09-30 | 2019-07-16 | 北京搜狗科技发展有限公司 | A kind of data processing method, device and equipment and storage medium |
| CN110673786A (en) * | 2019-09-03 | 2020-01-10 | 浪潮电子信息产业股份有限公司 | Data caching method and device |
| CN111355673A (en) * | 2018-12-24 | 2020-06-30 | 深圳市中兴微电子技术有限公司 | Data processing method, device, equipment and storage medium |
| CN111865741A (en) * | 2019-04-24 | 2020-10-30 | 瑞昱半导体股份有限公司 | Data transmission method and data transmission system |
| CN114500403A (en) * | 2022-01-24 | 2022-05-13 | 中国联合网络通信集团有限公司 | Data processing method and device and computer readable storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6728265B1 (en) * | 1999-07-30 | 2004-04-27 | Intel Corporation | Controlling frame transmission |
| CN1508709A (en) * | 2002-11-25 | 2004-06-30 | ��ʽ���������Ƽ� | Judging circuit and data processing system |
| CN1581835A (en) * | 2003-07-31 | 2005-02-16 | 富士通株式会社 | Media access control device for wireless LAN |
| CN101478472A (en) * | 2008-10-21 | 2009-07-08 | 北京闪联讯通数码科技有限公司 | Socket data transmission processing method and apparatus |
| CN103488574A (en) * | 2012-06-12 | 2014-01-01 | Ls产电株式会社 | Circuit for memory sharing |
| US20140185451A1 (en) * | 2012-12-31 | 2014-07-03 | Telefonaktiebolaget L M Ericsson (Publ) | Low pass filter for hierarchical pipelined distributed scheduling traffic manager |
-
2014
- 2014-08-21 CN CN201410415527.0A patent/CN104199790B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6728265B1 (en) * | 1999-07-30 | 2004-04-27 | Intel Corporation | Controlling frame transmission |
| CN1508709A (en) * | 2002-11-25 | 2004-06-30 | ��ʽ���������Ƽ� | Judging circuit and data processing system |
| CN1581835A (en) * | 2003-07-31 | 2005-02-16 | 富士通株式会社 | Media access control device for wireless LAN |
| CN101478472A (en) * | 2008-10-21 | 2009-07-08 | 北京闪联讯通数码科技有限公司 | Socket data transmission processing method and apparatus |
| CN103488574A (en) * | 2012-06-12 | 2014-01-01 | Ls产电株式会社 | Circuit for memory sharing |
| US20140185451A1 (en) * | 2012-12-31 | 2014-07-03 | Telefonaktiebolaget L M Ericsson (Publ) | Low pass filter for hierarchical pipelined distributed scheduling traffic manager |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105760236A (en) * | 2016-05-13 | 2016-07-13 | 中国工商银行股份有限公司 | Data collection method and system of distributed computer cluster |
| CN106569746A (en) * | 2016-11-01 | 2017-04-19 | 北京信安世纪科技有限公司 | Data writing method and device |
| CN107171918B (en) * | 2017-04-26 | 2020-06-16 | 成都成电光信科技股份有限公司 | Message transceiving method in GJB289A bus module supporting priority |
| CN107171918A (en) * | 2017-04-26 | 2017-09-15 | 成都成电光信科技股份有限公司 | Support the messaging method in the GJB289A bus modules of priority |
| CN110019358A (en) * | 2017-09-30 | 2019-07-16 | 北京搜狗科技发展有限公司 | A kind of data processing method, device and equipment and storage medium |
| CN110019358B (en) * | 2017-09-30 | 2021-08-24 | 北京搜狗科技发展有限公司 | Data processing method, device and equipment and storage medium |
| CN109471731A (en) * | 2018-11-21 | 2019-03-15 | 阿里巴巴集团控股有限公司 | A kind of data processing, EMS memory management process, device, equipment and medium |
| CN111355673A (en) * | 2018-12-24 | 2020-06-30 | 深圳市中兴微电子技术有限公司 | Data processing method, device, equipment and storage medium |
| WO2020134425A1 (en) * | 2018-12-24 | 2020-07-02 | 深圳市中兴微电子技术有限公司 | Data processing method, apparatus, and device, and storage medium |
| CN111865741A (en) * | 2019-04-24 | 2020-10-30 | 瑞昱半导体股份有限公司 | Data transmission method and data transmission system |
| CN110673786A (en) * | 2019-09-03 | 2020-01-10 | 浪潮电子信息产业股份有限公司 | Data caching method and device |
| US11803475B2 (en) | 2019-09-03 | 2023-10-31 | Inspur Electronic Information Industry Co., Ltd. | Method and apparatus for data caching |
| CN114500403A (en) * | 2022-01-24 | 2022-05-13 | 中国联合网络通信集团有限公司 | Data processing method and device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104199790B (en) | 2017-10-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104199790A (en) | Data processing method and device | |
| TWI691855B (en) | Blockchain-based data processing method and equipment | |
| US9489763B2 (en) | Techniques for setting up and executing draw calls | |
| US10325219B2 (en) | Parallel retrieval of training data from multiple producers for machine learning systems | |
| CN103019810A (en) | Scheduling and management of compute tasks with different execution priority levels | |
| KR20180069807A (en) | Accelerating task subgraphs by remapping synchronization | |
| WO2010137822A3 (en) | Ray tracing core and ray tracing chip including same | |
| US8312196B2 (en) | Dual processor system and method for using the same | |
| AU2012217617B2 (en) | Sorting | |
| CN106598760A (en) | Message queue consumption method and apparatus | |
| US8875008B2 (en) | Presentation progress as context for presenter and audience | |
| US9007370B2 (en) | Computing device and method for processing curved surface | |
| US20110193785A1 (en) | Intuitive Grouping and Viewing of Grouped Objects Using Touch | |
| CN104182360A (en) | Multi-core environment trace log processing method and system | |
| CN108701102A (en) | Direct memory access controller, method for reading data and method for writing data | |
| CN104572275A (en) | Process loading method, process loading device and process loading system | |
| CN107239343B (en) | Data processing method and device | |
| CN109144911B (en) | Scheduling method, PCIe controller and related electronic system thereof | |
| US20170109835A1 (en) | Standard working hours management system | |
| CN103631571A (en) | Method and device for controlling progress bar | |
| CN102004811B (en) | Simulation test method and device of chip circuit | |
| CN102638403A (en) | Method and device for processing messages | |
| CN105760317A (en) | Data Writing System And Data Writing Method For Core Processor | |
| JP6584655B2 (en) | Graphics context scheduling based on flip-queue management | |
| CN104811161A (en) | Median filtering implementation method for integrated circuit design |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |