US20160124763A1 - Limited virtual device polling based on virtual cpu pre-emption - Google Patents

Limited virtual device polling based on virtual cpu pre-emption Download PDF

Info

Publication number
US20160124763A1
US20160124763A1 US14/527,661 US201414527661A US2016124763A1 US 20160124763 A1 US20160124763 A1 US 20160124763A1 US 201414527661 A US201414527661 A US 201414527661A US 2016124763 A1 US2016124763 A1 US 2016124763A1
Authority
US
United States
Prior art keywords
hypervisor
virtual
shared device
polling
virtual machine
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US14/527,661
Other versions
US9766917B2 (en
Inventor
Michael S. Tsirkin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Red Hat Israel Ltd
Original Assignee
Red Hat Israel Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Red Hat Israel Ltd filed Critical Red Hat Israel Ltd
Priority to US14/527,661 priority Critical patent/US9766917B2/en
Assigned to RED HAT ISRAEL, LTD. reassignment RED HAT ISRAEL, LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TSIRKIN, MICHAEL S.
Publication of US20160124763A1 publication Critical patent/US20160124763A1/en
Application granted granted Critical
Publication of US9766917B2 publication Critical patent/US9766917B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/445Program loading or initiating
    • G06F9/44505Configuring for program initiating, e.g. using registry, configuration files
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5077Logical partitioning of resources; Management or configuration of virtualized resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083Techniques for rebalancing the load in a distributed system
    • G06F9/5088Techniques for rebalancing the load in a distributed system involving task migration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/544Buffers; Shared memory; Pipes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G06F11/3419Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment by assessing time
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3688Test management for test execution, e.g. scheduling of test suites
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/4557Distribution of virtual machine instances; Migration and load balancing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45579I/O management, e.g. providing access to device drivers or storage
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5022Workload threshold

Definitions

  • the present disclosure is generally related to computer systems, and more particularly, to shared device polling in virtualized computer systems.
  • a virtual machine is a portion of software that, when executed on appropriate hardware, creates an environment allowing the virtualization of an actual physical computer system (e.g., a server, a mainframe computer, etc.).
  • the actual physical computer system is typically referred to as a “host machine,” and the operating system of the host machine is typically referred to as the “host operating system.”
  • software on the host machine known as a “hypervisor” (or a “virtual machine monitor”) manages the execution of one or more virtual machines, providing a variety of functions such as virtualizing and allocating resources, context switching among virtual machines, etc.
  • a virtual machine may comprise one or more “virtual processors,” each of which maps, possibly in a many-to-one fashion, to a central processing unit (CPU) of the host machine.
  • CPU central processing unit
  • a guest operating system running within a virtual machine needs to execute a privileged operation (such as an I/O instruction to a physical storage device attached to the host hardware), one way of doing so can be by writing requests to a virtual shared device based on shared memory.
  • the host operating system typically accesses the shared memory to determine whether there are any requests pending from the guest operating system, which is referred to as “polling” the shared device
  • FIG. 1 depicts a high-level component diagram of an example computer system architecture, in accordance with one or more aspects of the present disclosure.
  • FIG. 2 depicts a flow diagram of a method for disabling polling of a shared device based on the execution states of each virtual processor of a virtual machine, in accordance with one or more aspects of the present disclosure.
  • FIG. 3 depicts a flow diagram of a method for enabling polling of a shared device based on the processing load of all virtual processors of a virtual machine, in accordance with one or more aspects of the present disclosure.
  • FIG. 4 depicts a block diagram of an illustrative computer system operating in accordance with examples of the invention.
  • a hypervisor determines whether or not to disable polling of a shared device for requests from a guest operating system (OS).
  • OS guest operating system
  • a guest OS running within a virtual machine (VM) needs to execute a privileged operation (such as an I/O instruction to a physical storage device attached to the host hardware)
  • a privileged operation such as an I/O instruction to a physical storage device attached to the host hardware
  • the shared device may be a data structure in a shared memory location that is accessible to both the VM and the host OS.
  • the host OS typically accesses the shared memory to determine whether there are any requests pending from the guest OS, which is referred to as “polling” the shared device.
  • CPU resources can be similarly wasted if the host continues polling the shared device after which a guest OS is no longer writing to the shared device. This can occur when any of the virtual processors of the VM have been pre-empted by the host so that the associated physical processor can execute a host related task. Pre-emption is the interruption by the host of the virtual processor without its cooperation in order to perform another task with the intention of resuming execution of the virtual processor at a later time.
  • VMs can often have multi-threaded tasks that run on all of its virtual processors. For example, a VM may be executing a task on virtual processor A that triggers virtual processor B to write requests to a shared device. If the host pre-empts virtual processor A to execute a host task, then virtual processor A will no longer be triggering virtual processor B to write requests to the shared device. Thus, continued polling of the shared device would waste resources since virtual processor B would not be writing any additional requests to the shared device so long as virtual processor A is pre-empted by the host.
  • the hypervisor identifies a request of a guest OS of a VM that has been associated with a shared device.
  • the shared device may be a data structure in a shared memory location that is accessible to both the VM and the hypervisor.
  • the VM may have a plurality of virtual processors.
  • the hypervisor processes any requests found in the shared device, and subsequently engages in polling of the shared device for additional requests. Upon determining that there are no requests remaining to be processed, the hypervisor can identify each virtual processor of the virtual machine to determine their execution state.
  • the hypervisor can disable polling the shared device for new requests.
  • the hypervisor may then notify the guest operating system that polling has been disabled.
  • the hypervisor may notify the guest via, for example, an interrupt request or a message written to a data structure in a shared memory location.
  • the notification may also indicate, to the guest operating system, the reason for disabling the polling (e.g., polling time has exceeded the predetermined threshold, etc.).
  • the hypervisor may determine whether the execution state of the pre-empted virtual processor changes. For example, the hypervisor may determine that the virtual processor that had been previously pre-empted has resumed running The hypervisor may then enable polling of the shared device for new requests in response to determining that the virtual processor is no longer pre-empted. In certain implementations, the hypervisor can notify the guest operating system that polling has been enabled. The hypervisor may notify the guest via, for example, an interrupt request or a message written to a data structure in a shared memory location.
  • the hypervisor may determine whether no additional requests of the guest OS are associated with the shared device for a period of time that exceeds a predetermined threshold.
  • the hypervisor may disable polling of the shared device upon determining that the threshold has been met.
  • the threshold value may represent a total amount of time elapsed during polling of a shared device. Alternatively, the threshold value may be a total amount of CPU clock cycles executed during polling of a shared device.
  • the threshold may be defined by an administrator (e.g., via a configuration file, via a graphical user interface, etc.), hard-coded in the hypervisor, or set in any other manner.
  • the virtualized environment may be overcommitted, meaning that the number of virtual processors on the VM is equal to the number of physical processors on the host.
  • any instance of polling by the hypervisor could lead to a virtual processor being pre-empted, and as a result cause the hypervisor to immediately disable polling.
  • the hypervisor can enable polling by assigning the plurality of virtual processors of the virtual machine to a single host processor. The hypervisor may first analyze the load placed by each of the virtual processors of the VM on the host.
  • the hypervisor may then identify a single host processor capable of processing the load for all of the virtual processors of that virtual machine collectively.
  • the threshold condition may be implemented as a percentage of CPU load, the total amount of CPU clock cycles executed during a particular time period, or in any similar manner.
  • the predefined threshold condition may be set such that for a VM with two virtual processors, any virtual processor that exerts a load of less than 25% of the total capacity of the physical processor may be reassigned.
  • the hypervisor may determine that each of the virtual processors of the VM only places a load of 10% of the capacity of the physical processor.
  • the hypervisor may then identify a single host processor capable of processing the load for the plurality of virtual processors of the virtual machine and assign the virtual processors to the identified host processor. Subsequently, the hypervisor can enable polling of the shared device and notify the guest OS that polling has been enabled.
  • aspects of the present disclosure are thus capable of limiting CPU usage associated with polling by the host operating system. More particularly, aspects of the present disclosure prevent continued polling of the shared memory from wasting CPU resources in the event that any virtual processor of the VM has been pre-empted.
  • FIG. 1 depicts a high-level component diagram of an illustrative example of a computer system 100 , in accordance with one or more aspects of the present disclosure.
  • FIG. 1 depicts a high-level component diagram of an illustrative example of a computer system 100 , in accordance with one or more aspects of the present disclosure.
  • One skilled in the art will appreciate that other architectures for computer system 100 are possible, and that the implementation of a computer system utilizing examples of the invention are not necessarily limited to the specific architecture depicted by FIG. 1 .
  • the computer system 100 is connected to a network 150 and comprises one or more central processing units (CPUs) 160 - 1 through 160 -N, where N is a positive integer, main memory 170 , which may include volatile memory devices (e.g., random access memory (RAM)), non-volatile memory devices (e.g., flash memory) and/or other types of memory devices, and a storage device 180 (e.g., one or more hard disk drives, solid-state drives, etc.).
  • main memory 170 may be non-uniform access (NUMA), such that memory access time depends on the memory location relative to CPUs 160 - 1 through 160 -N.
  • NUMA non-uniform access
  • the computer system 100 may be a server, a mainframe, a workstation, a personal computer (PC), a mobile phone, a palm-sized computing device, etc.
  • the network 150 may be a private network (e.g., a local area network (LAN), a wide area network (WAN), intranet, etc.) or a public network (e.g., the Internet).
  • LAN local area network
  • WAN wide area network
  • intranet e.g., the Internet
  • Computer system 100 may additionally comprise virtual machine (VM) 130 , host operating system (OS) 120 , and shared device 190 .
  • VM 130 is a software implementation of a machine that executes programs as though it were an actual physical machine.
  • Host OS 120 manages the hardware resources of the computer system and provides functions such as inter-process communication, scheduling, memory management, and so forth.
  • Shared device 190 is a data structure in shared memory accessible to both guest OS 135 of VM 130 and hypervisor 125 of host OS 120 .
  • VM 130 comprises a guest operating system (OS) 135 that handles the execution of applications within the virtual machine, and one or more virtual processors 131 - 1 through 131 -N, where N is a positive integer.
  • OS guest operating system
  • One or more virtual processors 131 - 1 through 131 -N may be associated with a shared device 190 .
  • Guest OS 135 may write requests to shared device 190 to be processed by host OS 190 . It should be noted that although, for simplicity, a single virtual machine 130 is depicted in FIG. 1 , computer system 100 may host a plurality of VMs 130 .
  • Host OS 120 may comprise a hypervisor 125 , which provides a virtual operating platform for virtual machine 130 and manages its execution.
  • Hypervisor 125 may comprise polling manager 128 , virtual processor data structures 126 , and virtual processor mapping table 127 . It should be noted that in some alternative implementations, hypervisor 125 may be external to host OS 120 , rather than embedded within host OS 120 , or may replace host OS 120 .
  • Polling manager 128 can identify requests in a shared device 190 of guest OS 135 , process the requests, and determine whether continued polling of the shared device should be disabled based on the execution state of the virtual processors associated with VM 130 , as described in detail below with respect to FIG. 2 . Polling manager 128 may also determine whether polling of the shared device should be re-enabled based on the processing load of all virtual processors of VM 130 , as described in detail below with respect to FIG. 3 .
  • Hypervisor 125 can use virtual processor data structures 126 to maintain information regarding virtual processors 131 - 1 through 131 -N of VM 130 .
  • Hypervisor 125 may store the execution state of virtual processors 131 - 1 through 131 -N in virtual processor data structures 126 for use by polling manager 128 in determining whether to disable polling of shared device 190 .
  • Hypervisor 125 may update virtual processor data structures 126 each time the execution state of any of virtual processors 131 - 1 through 131 -N changes. For example, if the execution state one of the virtual processors changes from running to pre-empted.
  • Hypervisor 125 may also store a unique identifier in virtual processor data structures 126 that maps to a specific virtual processor of virtual processors 131 - 1 through 131 -N for use by polling manager 128 .
  • Hypervisor 125 may create separate virtual data structures 126 for each of virtual processors 131 - 1 through 131 -N, or in the alternative, create a single virtual processor data structure 126 for all virtual processors 131 - 1 through 131 -N.
  • Virtual processor data structures 126 may be a memory location within hypervisor 125 . Alternatively, virtual processor data structures 126 may be written to a location in storage device 180 .
  • Hypervisor 125 can use virtual processor mapping table 127 to store cross reference information regarding which of the virtual processors 131 - 1 through 131 -N of VM 130 are associated with shared device 190 .
  • Hypervisor 125 may store a unique identifier that is associated with each of the virtual processors 131 - 1 through 131 -N, along with an identifier for the shared memory location of the associated shared device 190 for use by polling manager 128 . If hypervisor 125 determines that a virtual processor of VM 130 needs to be reassigned to another physical processor as described below in FIG. 3 , it may use virtual processor mapping table 127 to assist with identification of eligible physical processors.
  • Virtual processor mapping table 127 may be a memory location within hypervisor 125 . Alternatively, virtual processor mapping table 127 may be written to a location in storage device 180 .
  • FIG. 2 depicts a flow diagram of an example method 200 for disabling polling of a shared device based on the execution states of each virtual processor of a virtual machine.
  • the method may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both.
  • method 200 may be performed by polling manager 128 of hypervisor 125 in FIG. 1 .
  • some or all of method 200 might be performed by another machine. It should be noted that blocks depicted in FIG. 2 could be performed simultaneously or in a different order than that depicted.
  • processing logic identifies a request of a guest operating system associated with a shared device.
  • the shared device may be represented by a data structure in shared memory, such as shared device 190 which is accessible to both guest OS 135 of VM 130 and hypervisor 125 in FIG. 1 .
  • the VM may have a plurality of virtual processors, such as virtual processors 131 - 1 through 131 -N of VM 130 in FIG. 1 .
  • processing logic processes the request associated with the shared device.
  • processing logic polls the shared device for additional requests of the guest operating system.
  • processing logic determines whether there are additional requests of the guest operating system associated with the shared device to be processed. If processing logic determines that there are additional requests in the shared device to be processed, processing logic returns to block 202 to process the requests and continue polling. Otherwise, execution proceeds to block 205 .
  • processing logic may first determine whether no additional requests were associated with the shared device for a period of time that exceeds a predetermined threshold. If processing logic determines that the threshold has not yet been met, execution returns to block 202 , otherwise execution proceeds to block 205 .
  • the threshold value may represent a total amount of time elapsed during polling of a shared device.
  • the threshold value may be a total amount of CPU clock cycles executed during polling of a shared device.
  • the threshold may be defined by an administrator (e.g., via a configuration file, via a graphical user interface, etc.), hard-coded in the hypervisor, or set in any other manner.
  • processing logic determines the execution state of each virtual processor of the virtual machine (e.g., the virtual processor is currently running and busy, currently running but idle, not currently running, pre-empted by the host, etc.).
  • the various execution states may be defined by an administrator (e.g., via a configuration file, via a graphical user interface, etc.), hard-coded in polling manager 128 of hypervisor 125 in FIG. 1 , or set in any other manner.
  • Processing logic may access a data structure that stores the execution state of the virtual processor such as virtual processor data structures 126 of FIG. 1 .
  • processing logic can use the unique identifier of the virtual processor from block 205 to access the applicable data structure for the virtual processor.
  • the data structure may contain the current execution state of the applicable virtual processor.
  • processing logic branches based on the determination of block 205 . If processing logic determined that at least one virtual processor of the virtual machine has been pre-empted (e.g., if the host has suspended execution of the virtual processor in order to devote the associated host processor to a host-related task), execution continues to block 207 , otherwise execution returns to block 203 to continue polling.
  • processing logic disables polling of the shared device.
  • processing logic notifies the guest OS that polling has been disabled. This notification may be made via an interrupt request, or alternatively, via a data structure stored in shared memory. In certain implementations, the notification may also indicate to the guest OS the reason that polling is disabled (e.g., the virtual processor is pre-empted, etc.). After block 208 , the method of FIG. 2 terminates.
  • FIG. 3 depicts a flow diagram of an example method 300 for enabling polling of a shared device based on the processing load of all virtual processors of a virtual machine.
  • the method may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both.
  • method 300 may be performed by polling manager 128 of hypervisor 125 in FIG. 1 .
  • some or all of method 300 might be performed by another machine. It should be noted that blocks depicted in FIG. 3 could be performed simultaneously or in a different order than that depicted.
  • processing logic may first analyze the load placed by each of the virtual processors of the VM on the host.
  • processing logic determines whether the load placed by each virtual processor of the VM on the associated physical processor meets a predefined threshold condition. If processing logic determines that the threshold has not yet been met, the method of FIG. 3 terminates, otherwise execution proceeds to block 303 .
  • the threshold condition may be implemented as a percentage of CPU load, the total amount of CPU clock cycles executed during a particular time period, or in any similar manner.
  • the predefined threshold condition may be set such that for a VM with two virtual processors, any virtual processor that exerts a load of less than 25% of the total capacity of the physical processor may reassigned.
  • the hypervisor determines that each of the virtual processors of the VM only places a load of 10% of the capacity of the physical processor, the hypervisor may determine that that the threshold condition for reassignment has been met.
  • processing logic may identify a single host processor capable of processing the load for the plurality of the virtual processors of that virtual machine collectively.
  • processing logic assigns the plurality of virtual processors to the single host processor identified at block 303 .
  • processing logic enables polling of the shared device.
  • processing logic notifies the guest OS that polling has been enabled. This notification may be made via an interrupt request, or alternatively, via a data structure stored in shared memory. In certain implementations, the notification may also indicate to the guest OS the reason that polling has been enabled. After block 306 , the method of FIG. 3 terminates.
  • FIG. 4 depicts an example computer system 400 which can perform any one or more of the methods described herein.
  • computer system 400 may correspond to computer system 100 of FIG. 1 .
  • the computer system may be connected (e.g., networked) to other computer systems in a LAN, an intranet, an extranet, or the Internet.
  • the computer system may operate in the capacity of a server in a client-server network environment.
  • the computer system may be a personal computer (PC), a set-top box (STB), a server, a network router, switch or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device.
  • PC personal computer
  • STB set-top box
  • server a server
  • network router switch or bridge
  • the exemplary computer system 400 includes a processing system (processor) 402 , a main memory 404 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)), a static memory 406 (e.g., flash memory, static random access memory (SRAM)), and a data storage device 416 , which communicate with each other via a bus 408 .
  • processor processing system
  • main memory 404 e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)
  • DRAM dynamic random access memory
  • SDRAM synchronous DRAM
  • static memory 406 e.g., flash memory, static random access memory (SRAM)
  • SRAM static random access memory
  • Processor 402 represents one or more general-purpose processing devices such as a microprocessor, central processing unit, or the like. More particularly, the processor 402 may be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets.
  • the processor 402 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like.
  • the processor 402 is configured to execute instructions 426 for performing the operations and steps discussed herein.
  • the computer system 400 may further include a network interface device 422 .
  • the computer system 400 also may include a video display unit 410 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 412 (e.g., a keyboard), a cursor control device 414 (e.g., a mouse), and a signal generation device 420 (e.g., a speaker).
  • a video display unit 410 e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)
  • an alphanumeric input device 412 e.g., a keyboard
  • a cursor control device 414 e.g., a mouse
  • a signal generation device 420 e.g., a speaker
  • the data storage device 416 may include a computer-readable medium 424 on which is stored one or more sets of instructions 426 (e.g., instructions corresponding to the methods of FIGS. 2 and 3 , etc.) embodying any one or more of the methodologies or functions described herein. Instructions 426 may also reside, completely or at least partially, within the main memory 404 and/or within the processor 402 during execution thereof by the computer system 400 , the main memory 404 and the processor 402 also constituting computer-readable media. Instructions 426 may further be transmitted or received over a network via the network interface device 422 .
  • instructions 426 may also reside, completely or at least partially, within the main memory 404 and/or within the processor 402 during execution thereof by the computer system 400 , the main memory 404 and the processor 402 also constituting computer-readable media. Instructions 426 may further be transmitted or received over a network via the network interface device 422 .
  • While the computer-readable storage medium 424 is shown in the illustrative examples to be a single medium, the term “computer-readable storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions.
  • the term “computer-readable storage medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present invention.
  • the term “computer-readable storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.
  • the present invention also relates to an apparatus for performing the operations herein.
  • This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer.
  • a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.
  • the present invention may be provided as a computer program product, or software, that may include a machine-readable medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to the present invention.
  • a machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer).
  • a machine-readable (e.g., computer-readable) medium includes a machine (e.g., a computer) readable storage medium (e.g., read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, etc.), a machine (e.g., computer) readable transmission medium (electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Hardware Redundancy (AREA)

Abstract

A hypervisor executing on a computer system identifies a request of a guest operating system of a virtual machine associated with a shared device. The shared device comprises a shared memory space between a virtual processor of the virtual machine and the hypervisor and the virtual machine has a plurality of virtual processors. The hypervisor processes the request of the guest operating system and polls the shared device for additional requests of the guest operating system. Upon determining that there are no additional requests associated with the shared device to be processed, the hypervisor determines the execution state of each virtual processor of the virtual machine. The hypervisor disables polling the shared device for requests upon determining that at least one of the plurality of virtual processors has been pre-empted.

Description

    TECHNICAL FIELD
  • The present disclosure is generally related to computer systems, and more particularly, to shared device polling in virtualized computer systems.
  • BACKGROUND
  • A virtual machine (VM) is a portion of software that, when executed on appropriate hardware, creates an environment allowing the virtualization of an actual physical computer system (e.g., a server, a mainframe computer, etc.). The actual physical computer system is typically referred to as a “host machine,” and the operating system of the host machine is typically referred to as the “host operating system.” Typically, software on the host machine known as a “hypervisor” (or a “virtual machine monitor”) manages the execution of one or more virtual machines, providing a variety of functions such as virtualizing and allocating resources, context switching among virtual machines, etc. A virtual machine may comprise one or more “virtual processors,” each of which maps, possibly in a many-to-one fashion, to a central processing unit (CPU) of the host machine.
  • When a guest operating system running within a virtual machine needs to execute a privileged operation (such as an I/O instruction to a physical storage device attached to the host hardware), one way of doing so can be by writing requests to a virtual shared device based on shared memory. The host operating system typically accesses the shared memory to determine whether there are any requests pending from the guest operating system, which is referred to as “polling” the shared device
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The present disclosure is illustrated by way of example, and not by way of limitation, and can be more fully understood with reference to the following detailed description when considered in connection with the figures in which:
  • FIG. 1 depicts a high-level component diagram of an example computer system architecture, in accordance with one or more aspects of the present disclosure.
  • FIG. 2 depicts a flow diagram of a method for disabling polling of a shared device based on the execution states of each virtual processor of a virtual machine, in accordance with one or more aspects of the present disclosure.
  • FIG. 3 depicts a flow diagram of a method for enabling polling of a shared device based on the processing load of all virtual processors of a virtual machine, in accordance with one or more aspects of the present disclosure.
  • FIG. 4 depicts a block diagram of an illustrative computer system operating in accordance with examples of the invention.
  • DETAILED DESCRIPTION
  • Described herein are methods and systems by which a hypervisor determines whether or not to disable polling of a shared device for requests from a guest operating system (OS). When a guest OS running within a virtual machine (VM) needs to execute a privileged operation (such as an I/O instruction to a physical storage device attached to the host hardware), one way of doing so can be by writing requests to a virtual shared device based. The shared device may be a data structure in a shared memory location that is accessible to both the VM and the host OS. The host OS typically accesses the shared memory to determine whether there are any requests pending from the guest OS, which is referred to as “polling” the shared device. If the host OS processes these requests at a faster rate than that with which the guest OS adds them, continued polling of the shared memory can lead to a waste of CPU resources. CPU resources can be similarly wasted if the host continues polling the shared device after which a guest OS is no longer writing to the shared device. This can occur when any of the virtual processors of the VM have been pre-empted by the host so that the associated physical processor can execute a host related task. Pre-emption is the interruption by the host of the virtual processor without its cooperation in order to perform another task with the intention of resuming execution of the virtual processor at a later time.
  • VMs can often have multi-threaded tasks that run on all of its virtual processors. For example, a VM may be executing a task on virtual processor A that triggers virtual processor B to write requests to a shared device. If the host pre-empts virtual processor A to execute a host task, then virtual processor A will no longer be triggering virtual processor B to write requests to the shared device. Thus, continued polling of the shared device would waste resources since virtual processor B would not be writing any additional requests to the shared device so long as virtual processor A is pre-empted by the host.
  • Aspects of the present disclosure address the above noted deficiency by disabling polling if any of the virtual processors of the same virtual machine have been pre-empted by the host. In an illustrative example, the hypervisor identifies a request of a guest OS of a VM that has been associated with a shared device. The shared device may be a data structure in a shared memory location that is accessible to both the VM and the hypervisor. In some implementations, the VM may have a plurality of virtual processors. The hypervisor processes any requests found in the shared device, and subsequently engages in polling of the shared device for additional requests. Upon determining that there are no requests remaining to be processed, the hypervisor can identify each virtual processor of the virtual machine to determine their execution state. If the hypervisor detects that at least one virtual processor of the virtual machine has been pre-empted (e.g., if the host has suspended execution of the virtual processor in order to devote the associated host processor to a host-related task), the hypervisor can disable polling the shared device for new requests. The hypervisor may then notify the guest operating system that polling has been disabled. The hypervisor may notify the guest via, for example, an interrupt request or a message written to a data structure in a shared memory location. The notification may also indicate, to the guest operating system, the reason for disabling the polling (e.g., polling time has exceeded the predetermined threshold, etc.).
  • Subsequently, the hypervisor may determine whether the execution state of the pre-empted virtual processor changes. For example, the hypervisor may determine that the virtual processor that had been previously pre-empted has resumed running The hypervisor may then enable polling of the shared device for new requests in response to determining that the virtual processor is no longer pre-empted. In certain implementations, the hypervisor can notify the guest operating system that polling has been enabled. The hypervisor may notify the guest via, for example, an interrupt request or a message written to a data structure in a shared memory location.
  • In accordance with one example, the hypervisor may determine whether no additional requests of the guest OS are associated with the shared device for a period of time that exceeds a predetermined threshold. The hypervisor may disable polling of the shared device upon determining that the threshold has been met. The threshold value may represent a total amount of time elapsed during polling of a shared device. Alternatively, the threshold value may be a total amount of CPU clock cycles executed during polling of a shared device. The threshold may be defined by an administrator (e.g., via a configuration file, via a graphical user interface, etc.), hard-coded in the hypervisor, or set in any other manner.
  • In certain implementations, the virtualized environment may be overcommitted, meaning that the number of virtual processors on the VM is equal to the number of physical processors on the host. In this situation, any instance of polling by the hypervisor could lead to a virtual processor being pre-empted, and as a result cause the hypervisor to immediately disable polling. In this situation, the hypervisor can enable polling by assigning the plurality of virtual processors of the virtual machine to a single host processor. The hypervisor may first analyze the load placed by each of the virtual processors of the VM on the host. Upon determining that the load placed by each of the plurality of virtual processors on the host meets a predefined threshold condition, the hypervisor may then identify a single host processor capable of processing the load for all of the virtual processors of that virtual machine collectively. The threshold condition may be implemented as a percentage of CPU load, the total amount of CPU clock cycles executed during a particular time period, or in any similar manner.
  • For example, the predefined threshold condition may be set such that for a VM with two virtual processors, any virtual processor that exerts a load of less than 25% of the total capacity of the physical processor may be reassigned. Thus, if the hypervisor determines that each of the virtual processors of the VM only places a load of 10% of the capacity of the physical processor, the hypervisor may determine that that the threshold condition for reassignment has been met. The hypervisor may then identify a single host processor capable of processing the load for the plurality of virtual processors of the virtual machine and assign the virtual processors to the identified host processor. Subsequently, the hypervisor can enable polling of the shared device and notify the guest OS that polling has been enabled.
  • Aspects of the present disclosure are thus capable of limiting CPU usage associated with polling by the host operating system. More particularly, aspects of the present disclosure prevent continued polling of the shared memory from wasting CPU resources in the event that any virtual processor of the VM has been pre-empted.
  • FIG. 1 depicts a high-level component diagram of an illustrative example of a computer system 100, in accordance with one or more aspects of the present disclosure. One skilled in the art will appreciate that other architectures for computer system 100 are possible, and that the implementation of a computer system utilizing examples of the invention are not necessarily limited to the specific architecture depicted by FIG. 1.
  • As shown in FIG. 1, the computer system 100 is connected to a network 150 and comprises one or more central processing units (CPUs) 160-1 through 160-N, where N is a positive integer, main memory 170, which may include volatile memory devices (e.g., random access memory (RAM)), non-volatile memory devices (e.g., flash memory) and/or other types of memory devices, and a storage device 180 (e.g., one or more hard disk drives, solid-state drives, etc.). In certain implementations, main memory 170 may be non-uniform access (NUMA), such that memory access time depends on the memory location relative to CPUs 160-1 through 160-N.
  • The computer system 100 may be a server, a mainframe, a workstation, a personal computer (PC), a mobile phone, a palm-sized computing device, etc. The network 150 may be a private network (e.g., a local area network (LAN), a wide area network (WAN), intranet, etc.) or a public network (e.g., the Internet).
  • Computer system 100 may additionally comprise virtual machine (VM) 130, host operating system (OS) 120, and shared device 190. VM 130 is a software implementation of a machine that executes programs as though it were an actual physical machine. Host OS 120 manages the hardware resources of the computer system and provides functions such as inter-process communication, scheduling, memory management, and so forth. Shared device 190 is a data structure in shared memory accessible to both guest OS 135 of VM 130 and hypervisor 125 of host OS 120.
  • VM 130 comprises a guest operating system (OS) 135 that handles the execution of applications within the virtual machine, and one or more virtual processors 131-1 through 131-N, where N is a positive integer. One or more virtual processors 131-1 through 131-N may be associated with a shared device 190. Guest OS 135 may write requests to shared device 190 to be processed by host OS 190. It should be noted that although, for simplicity, a single virtual machine 130 is depicted in FIG. 1, computer system 100 may host a plurality of VMs 130.
  • Host OS 120 may comprise a hypervisor 125, which provides a virtual operating platform for virtual machine 130 and manages its execution. Hypervisor 125 may comprise polling manager 128, virtual processor data structures 126, and virtual processor mapping table 127. It should be noted that in some alternative implementations, hypervisor 125 may be external to host OS 120, rather than embedded within host OS 120, or may replace host OS 120.
  • Polling manager 128 can identify requests in a shared device 190 of guest OS 135, process the requests, and determine whether continued polling of the shared device should be disabled based on the execution state of the virtual processors associated with VM 130, as described in detail below with respect to FIG. 2. Polling manager 128 may also determine whether polling of the shared device should be re-enabled based on the processing load of all virtual processors of VM 130, as described in detail below with respect to FIG. 3.
  • Hypervisor 125 can use virtual processor data structures 126 to maintain information regarding virtual processors 131-1 through 131-N of VM 130. Hypervisor 125 may store the execution state of virtual processors 131-1 through 131-N in virtual processor data structures 126 for use by polling manager 128 in determining whether to disable polling of shared device 190. Hypervisor 125 may update virtual processor data structures 126 each time the execution state of any of virtual processors 131-1 through 131-N changes. For example, if the execution state one of the virtual processors changes from running to pre-empted. Hypervisor 125 may also store a unique identifier in virtual processor data structures 126 that maps to a specific virtual processor of virtual processors 131-1 through 131-N for use by polling manager 128. Hypervisor 125 may create separate virtual data structures 126 for each of virtual processors 131-1 through 131-N, or in the alternative, create a single virtual processor data structure 126 for all virtual processors 131-1 through 131-N. Virtual processor data structures 126 may be a memory location within hypervisor 125. Alternatively, virtual processor data structures 126 may be written to a location in storage device 180.
  • Hypervisor 125 can use virtual processor mapping table 127 to store cross reference information regarding which of the virtual processors 131-1 through 131-N of VM 130 are associated with shared device 190. Hypervisor 125 may store a unique identifier that is associated with each of the virtual processors 131-1 through 131-N, along with an identifier for the shared memory location of the associated shared device 190 for use by polling manager 128. If hypervisor 125 determines that a virtual processor of VM 130 needs to be reassigned to another physical processor as described below in FIG. 3, it may use virtual processor mapping table 127 to assist with identification of eligible physical processors. For example, if a virtual processor is associated with a shared device, and a host processor normally conducts polling of that shared device, then that host processor may not be deemed eligible to assume the load for all virtual processors since it could interfere with polling operations. Virtual processor mapping table 127 may be a memory location within hypervisor 125. Alternatively, virtual processor mapping table 127 may be written to a location in storage device 180.
  • FIG. 2 depicts a flow diagram of an example method 200 for disabling polling of a shared device based on the execution states of each virtual processor of a virtual machine. The method may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both. In one illustrative example, method 200 may be performed by polling manager 128 of hypervisor 125 in FIG. 1. Alternatively, some or all of method 200 might be performed by another machine. It should be noted that blocks depicted in FIG. 2 could be performed simultaneously or in a different order than that depicted.
  • At block 201, processing logic identifies a request of a guest operating system associated with a shared device. The shared device may be represented by a data structure in shared memory, such as shared device 190 which is accessible to both guest OS 135 of VM 130 and hypervisor 125 in FIG. 1. In some implementations, the VM may have a plurality of virtual processors, such as virtual processors 131-1 through 131-N of VM 130 in FIG. 1. At block 202, processing logic processes the request associated with the shared device. At block 203, processing logic polls the shared device for additional requests of the guest operating system.
  • At block 204, processing logic determines whether there are additional requests of the guest operating system associated with the shared device to be processed. If processing logic determines that there are additional requests in the shared device to be processed, processing logic returns to block 202 to process the requests and continue polling. Otherwise, execution proceeds to block 205. In one illustrative example, before proceeding to block 205, processing logic may first determine whether no additional requests were associated with the shared device for a period of time that exceeds a predetermined threshold. If processing logic determines that the threshold has not yet been met, execution returns to block 202, otherwise execution proceeds to block 205. In certain implementations, the threshold value may represent a total amount of time elapsed during polling of a shared device. Alternatively, the threshold value may be a total amount of CPU clock cycles executed during polling of a shared device. The threshold may be defined by an administrator (e.g., via a configuration file, via a graphical user interface, etc.), hard-coded in the hypervisor, or set in any other manner.
  • At block 205, processing logic determines the execution state of each virtual processor of the virtual machine (e.g., the virtual processor is currently running and busy, currently running but idle, not currently running, pre-empted by the host, etc.). The various execution states may be defined by an administrator (e.g., via a configuration file, via a graphical user interface, etc.), hard-coded in polling manager 128 of hypervisor 125 in FIG. 1, or set in any other manner. Processing logic may access a data structure that stores the execution state of the virtual processor such as virtual processor data structures 126 of FIG. 1. In one illustrative example, processing logic can use the unique identifier of the virtual processor from block 205 to access the applicable data structure for the virtual processor. In certain implementations, the data structure may contain the current execution state of the applicable virtual processor.
  • At block 206, processing logic branches based on the determination of block 205. If processing logic determined that at least one virtual processor of the virtual machine has been pre-empted (e.g., if the host has suspended execution of the virtual processor in order to devote the associated host processor to a host-related task), execution continues to block 207, otherwise execution returns to block 203 to continue polling.
  • At block 207, processing logic disables polling of the shared device. At block 208, processing logic notifies the guest OS that polling has been disabled. This notification may be made via an interrupt request, or alternatively, via a data structure stored in shared memory. In certain implementations, the notification may also indicate to the guest OS the reason that polling is disabled (e.g., the virtual processor is pre-empted, etc.). After block 208, the method of FIG. 2 terminates.
  • FIG. 3 depicts a flow diagram of an example method 300 for enabling polling of a shared device based on the processing load of all virtual processors of a virtual machine. The method may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both. In one illustrative example, method 300 may be performed by polling manager 128 of hypervisor 125 in FIG. 1. Alternatively, some or all of method 300 might be performed by another machine. It should be noted that blocks depicted in FIG. 3 could be performed simultaneously or in a different order than that depicted.
  • At block 301, processing logic may first analyze the load placed by each of the virtual processors of the VM on the host. At block 302, processing logic determines whether the load placed by each virtual processor of the VM on the associated physical processor meets a predefined threshold condition. If processing logic determines that the threshold has not yet been met, the method of FIG. 3 terminates, otherwise execution proceeds to block 303. The threshold condition may be implemented as a percentage of CPU load, the total amount of CPU clock cycles executed during a particular time period, or in any similar manner.
  • For example, the predefined threshold condition may be set such that for a VM with two virtual processors, any virtual processor that exerts a load of less than 25% of the total capacity of the physical processor may reassigned. Thus, if the hypervisor determines that each of the virtual processors of the VM only places a load of 10% of the capacity of the physical processor, the hypervisor may determine that that the threshold condition for reassignment has been met.
  • At block 303, processing logic may identify a single host processor capable of processing the load for the plurality of the virtual processors of that virtual machine collectively. At block 304, processing logic assigns the plurality of virtual processors to the single host processor identified at block 303. At block 305, processing logic enables polling of the shared device. At block 306, processing logic notifies the guest OS that polling has been enabled. This notification may be made via an interrupt request, or alternatively, via a data structure stored in shared memory. In certain implementations, the notification may also indicate to the guest OS the reason that polling has been enabled. After block 306, the method of FIG. 3 terminates.
  • FIG. 4 depicts an example computer system 400 which can perform any one or more of the methods described herein. In one example, computer system 400 may correspond to computer system 100 of FIG. 1. The computer system may be connected (e.g., networked) to other computer systems in a LAN, an intranet, an extranet, or the Internet. The computer system may operate in the capacity of a server in a client-server network environment. The computer system may be a personal computer (PC), a set-top box (STB), a server, a network router, switch or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device. Further, while only a single computer system is illustrated, the term “computer” shall also be taken to include any collection of computers that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methods discussed herein.
  • The exemplary computer system 400 includes a processing system (processor) 402, a main memory 404 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)), a static memory 406 (e.g., flash memory, static random access memory (SRAM)), and a data storage device 416, which communicate with each other via a bus 408.
  • Processor 402 represents one or more general-purpose processing devices such as a microprocessor, central processing unit, or the like. More particularly, the processor 402 may be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets. The processor 402 may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. The processor 402 is configured to execute instructions 426 for performing the operations and steps discussed herein.
  • The computer system 400 may further include a network interface device 422. The computer system 400 also may include a video display unit 410 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 412 (e.g., a keyboard), a cursor control device 414 (e.g., a mouse), and a signal generation device 420 (e.g., a speaker).
  • The data storage device 416 may include a computer-readable medium 424 on which is stored one or more sets of instructions 426 (e.g., instructions corresponding to the methods of FIGS. 2 and 3, etc.) embodying any one or more of the methodologies or functions described herein. Instructions 426 may also reside, completely or at least partially, within the main memory 404 and/or within the processor 402 during execution thereof by the computer system 400, the main memory 404 and the processor 402 also constituting computer-readable media. Instructions 426 may further be transmitted or received over a network via the network interface device 422.
  • While the computer-readable storage medium 424 is shown in the illustrative examples to be a single medium, the term “computer-readable storage medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term “computer-readable storage medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present invention. The term “computer-readable storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.
  • Although the operations of the methods herein are shown and described in a particular order, the order of the operations of each method may be altered so that certain operations may be performed in an inverse order or so that certain operation may be performed, at least in part, concurrently with other operations. In certain implementations, instructions or sub-operations of distinct operations may be in an intermittent and/or alternating manner.
  • It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other implementations will be apparent to those of skill in the art upon reading and understanding the above description. The scope of the invention should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
  • In the above description, numerous details are set forth. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form, rather than in detail, in order to avoid obscuring the present invention.
  • Some portions of the detailed descriptions above are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
  • It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise, as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as “receiving”, “determining”, “allocating”, “notifying”, or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
  • The present invention also relates to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.
  • The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will appear as set forth in the description below. In addition, the present invention is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the invention as described herein.
  • The present invention may be provided as a computer program product, or software, that may include a machine-readable medium having stored thereon instructions, which may be used to program a computer system (or other electronic devices) to perform a process according to the present invention. A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., a computer) readable storage medium (e.g., read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, etc.), a machine (e.g., computer) readable transmission medium (electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.)), etc.

Claims (20)

What is claimed is:
1. A method comprising:
identifying, by a processing device executing a hypervisor, a request of a guest operating system of a virtual machine, the request associated with a shared device, the virtual machine having a plurality of virtual processors;
processing, by the hypervisor, the request of the guest operating system;
polling, by the hypervisor, the shared device for additional requests of associated with the guest operating system; and
disabling, by the hypervisor, polling of the shared device upon determining that the execution state of at least one virtual processor indicates that the at least one virtual processor has been pre-empted.
2. The method of claim 1 further comprising determining, by the hypervisor, that no additional requests of the guest operating system are associated with the shared device.
3. The method of claim 2 further comprising:
notifying, by the hypervisor, the guest operating system of the virtual machine that the polling of the shared device has been disabled upon disabling the polling of the shared device;
determining, by the hypervisor, whether the execution state of the at least one virtual processor of the virtual machine has changed;
enabling, by the hypervisor, the polling of the shared device for additional requests upon determining that the execution state of the at least one virtual processor indicates that the at least one virtual processor is not pre-empted; and
notifying, by the hypervisor, the guest operating system of the virtual machine that the polling of the shared device has been enabled upon enabling the polling of the shared device.
4. The method of claim 1, wherein determining that no additional requests of the guest operating system are associated with the shared device comprises:
determining, by the hypervisor, that no additional requests of the guest operating system are associated with the shared device for a period of time that exceeds a predetermined threshold.
5. The method of claim 2 further comprising:
assigning, by the hypervisor, the plurality of virtual processors of the virtual machine to a single host processor;
enabling, by the hypervisor, the polling of the shared device for the additional requests; and
notifying, by the hypervisor, the guest operating system of the virtual machine that the polling of the shared device has been enabled upon enabling the polling of the shared device.
6. The method of claim 5, wherein assigning the plurality of virtual processors of the virtual machine to the single host processor comprises:
analyzing, by the hypervisor, the load placed by each of the plurality of virtual processors on the host;
determining, by the hypervisor, that the load placed by each of the plurality of virtual processors on the host meets a predefined threshold condition;
identifying, by the hypervisor, the single host processor capable of processing the load for the plurality of virtual processors of the virtual machine; and
assigning, by the hypervisor, the plurality of virtual processors of the virtual machine to the identified host processor.
7. The method of claim 1 wherein the shared device is associated with a shared memory space between the virtual machine and the hypervisor.
8. A computing apparatus comprising:
a memory to store instructions; and
a processing device, coupled to the memory, to execute the instructions, wherein the processing device is to:
identify a request of a guest operating system of a virtual machine, the request associated with the shared device, the virtual machine having a plurality of virtual processors;
process the request of the guest operating system;
poll the shared device for additional requests of the guest operating system; and
disable the polling of the shared device for the additional requests upon determining that the execution state of at least one virtual processor indicates that the at least one virtual processor has been pre-empted.
9. The apparatus of claim 8 wherein the processing device is further to determine that no additional requests of the guest operating system are associated with the shared device.
10. The apparatus of claim 9 wherein the processing device is further to:
notify, via the hypervisor, the guest operating system of the virtual machine that polling has been disabled upon disabling polling of the shared device;
determine, by the hypervisor, whether the execution state of the at least one virtual processor of the virtual machine has changed;
enable, by the hypervisor, the polling of the shared device for additional requests upon determining that the execution state of the at least one virtual processor indicates that the at least one virtual processor not pre-empted; and
notify, by the hypervisor, the guest operating system of the virtual machine that the polling of the shared device has been enabled upon enabling the polling of the shared device.
11. The apparatus of claim 8, wherein to determine that no additional requests of the guest operating system are associated with the shared device, the processing device is to:
determine, by the hypervisor, that no additional requests of the guest operating system are associated with the shared device for a period of time that exceeds a predetermined threshold.
12. The apparatus of claim 9 wherein the processing device is further to:
assign, by the hypervisor, the plurality of virtual processors of the virtual machine to a single host processor;
enable, by the hypervisor, the polling of the shared device for the additional requests; and
notify, by the hypervisor, the guest operating system of the virtual machine that the polling of the shared device has been enabled upon enabling the polling of the shared device.
13. The apparatus of claim 12, wherein to assign the plurality of virtual processors of the virtual machine to a single host processor, the processing device is to:
analyze, by the hypervisor, the load placed by each of the plurality of virtual processors on the host;
determine, by the hypervisor, that the load placed by each of the plurality of virtual processors on the host meets a predefined threshold condition;
identify, by the hypervisor, the single host processor capable of processing the load for the plurality of virtual processors of the virtual machine; and
assign, by the hypervisor, the plurality of virtual processors of the virtual machine to the identified host processor.
14. The apparatus of claim 8 wherein the shared device is associated with a shared memory space between the virtual machine and the hypervisor.
15. A non-transitory computer readable storage medium, having instructions stored therein, which when executed by a processing device of a computer system, cause the processing device to perform operations comprising:
identifying, by the processing device executing a hypervisor, a request of a guest operating system of a virtual machine, the request associated with a shared device, the virtual machine having a plurality of virtual processors;
processing, by the hypervisor, the request of the guest operating system;
polling, by the hypervisor, the shared device for additional requests of the guest operating system;
determining, by the hypervisor, that no additional requests of the guest operating system are associated with the shared device;
disabling, by the hypervisor, the polling of the shared device upon determining that the execution state of at least one virtual processor indicates that the at least one virtual processor has been pre-empted; and
notifying, by the hypervisor, the guest operating system of the virtual machine that polling has been disabled in response to disabling polling of the shared device.
16. The non-transitory computer readable storage medium of claim 15, wherein the operations further comprise:
determining, by the hypervisor, whether the execution state of the first virtual processor of the virtual machine has changed;
enabling, by the hypervisor, polling the shared device for requests in response to determining that the first virtual processor is not pre-empted; and
notifying, by the hypervisor, the guest operating system of the virtual machine that polling has been enabled upon enabling polling of the shared device.
17. The non-transitory computer readable storage medium of claim 15, wherein determining that no additional requests of the guest operating system are associated with the shared device comprises:
determining, by the hypervisor, that no additional requests of the guest operating system are associated with the shared device for a period of time that exceeds a predetermined threshold.
18. The non-transitory computer readable storage medium of claim 15, wherein the operations further comprise:
assigning, by the hypervisor, the plurality of virtual processors of the virtual machine to a single host processor;
enabling, by the hypervisor, the polling of the shared device for the additional requests; and
notifying, by the hypervisor, the guest operating system of the virtual machine that the polling of the shared device has been enabled upon enabling the polling of the shared device.
19. The non-transitory computer readable storage medium of claim 18, wherein assigning the plurality of virtual processors of the virtual machine to the single host processor comprises:
analyzing, by the hypervisor, the load placed by each of the plurality of virtual processors on the host;
determining, by the hypervisor, that the load placed by each of the plurality of virtual processors on the host meets a predefined threshold condition;
identifying, by the hypervisor, the single host processor capable of processing the load for the plurality of virtual processors of the virtual machine; and
assigning, by the hypervisor, the plurality of virtual processors of the virtual machine to the identified host processor.
20. The non-transitory computer readable storage medium of claim 15 wherein the shared device is associated with a shared memory space between the virtual machine and the hypervisor.
US14/527,661 2014-10-29 2014-10-29 Limited virtual device polling based on virtual CPU pre-emption Active 2034-11-09 US9766917B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/527,661 US9766917B2 (en) 2014-10-29 2014-10-29 Limited virtual device polling based on virtual CPU pre-emption

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US14/527,661 US9766917B2 (en) 2014-10-29 2014-10-29 Limited virtual device polling based on virtual CPU pre-emption

Publications (2)

Publication Number Publication Date
US20160124763A1 true US20160124763A1 (en) 2016-05-05
US9766917B2 US9766917B2 (en) 2017-09-19

Family

ID=55852747

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/527,661 Active 2034-11-09 US9766917B2 (en) 2014-10-29 2014-10-29 Limited virtual device polling based on virtual CPU pre-emption

Country Status (1)

Country Link
US (1) US9766917B2 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160179725A1 (en) * 2014-12-22 2016-06-23 Seoul National University R&Db Foundation Input/output method in virtual machine environments
US20170295107A1 (en) * 2016-04-07 2017-10-12 International Business Machines Corporation Specifying a disaggregated compute system
US9836323B1 (en) 2017-02-24 2017-12-05 Red Hat, Inc. Scalable hypervisor scheduling of polling tasks
US20210042143A1 (en) * 2018-03-20 2021-02-11 Mitsubishi Electric Corporation Information processing device, method, and program

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7765543B1 (en) * 2003-12-17 2010-07-27 Vmware, Inc. Selective descheduling of idling guests running on a host computer system
US20120089980A1 (en) * 2010-10-12 2012-04-12 Richard Sharp Allocating virtual machines according to user-specific virtual machine metrics
US20130060837A1 (en) * 2011-09-07 2013-03-07 Microsoft Corporation Monitoring And Benchmarking Client Performance From The Server-Side
US20140351471A1 (en) * 2013-05-21 2014-11-27 Arm Limited Handling and routing interrupts to virtual processes
US20140380309A1 (en) * 2013-06-19 2014-12-25 Hitachi, Ltd. Virtual machine system and method of measuring processor performance
US20150026687A1 (en) * 2013-07-18 2015-01-22 International Business Machines Corporation Monitoring system noises in parallel computer systems
US20150081943A1 (en) * 2013-09-13 2015-03-19 International Business Machines Corporation Virtual machine suspension in checkpoint system
US20150248303A1 (en) * 2014-02-28 2015-09-03 Red Hat Israel, Ltd. Paravirtualized migration counter
US20150339216A1 (en) * 2014-05-22 2015-11-26 Citrix Systems, Inc. Providing Testing Environments Using Virtualization
US20150347166A1 (en) * 2014-06-02 2015-12-03 Red Hat Israel, Ltd. Paravirtualized migration counter for migrating a virtual cpu to a different physical cpu
US20160077571A1 (en) * 2014-09-12 2016-03-17 Microsoft Corporation Heuristic Processor Power Management in Operating Systems
US20160154710A1 (en) * 2014-12-01 2016-06-02 Citrix Systems, Inc. Live rollback for a computing environment
US20170091047A1 (en) * 2015-09-30 2017-03-30 Commvault Systems, Inc. Dynamic triggering of block-level backups based on block change thresholds and corresponding file identities in a data storage management system

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8171488B1 (en) 2007-02-16 2012-05-01 Vmware, Inc. Alternating scheduling and descheduling of coscheduled contexts
US9009727B2 (en) 2008-05-30 2015-04-14 Vmware, Inc. Virtualization with in-place translation
US9600332B2 (en) 2009-04-28 2017-03-21 Cisco Technology, Inc. Server load balancing based on virtual utilization, physical utilization, and feedback
US8122167B1 (en) * 2010-08-06 2012-02-21 International Business Machines Corporation Polling in a virtualized information handling system
US8468318B2 (en) 2010-09-15 2013-06-18 Pure Storage Inc. Scheduling of I/O writes in a storage environment
JP5846836B2 (en) 2011-10-11 2016-01-20 株式会社日立製作所 Virtual machine, virtual machine system, and virtual machine control method
US8924501B2 (en) * 2011-11-30 2014-12-30 Red Hat Israel, Ltd. Application-driven shared device queue polling
US9009702B2 (en) 2011-11-30 2015-04-14 Red Hat Israel, Ltd. Application-driven shared device queue polling in a virtualized computing environment
US10019159B2 (en) 2012-03-14 2018-07-10 Open Invention Network Llc Systems, methods and devices for management of virtual memory systems
US9164809B2 (en) * 2012-09-04 2015-10-20 Red Hat Israel, Ltd. Virtual processor provisioning in virtualized computer systems
US10114662B2 (en) * 2013-02-26 2018-10-30 Red Hat Israel, Ltd. Updating processor topology information for virtual machines
US9910689B2 (en) * 2013-11-26 2018-03-06 Dynavisor, Inc. Dynamic single root I/O virtualization (SR-IOV) processes system calls request to devices attached to host
US9600314B2 (en) * 2014-05-27 2017-03-21 Red Hat Israel, Ltd. Scheduler limited virtual device polling

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7765543B1 (en) * 2003-12-17 2010-07-27 Vmware, Inc. Selective descheduling of idling guests running on a host computer system
US20120089980A1 (en) * 2010-10-12 2012-04-12 Richard Sharp Allocating virtual machines according to user-specific virtual machine metrics
US20130060837A1 (en) * 2011-09-07 2013-03-07 Microsoft Corporation Monitoring And Benchmarking Client Performance From The Server-Side
US9378162B2 (en) * 2013-05-21 2016-06-28 Arm Limited Handling and routing interrupts to virtual processors
US20140351471A1 (en) * 2013-05-21 2014-11-27 Arm Limited Handling and routing interrupts to virtual processes
US20140380309A1 (en) * 2013-06-19 2014-12-25 Hitachi, Ltd. Virtual machine system and method of measuring processor performance
US20150026687A1 (en) * 2013-07-18 2015-01-22 International Business Machines Corporation Monitoring system noises in parallel computer systems
US20150081943A1 (en) * 2013-09-13 2015-03-19 International Business Machines Corporation Virtual machine suspension in checkpoint system
US9323553B2 (en) * 2013-09-13 2016-04-26 International Business Machines Corporation Reducing virtual machine suspension time in checkpoint system
US20150248303A1 (en) * 2014-02-28 2015-09-03 Red Hat Israel, Ltd. Paravirtualized migration counter
US20150339216A1 (en) * 2014-05-22 2015-11-26 Citrix Systems, Inc. Providing Testing Environments Using Virtualization
US20150347166A1 (en) * 2014-06-02 2015-12-03 Red Hat Israel, Ltd. Paravirtualized migration counter for migrating a virtual cpu to a different physical cpu
US20160077571A1 (en) * 2014-09-12 2016-03-17 Microsoft Corporation Heuristic Processor Power Management in Operating Systems
US20160154710A1 (en) * 2014-12-01 2016-06-02 Citrix Systems, Inc. Live rollback for a computing environment
US20170091047A1 (en) * 2015-09-30 2017-03-30 Commvault Systems, Inc. Dynamic triggering of block-level backups based on block change thresholds and corresponding file identities in a data storage management system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Guan et al., "SR-IOV Based Network Interrupt-Free Virtualization with Event Based Polling", IEEE, December 2013, Vol. 31, No. 12, pp. 2596-2609;<http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6678107> *
Liu et al., "Virtualization Polling Engine (VPE): Using Dedicated CPU Cores to Accelerate I/O Virtualization", ACM, ICS’09, June 2009, pp. 225-234;<http://dl.acm.org/citation.cfm?id=1542309&CFID=761601880&CFTOKEN=98351079> *
Uhlig et al., "Towards Scalable Multiprocessor Virtual Machines", USENIX, VM’04, May 2004, 3rd Virtual Machine Research & Technology Symposium, pp. 1-15;<https://www.usenix.org/event/vm04/tech/full_papers/uhlig/uhlig.pdf> *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160179725A1 (en) * 2014-12-22 2016-06-23 Seoul National University R&Db Foundation Input/output method in virtual machine environments
US9910804B2 (en) * 2014-12-22 2018-03-06 Samsung Electronics Co., Ltd. Input/output method in virtual machine environments
US20170295107A1 (en) * 2016-04-07 2017-10-12 International Business Machines Corporation Specifying a disaggregated compute system
US11153223B2 (en) * 2016-04-07 2021-10-19 International Business Machines Corporation Specifying a disaggregated compute system
US9836323B1 (en) 2017-02-24 2017-12-05 Red Hat, Inc. Scalable hypervisor scheduling of polling tasks
US20210042143A1 (en) * 2018-03-20 2021-02-11 Mitsubishi Electric Corporation Information processing device, method, and program
US11656898B2 (en) * 2018-03-20 2023-05-23 Mitsubishi Electric Corporation Information processing device, method, and program

Also Published As

Publication number Publication date
US9766917B2 (en) 2017-09-19

Similar Documents

Publication Publication Date Title
US9697029B2 (en) Guest idle based VM request completion processing
US9164809B2 (en) Virtual processor provisioning in virtualized computer systems
US9600314B2 (en) Scheduler limited virtual device polling
US8312195B2 (en) Managing interrupts using a preferred binding between a device generating interrupts and a CPU
US10365936B2 (en) Idle processor management by guest in virtualized systems
US9201823B2 (en) Pessimistic interrupt affinity for devices
US10409633B2 (en) Hypervisor-visible guest thread management
US9959134B2 (en) Request processing using VM functions
US9003094B2 (en) Optimistic interrupt affinity for devices
US10394586B2 (en) Using capability indicators to indicate support for guest driven surprise removal of virtual PCI devices
US20160246636A1 (en) Cross hypervisor migration of virtual machines with vm functions
US20150193249A1 (en) Idle processor management in virtualized systems via paravirtualization
US20170024231A1 (en) Configuration of a computer system for real-time response from a virtual machine
US9256455B2 (en) Delivery of events from a virtual machine to host CPU using memory monitoring instructions
US9766917B2 (en) Limited virtual device polling based on virtual CPU pre-emption
US10146589B2 (en) Processor unplug in virtualized computer systems
US10387178B2 (en) Idle based latency reduction for coalesced interrupts
JP2009223842A (en) Virtual machine control program and virtual machine system
US11237859B2 (en) Securing virtual machines in computer systems
US9575788B2 (en) Hypervisor handling of processor hotplug requests
US20170147501A1 (en) Post-copy vm migration speedup using free page hinting
US10671425B2 (en) Lazy timer programming for virtual machines
US9792152B2 (en) Hypervisor managed scheduling of virtual machines

Legal Events

Date Code Title Description
AS Assignment

Owner name: RED HAT ISRAEL, LTD., ISRAEL

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TSIRKIN, MICHAEL S.;REEL/FRAME:034076/0228

Effective date: 20141029

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction
MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4