Multi-OS gaming w/o dual-booting: Excelent graphics performance in a VM with VGA passthrough

Note: This articles is a technology/technique outline, not a detailed guide and not a how-to. It explains what is VGA passthrough, why you might be interested in it, and where to start.

Even with the current abundance of Linux native games (both indies and AAAs), with WINE reliably running almost any not-so-new software, many gamers who use Linux on a daily basis tend to switch to Windows for playing games. Regardless of one’s attitude towards non-free software, it has to be admitted that if you wish to try out some of the newest titles, you have no other choice than running them on a Windows installation. This is why so many gamers dual-boot: having installed two operating systems on the same machine and using Windows for playing games and Linux for virtually anything else, they limit their usage of Microsoft’s OS for gaming only. This popular technique seems handy – you get the luxury of using a Linux, and the gaming performance of Windows.

But dual-booting is annoying because of the need of reboot to switch your context. Need to IM your friend while playing? Save your game, shut down Windows, reboot to Linux, launch IM, reboot to Windows, load your game. Switching takes a long time, is inconvenient, and therefore the player may feel discouraged to do so.

What if you could run both operating systems at once? That’s nothing new, run a virtual machine in your Linux, install Windows within it, and voilà! But a virtual machine is no good for gaming, the performance will be utter cr terrible. Playing chess might work, but any 3D graphics won’t do because of the lack of hardware acceleration. The VM emulates a simple graphics adapter to display it’s output in a window of the host OS.

And that is where VGA passthrough comes in, and solves this issue.

1. The idea

The key to getting neat graphics in a VM is to grant the virtual machine a full access to your graphics card. This means that your host OS will not touch this piece of hardware at all, and the guest OS will be able to use it as any other (emulated) hardware. The guest OS (presumably Windows) will load it’s own drivers for the graphics adapter, and will communicate with it natively! Therefore it will have full access to hardware acceleration and any other goodies that gear might provide (eg. HDMI audio). The idea of passing a VGA adapter to a virtual machine is usually named VGA passthrough.

Sounds crazy? Let me tease you: my setup is capable of smoothly running Watch_Dogs, Tomb Raider (2013) on Ultra settings at 60+ FPS within that virtual machine, using NVIDIA’s GTX 770. And I get the luxury of running both OS at once – so I can switch between them in just a glimpse, without shutting down either one! This is astonishingly convenient.

Because the dedicated graphics hardware will be reserved for the guest system, the host will need another graphics adapter to display anything. So there comes the first hardware requirement: you need at least two graphic adapters. However, it is not uncommon – many new Intel processors have a build-in GMA – and if you are a gamer, chances are you have invested in a dedicated graphics card – so that makes two graphics adapters already. Let the host system use integrated graphics, and the guest will get the powerful dedicated graphics for games. Because both graphic adapters will work independently and there is no way to compose their video output¹, you will need two separate displays, one for each system. This means either a set of two monitors, or a monitor with two video inputs (so that you can switch between them). You might also experiment with a KVM switch.

Also keep in mind that it is not an easy thing to set up. While some claim they have succeeded on their first try, many others have struggled a lot. Personally, I spend about two weeks tuning things up to get my VGA passthrough running – and if we count hardware searching and preparations, then it took me two months. But it was completely worth it! My current setup contains of:

  • Intel i7-4790K (4 x 2 x 4.0GHz)
  • ASRock Z97 Extreme6
  • NVIDIA GTX 770 4GB
  • and some 16 gigs of RAM
  • also, a monitor with multiple video inputs (I switch video source using buttons on the monitor)
  • Ubuntu 14.04

As I have mentioned, this set is capable of running very demanding games at maxed settings with amazing results. How does it work in practice? It feels as if I was running both systems at once. For example, while playing a game under Windows, my Linux has an IM client running. Because I mix the sound from both systems, I can hear the notification when I get a message. So I pause the game, switch monitor video source with a hotkey shortcut, respond to the message, and switch the video back. If only I had two monitors, I would play on one of them, with the host system using the other one – so I wouldn’t even need to touch the monitor to switch the OS, I would just need to rotate my head a little bit ;-)

Getting here was a lot of work, but a lot of fun too! The first step is to meet the…

2. Hardware requirements

Yeah. Not every machine will be able to do this trick. As already mentioned, you need two graphics adapters. However, it is not possible to passthrough the graphics integrated in your CPU! This is because passthrough works by separating a PCI device from the host system, and attaching it to the guest OS. Therefore you can only pass a dedicated graphics hardware. Not much of a problem, probably, but it’s probably an important note.

You also need to ensure that your CPU and mainboard support IOMMU – extensions for I/O visualisation, which are necessary for passing through a PCI device. Intel calls their IOMMU technology “VT-d“, while AMD refers to it as “AMD-V“. This is an absolute must, so if you are buying new hardware, make sure both your processor and the chipset will support IOMMU²!

Also, if you plan to use a CPU integrated graphics adapter for the host system, make sure that the mainboard supports it, and that it has a video output!

You will get best results if you use a multi-core CPU. Demanding games will require not only powerful graphics hardware, but a decent CPU as well! It is possible to reserve some of CPU’s cores for the VM – this way you can ensure that the guest OS will be granted enough computational power. For example, in my setup, the host OS uses 2 cores, while other 6 are at Windows’ disposal.

Also, as explained, you need a monitor with several inputs, or a set of two. I am not aware of any way to get this working on a laptop, as most of laptops I know have just one monitor, and you cannot manually switch between video sources¹.

So the full list of requirements is:

  • IOMMU compatible CPU and mainboard
  • A dedicated PCI graphics adapter (for passing through)
  • Graphics hardware for the host OS (can be integrated in CPU)
  • Monitor with multiple video inputs (recommended two monitors)
  • (Recommended: multi-core CPU).

Warning: Note that you DO NOT NEED a multi-OS graphics card! Contrary to popular belief, non-Quatro NVIDIA cards will work well, with no hardware modifications of any kind!

3. Methods

There are two popular passthrough techniques – one involves Xen virtualization, and one using Qemu and VFIO. Having played around with both, I am personally a fan of the Qemu way – it seems it is much easier to set up, I get more control over my VM, customizations are easier, and, most importantly, it works with virtually any PCI graphics adapter!

There is a lot of confusion on the Internet concerning what results each method may yield. Some say that Qemu method can never grant any decent performance, they claim that only Xen can perform primary VGA passthrough, while Qemu’s secondary VGA passthrough will be very inefficient. However, numerous people (including me) confirm that they have awesome performance with Qemu. On the other hand, it is clear that passthrough with Xen will only work with multi-OS graphic cards. This is not a problem for Radeon users, as probably all new Radeons will do just fine with Xen. However, if your NVIDIA card is not an NVIDIA Quadro, you have no chances with Xen! – unless you burn several resistors on the board, which can mod your card so that it thinks it is a Quadro… I do not recommend such hardware modifications to anyone, even if you trust the Internet too much, the risk of rendering your precious hardware useless is far too high to make it work the effort. Qemu, on the other hand, should work well with absolutely any PCI card.

Given these reasons, as well as customization options, I have decided to stick with Qemu. For the rest of this article, I will be describing this particular method.

There is one particular comprehensive guide on how to setup everything using the Qemu method here – at the time of writing this forums thread has more than 2500 replies, so learning details from here may be hard, but on the other hand every possible scenario is covered somewhere in there :) I can highly recommend that guide, but if you want to learn about the general idea first, stay with me before you jump there!

4. The software

Obviously things won’t work out of the box. There are also necessary preparations on the software side.

First, you will need to patch your kernel a bit, and compile it with several options enabled. At the time of writing, ASC override patches and VGA arbiter fixes need to be applied manually, as they are not (yet?) included in the kernel. You can find details in that guide I linked.

You will need to configure your kernel a bit. The key is not only to ensure it activates appropriate IOMMU modules, but also to forbid it from loading any drivers to the card you will want to pass through.

Most likely it will be also necessary to use the git development version of Qemu – some necessary features are not yet available in stable releases. Also, when playing with qemu, it is worth to try KVM – chances are that hardware virtualization might significantly improve virtual machine’s CPU performance.

You may want to write a bit of scripts that set up few other details (binding the PCI card to vfio module) before you start qemu to run the virtual machine.

Also, it may be tricky to get the right order of installing drivers in the guest OS. It took me a while to realize that I need to disable qemu’s emulated VGA – otherwise NVIDIA drivers won’t detect the dedicated hardware :-)

The greatest issue I have met is that Windows is very sensitive to hardware changes. Even slightest changes in my virtual machine (different qemu options) would immediately cause my Windows to never boot anymore, and any web guides on dealing with these particular BSoDs on boot never helped… So eventually I had to re-install the whole guest OS, after ~10 times I am completely fed up with it. However, if I do not experiment with qemu settings, there are no such problems at all.

There is one more thing that I believe is worth setting up. It’s cool to play Windows games, but there are also many great Linux-native titles. Obviously, if your system boots up with disabled dedicated graphics hardware, any demanding game won’t run. For this reason I have configured my GRUB so that I can choose whether I wish to boot my system to use the graphics card, or whether I want to disable it. This probably can’t be done any simpler, there is no way to get Xorg to switch from one VGA adapter to another while it is running… But it’s not that much of a hassle anyway.

5. Peryphetials

How about keyboard/mouse, should you pass them through too? You might, but this is not necessary; I use Synergy for sharing my mouse/keyboard between systems just as if they were two displays of one system. Very convenient. The script that starts qemu for me also launches synergy server on my Linux, the client running in Windows starts automatically on boot.

If you want, you can also setup networking for the guest system – qemu has very good support for interface bridging, so it is not difficult to grant internet access for the guest OS.

One could also pass-through audio devices, but I believe this is not necessary – especially if you do not care about hardware audio acceleration; in such case you can get qemu to emulate a sound device and play it as any other app in the host OS would do. In result you can hear both systems on same speakers/headphones!

Personally, I have even went so far that I prepared a simple app that talks to my monitor via I²C and tells it when to switch video input – this way I can use a hotkey shortcut instead of navigating it’s OSD menus. The same hotkey will switch my keyboard/mouse between systems, thanks to synergy’s customizability.

 6. Conclusions

I have used this configuration for a few weeks now, and I am yet to find a game that would not perform outstandingly in this environment. Graphics performance is just as if I dual-booted, CPU performance is only a tiny bit worse (but still awesome). The ability to keep all my apps running under Linux while I play games, be it a web browser, IM client, teamspeak or whatever else might be useful – is incredibly convenient!

Switching between systems in less then a second is really a game-changer for me (pun intended…)!

If you are excited about this technique, go ahead and read the guide. Be ready for a challenge, and do not give up it things won’t work – you won’t regret it! Good luck!

Want to know more? I will be happy to answer your general questions, but if you need help or want to learn about technical details, the best place to find answers is here.

¹) Unless your motherboard has a video multiplexer, like NVIDIA Optimus… but using it would be difficult, as you would need to manually control the mux. I believe this might be achievable, but most certainly would require specialized drivers, that do not exist right now.

²) It’s not as simple as “all new hardware supports it”, both in case of CPUs and mobos. You may find some lists of IOMMU-compatible hardware on the Internet, but it is probably best to ask the manufacturer itself – if they do not list it on their website, try dropping an email – from my experience all manufacturers are very keen to respond to enquiries concerning such sophisticated features! ;-)

About these ads

10 Responses to “Multi-OS gaming w/o dual-booting: Excelent graphics performance in a VM with VGA passthrough”

  1. TheArcher Says:

    Hello sir,

    I was readin your article and I would like, if possible, that you explain your method, step by step, to run the KVM emulation using the NVIDIA GTX 770 as PCI Passthrough.

    I have the same configuration as you so I’m really interested in your method.

    Thanks a lot or your contribution and I will wait for your fully detailed explanation if possible :)



  2. m3nt4l1ty Says:


    I was wondering if you could go into more detail about how you made the program to switch monitor inputs and keyboard/mouse via I2C. This sounds perfect for what I need.


    • Rafał Cieślak Says:

      Okay, so to make things clear: I do not switch keyboard/mouse. I keep them shared between both OS, using Synergy. The host OS runs a web server, and the guest OS runs a client – they exchange mouse position information, so that at any time only one of them receives ‘clicks’. This is great for both multi- and single-monitor setups.
      Another story is that I have two video outputs connected to a single monitor, and I would like to switch between them without using the buttons on the monitor itself. For this, I use the I2C that is embedded within HDMI or DP.
      Almost all monitors support DDC, but there are scarce apps that can make us of it. There is DDCcontrol for linux, but it is completely outdated, and failed to work with my particular monitor. So I ended up getting to know the VESA DDC standard, reading the source of that tool and several similar ones, and created a simple C program (source) that works in case of my setup. Most likely it will not work elsewhere, but it should give you a nice starting point for communicating with the monitor. The fragile spots are line 41 – you may need to change the i2c bus number the the one correct for your system, and you may need to prepare other data to send to the monitor (68 and 71) if you want to switch between other outputs than I do. Refer to the standard in order to prepare them.

    • Peter Says:

      I had some questions about using synergy. Now I understand from your article that you’re using synergy server on the Host OS. Firstly, I’ve had problems before where some games (maybe limited to windows full-screen) would go to the next monitor if you scrolled to far in that direction in game. So I was wondering if you had that problem or if running it full-screen mode (not windowed full-screen) prevented that. Second, I was wondering how you configured synergy to not passthrough a specific button so that teamspeak could still be running on the host os and usable? Or did you have to go back to the host OS monitor to be able to speak on it?

      • Rafał Cieślak Says:

        To prevent mouse from leaving full-screen windows, you can lock the mouse to a particular screen. Unless you configured Synergy to use a different key for that, simply press ScrollLock to keep the mouse pointer on the current screen. Furthermore, you may want to enable “relative movements” option, so that when the screen is locked, Synergy sends mouse position changes (contrary to absolute mouse position) which will fix jumpy free-look in full-screen games. And finally, you can configure Synergy shortcut keys as you wish. I have bound the ‘lock screen’ feature to an extra button on my mouse, so that locking/unlocking screens is done with mouse only – but it’s probably a matter of taste.
        I do not use teamspeak, but I am sure I speak without returning to the host OS. When configuring Synergy, it is easy to setup rules such as “When X key is pressed, send it ONLY to this particular screen (regardless of where the mouse is etc.)”. I use this trick to control volume and media player on the host OS – in my case it is “When Ctrl+Shift+F6 is pressed, send MediaVolumeDown key to the host OS screen”. It works perfectly, so I am sure you could do the same with teamspeak.
        Also, I believe it would be super-cool to run a game on one OS, and, simultaneously, teamspeak – on another OS. Good luck!

  3. zlice Says:

    where did you get the specs / commands on DDC for hdmi?

    ddccontrol works for me to switch to analog and display…but i’m already on display and analog is empty =p

    • Rafał Cieślak Says:

      There is no specific DDC for hdmi, the protocol stays the same regardless of the video connection used. To find the specification, I simply Google searched for “vesa ddc standard”, and that presented me with a number of PDF documents (not linking here, because these links look temporary).

      • zlice Says:

        hm, didn’t use the word ‘standard’ but i already saw most of these links according to google

        another thing is even with your code my input doesn’t switch, can’t get it to work with i2c-tools (i2cset).

        i’m not sure what i’m doing wrong…wouldn’t be such a big deal if my monitor recgonized the button press instead of destroying my thumb dozens of times to switch

      • zlice Says:

        =p i got it, hdmi was 0x05, 1 was analog, 3 was digital, just took a guess

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

%d bloggers like this: