Use crosvm instead of qemu for running Linux virtual machines on Linux.
Posted by fxzxmicah@reddit | linux | View on Reddit | 60 comments
But there seems to be no crosvm in any distribution repository.
Crosvm uses virtio infrastructure entirely, and I think crosvm works well with Linux virtual machines.
But crosvm also seems to have a lot of missing features, which may take a long time to complete.
What do you think?
Different-Suit-1233@reddit
It's now available at least on ArchLinux AUR [crosvm-git](https://aur.archlinux.org/packages/crosvm-git) and Nix package [crosvm](https://search.nixos.org/packages?channel=unstable&show=crosvm&query=crosvm). However, AUR is not accessible for other distributions and Nix's gpu support is bad on non-NixOS systems. [nix-system-graphics](https://github.com/soupglasses/nix-system-graphics) may help but not tested.
LostNtranslation_@reddit
What are the advatages of crosvm over qemu?
fxzxmicah@reddit (OP)
A technical path based on the virtio infrastructure, rather than simulating real devices like qemu.
shroddy@reddit
Is virtio that what will allow us to use (probably amd only) gpus in a vm without dedicating the gpu to the vm and needing a second gpu or integrated graphics for the host os? So, from the perspective of the host os and the gpu drivers, the vm is just like any other program?
AngryElPresidente@reddit
Pedantically, virtio-gpu is a part of the overall virtio spec. Virtio covers a lot of different paravirtualized devices, such as networking, storage, input, and etc…
As of right now, Venus (Vulkan over Virtio-GPU protocol) landed in Qemu 9.2 and Kernel 6.14 released with patches needed to correct some KVM bugs.
And no, I’m pretty sure it’s not AMD specific, but it is what gets the most testing; I personally intend to test it on a Nvidia RTX3000 series laptop I have laying around within the next few days. Currently I’m running to relatively great initial success on a RX7900XT
And Vulkan is really only the start, the next big step is Virtio-GPU Native Context which essentially allows a guest to use Mesa to communicate directly with the GPU without the need to encapsulate in a OpenGL or Vulkan context. You can read more about this here from Collabora: https://www.collabora.com/news-and-blog/blog/2025/01/15/the-state-of-gfx-virtualization-using-virglrenderer/
shroddy@reddit
I think I confused it with native context which will be AMD only. I have read it will have near native performance.
How hard is it to get Venus running? Is it enough to have Mesa 25 and Kernel 6.14 and Qemu 9.2 or does it require special patches on the guest (doable) or the host (would probably skip it for now)
Do you know if Cuda works with Venus?
AngryElPresidente@reddit
Technically, I think Intel also has a merge request on the Mesa Gitlab for Native Context. I'm not sure about Nvidia, but that'll probably need to wait until Nova gets more fleshed out.
Venus is extremely simple, just need to set hostmem, venus, and blob.
Cuda will not since Venus only handles Vulkan.
shroddy@reddit
I also have laptop with a 3000 series GPU so please tell me if your tests are successful and which additional steps you had to do to get Venus running.
Now we only need a way to use Cuda in a VM without giving the whole GPU to the VM and hoping the moon phases align and the Nvidia driver likes is, but probably can't have everything we want.
AngryElPresidente@reddit
As an unfortunate follow-up, I could not get Venus to run on my RTX3000 series laptop.
The environment was Fedora 42 KDE Beta, with testing done on a Fedora 41 Gnome and Fedora 42 Gnome guest install. On the host, the Nvidia proprietary driver was installed using rpmfusion with version of 570.113.
When running
vkcubeQemu freezes up and require switching TTYs to perform a kill command. Subsequent runs after restarting Qemu would indicate fencing problems in the terminal output forvkcube.My environment should reach the minimum requirements as listed on the Mesa Virtio-GPU page, so maybe there is some configuration error or other kernel isues listed.
Still, if you get the time I'd like to hear if you're successful.
fxzxmicah@reddit (OP)
Your understanding is right.
rx80@reddit
Ummm... Qemu provides both, real devices and virtio, and consequently lets you run more VMs than just those that support virtio. For VMs with virtio support, qemu is perfectly capable of doing it.
fxzxmicah@reddit (OP)
I didn't ask crosvm to replace qemu, and qemu will always exist. But isn't it good to have another small, specialized solution?
rx80@reddit
It depends who you ask.
For someone who is already using qemu, and runs a variety of VMs, then the answer is very likely "no". Because you'd have to either switch, or run 2 different, incompatible VMMs.
For new pepople who want to run VMs, the answer is also probably "no", you want to use something that is well documented, and has a big enough user base that you can be almost sure that all the wrinkles of the common use cases have already be ironed out.
For people who know what Google usually does, the answer is a very definite "no".
fxzxmicah@reddit (OP)
Yeah, you're right.
So this is more like the first step in a story that goes, “someone shows that this can be done, then someone develops a new piece of software for Linux similar to crosvm, keeps refining it, and more and more people start using it” — and it all begins with showing that it’s possible.
rx80@reddit
Sure. But first it needs to have at least high enough usage in intesting environments that some bigger/mainstream distro is prepared to maintain a package.
It happens all the time. A package needs to provide a compelling reason to use it. If it's just "qemu, but different" there's no real reason to switch or even try, apart from people who just wanna experiment.
fxzxmicah@reddit (OP)
I totally agree with what you said.
I think you can see now why I said the other person's comment was not very useful.
atuncer@reddit
This could also be of interest to you: https://firecracker-microvm.github.io/
fxzxmicah@reddit (OP)
Thanks.
LostNtranslation_@reddit
Thank You!
79215185-1feb-44c6@reddit
Use vs .
No.
fxzxmicah@reddit (OP)
Never heard of something that's not normal? Everything that is popular now used to be unknown.
79215185-1feb-44c6@reddit
It's a google project. Google projects are known to be made for exactly Google. See Bazel. Never use a project that uses Bazel.
blackdew@reddit
distroless base images for docker containers are made by google using bazel and are pretty amazing.
It's more of an exception to the rule tho
fxzxmicah@reddit (OP)
Perhaps a new project for linux could be spun out of this?
After all, you have never heard of it, which means that fewer people know about crosvm, and fewer people can get inspiration from crosvm.
Kevin_Kofler@reddit
Why would you want to do that?
That would already be one reason to not do that.
QEMU works at least as well, and it fully supports the KVM and virtio infrastructure. QEMU is just not completely dependent on it, i.e., it also supports software CPU emulation (so you can emulate a completely different CPU architecture if you wish) and emulation of physical hardware peripherals (which for some use cases works better than virtio). So it is more flexible than crosvm. I see the fact that the same infrastructure (QEMU) works for different use cases (hardware virtualization, software emulation, etc.) as an advantage, not a drawback.
And that would be why you should use use QEMU, which already has those features. And some might never be added to crosvm because they go against its design, e.g., you cannot have software CPU emulation in an infrastructure designed to work around KVM and only KVM, which inherently restricts it to hardware virtualization and hence to same-architecture virtualization.
I see a lot of reasons why one would use QEMU over crosvm and none why one would use crosvm over QEMU. Crosvm is designed by Google for Google's specific ChromeOS and/or Android use cases. QEMU is designed to be the one emulation and virtualization solution that works for all users.
fxzxmicah@reddit (OP)
In response to your other statements, I have only one comment: Crosvm brings a new path, which is small and specialized, and qemu is big and versatile.
I didn't say who wanted to replace who, and I didn't say qemu was not good.
Like systemd, which is already taking over linux, there are still plenty of alternatives to some of its subfeatures.
BallingAndDrinking@reddit
If this is the thing you had in mind :
Then your title is even worst than if you were just a dick.
If you don't find it in repo, you'll have to keep it running yourself. A single machine can be doable. Several may start to be really ass.
I did have a self hosted repo when I was fiddling around the qemu's microvms to emerge from to my gentoo box. So most of the work was basically figured out, but it's not a solution at scale. I'd believe it can get worst because crosvm as it is in rust.
So you'll need to keep the whole rust tooling in place. I know rust has good tools around it, but more tools mean more breaking points.
And looking up the project, while it seems to have some interesting stuff here and there, I also wonder why it should be so much more in to use it.
AFAIK, it doesn't go the extra mile for any of it's key points. So it's good to have variety, but it doesn't bring in what others truly lack.
Security ? I'd rather use a Xen system for that. Be it Qube if I want to have a single user computer, or XCP-NG for a server.
Safety ? What do you even mean, a VM crashing bringing down your virtualization isn't a thing, it's not even a toy then, it has no place to be in this category.
People pointing out it's a google product is also important to this : beyond "we are doing it for ourselves only and some fools will provide patches for free", there own use case isn't 99.9% of people's use case.
And this also has to be pointed out : Olivier Lambert's from the XCP-NG project put a blog post out recently out the issue with monoculture in ITn such as the whole vmware issue. It applies here too, which should mean having more virtualization option is good, isn't it ?
Well yes. But it's a google product, and if I'm wary of IBM, or Apple, or Microsoft, it'd be foolish of me to not be wary of Google, isn't it ? They have a terrible track record on supporting stuff or actually making stuff useful. I'd rather use something else. KVM, qemu and Xen have more people behind them.
final thought Firecracker may not be the thing that fit your need either (saw it advertised below), same-ish issues plus some extra funny bits. Sure it boot fast, but so does unikernels, containers and qemu microvms.
KlePu@reddit
As to your first edit: Why didn't you chose a better title and added the "discussion" flair? The way to write it I got the feeling that you wanted folks to use
crosvminstead ofqemu.Robsteady@reddit
"crosvm is currently used to run Linux/Android guests on ChromeOS devices."
I'll stick with qemu on my servers and virt-manager on my other machines for management, but thanks.
fxzxmicah@reddit (OP)
It actually works on linux now, with some missing features compared to qemu.
Kevin_Kofler@reddit
And those missing features are why you should just use QEMU that has those features.
fxzxmicah@reddit (OP)
I just thought of an example — not a perfect one, but still worth mentioning. OpenRC has service management features, like systemd does, but it doesn’t have something like systemd’s networkd. So does that mean OpenRC should never be used?
fxzxmicah@reddit (OP)
No, it's not the core functionality that's missing.
In addition, what Crosvm brings is a new development idea and implementation, not just a software.
BinkReddit@reddit
Use something that will be well supported in your operating system environment.
fxzxmicah@reddit (OP)
I'm not asking for help, so what you said is actually not very useful.
rx80@reddit
You ask for people's thoughts and then tell them not to share them. If this post isn't just a horrible attempt at self-promotion, i don't know what you're trying to achieve here.
fxzxmicah@reddit (OP)
I don't know what you misunderstood.
Were you the one who interpreted "not particularly useful" as "completely useless, shut up"?
All I said was that his thoughts aren't very useful in the context of the discussion — kind of like saying “the sun rises in the east.” It’s correct, but not very helpful.
rx80@reddit
Your question was "What do you think?". That question is still in your top post.
He answered: "[I think i will] use something that will be well supported in your operating system environment."
It's a perfectly legit and normal response to your question.
fxzxmicah@reddit (OP)
It does have a different meaning after you complete it.
In my understanding: "[You should] use something that will be well supported in your operating system environment."
But in my opinion, it's still not very useful.
BinkReddit@reddit
I gave you my thoughts.
fxzxmicah@reddit (OP)
Yeah, it's just that your thoughts aren't very useful.
devslashnope@reddit
Much more useful than numerous.
fxzxmicah@reddit (OP)
How can there be an "answer" in a discussion? I'm not even asking a question.
Also, did I say what he said was completely useless? I just said it wasn't very useful — that’s not the same as useless.
And finally, if all you're going to do is make useless complaints, then you can stop, this isn't the place for you to complain.
devslashnope@reddit
Oh. Ok.
devslashnope@reddit
Oh. Ok.
DoubleDotStudios@reddit
It’s not an official package but it’s on the AUR. Not sure how well it works though.
https://aur.archlinux.org/packages/crosvm-git
fxzxmicah@reddit (OP)
Thanks for the information.
srivasta@reddit
If you think this software needs to be packaged for your distribution you could just package it and contribute to your favorite distribution.
Things get fine faster when you help. Is that not the free software way?
shroddy@reddit
How does one package it? I know how it could be done for flathub, but how would I get a software into the Debian repos, as an example?
srivasta@reddit
A good place to start is with the wiki portal to packaging. Once you are happy with your package, and you test it locally, you can send in a request for packaging and join the mentoring program. Then a mentor can sponsor your package into Debian.
https://wiki.debian.org/Packaging
fxzxmicah@reddit (OP)
I've packaged it up, but I didn't contribute to the distribution upstream because there's a lot of unfinished business.
On the one hand, I want to tell everyone in the forum about the existence of crosvm through this post, and on the other hand, I want to hear the opinions of people who have used crosvm or just know it.
srivasta@reddit
It would help if you added a codeberg or GitHub link to the work in progress packaging. People might try it out and possibly contribute if they like it.
Also, getting it into the distribution pipeline can bring in suggestions or mentors (I've gotten lots of help from the mentors in the Debian new maintainers process in the past).
fxzxmicah@reddit (OP)
Packaging isn’t the main problem—the real problem is that crosvm is currently just usable, but not user-friendly. It feels like a great idea that hasn’t quite been fully realized yet.
srivasta@reddit
I have no experience with it. But the barrier to entry to trying it out would be lower if there was a repo where I could clone, inspect, and just say ./debian/rules binary and play with it.
You might like more eyes on the work, and perhaps people motivated to add missing features, if there was a way to poke at it.
The packaging might also help start a dialogue with the original set of people working on crosvm. Free software needs a community of workers. Seems like you have done the work partially to be part of the crosvm community.
fxzxmicah@reddit (OP)
Yes, you have a point.
autogyrophilia@reddit
Seems like an inferior version of the popular Linux VMM QEMU and the other, smaller in scope alternatives that are popping in production such as cloud-hypervisor or the more niche firecracker.
Go for it if it scratches an itch, but it's meant to be used in Android and ChromeOS and built around that.
IAmHappyAndAwesome@reddit
I think there is a distro called spectrumOS (based on nix) that wants to do that
fxzxmicah@reddit (OP)
Really? This is the news I don't know yet.
Mister_Magister@reddit
use libvirt
fxzxmicah@reddit (OP)
If I remember correctly, libvirt is just a wrapper around backend APIs for various virtualization software, not a real virtualization software?
It's not at all on the same level as the crosvm or qemu I'm talking about.