For me fighting with gatekeepers over email is not worth it when there so many other FOSS projects that have modern contributing processes. But I'm glad they are trying to attract more Rust developers.
There are many reasons Linux still uses the patch workflow, here's a talk from Greg that gets into
why.
TL;DW:
A significant amount of kernel devs are behind corporate firewalls or have limited access to the internet or even to git ⇒ Email is accessible to pretty much anybody
This also includes visually impaired devs, who have a hard time using web-based tools but can work using emails just fine.
Web-based tools are slow if you are reviewing massive amounts of patches and also don't work offline. ⇒ You can fetch email and process it on the plane and queue up your replies to be sent out later
He goes over several tools including github, gerrit, ... and lists the pros/cons of them. For instance, the github web ui makes responding to individual commits a pain
email is infinitely scriptable
Personally, I think email seems to be the right tool for the workflow the kernel uses. Yes, there is a barrier to entry but I think that is not that big of a deal if makes kernel development (including the review process) better in the long run.
Several kernel developers cannot use git, since it is blocked by their company's firewall. This is not even an argument, this is a fact stated in the presentation.It is also not uncommon for sites to be blocked by corporate firewalls, for whatever reason.
I don't know, my company does not block git, github, or any other git forge. But if one of the lead kernel developer states that this is an issue for several reviewers/contributors, I tend to believe him.
I don‘t. These people are stuck in their ways and have shown disdain for change of any kind many times in their past. Linus obviously isn‘t, but he has many insanely stubborn people working below him.
It's the right workflow for the kernel, but not really because of email's intrinsic superiority, but rather because they've spent over 30 years building up that workflow. Migrating to an entirely new platform would be a massive undertaking.
Had the kernel started on some more modern platform (or more likely a set of platforms), their development practices and conventions would have grown to match that platform, and they would have just as long of a list of reasons why it's superior to email.
This also overstates how well it "works" for the kernel, for example recent CVE CVE-2026-46333, the ptrace vulnerability, was explicitly known about in 2020,very soon after the introduction of pidfd_getfd which made it exploitable.
The patch died to useless bikeshedding and then got forgotten entirely. The kernel has no systems beyond "author spams the mailing list with pings every now and then" to actually track patches or make sure they go anywhere.
How many other nicely described security fixes are sitting forgotten, unfixed and unmerged, just waiting for some LLM to find? I can't remember the other one off hand right now but i believe this exact scenario has happened twice recently.
Sure, but thats project management, and its possible to see them at a glance, you can just look at open PRs.
With email the only way to know is to read every email thread in its entirety to find out if it got merged, moved to another patch series(and check manually if that got merged!), etc.
You know there are a lot of github projects with forgotten PRs because you can see it. There is no way to see that on the mailing list, and thats a problem.
Get even a little better project management and use labels and you suddenly have a lot more information than just "open", "closed", and "merged" for PRs too, PRs can be categorized, trivially and discoverably marked for stable rebasing in a way that maintainers can exhaustively see, marked as postponeed until the next cycle, etc.
The pure mailing list approach lacks all of this.
All of this is, of course, why in reality a lot of complex kernel development doesnt happen on the list anymore, but on freedesktop's gitlab, and has for awhile.
Just check out the drm gitlab project, you can see its very active https://gitlab.freedesktop.org/drm?sort=updated_desc including the kernel repo they use as the "Root DRM kernel tree, use this to create gitlab forks. Otherwise for subsystem maintainers only, issues/pr go to subtree repos."
This fact is very conveniently missing from many of the redditor discussions around the mailing lists efficacy, because none of them have ever cared enough to even look into how things work in practice, let alone contribute. They just spout their memes about how perfect it is and how the kernel could never use anything else, ignorant of the fact they do.
Sure, but thats project management, and its possible to see them at a glance, you can just look at open PRs and sort by creation or updated date.
How do you think that would work with hundreds of maintainers? The kernel did have a Bugzilla instance but it was more or less unused.
The scale of the kernel's operation makes any attempt to do more than sheer archival a losing enterprise.
Otherwise for subsystem maintainers only, issues/pr go to subtree repos
Note development is still mailing list based:
* https://gitlab.freedesktop.org/drm/i915/kernel/-/merge_requests empty
* https://gitlab.freedesktop.org/drm/xe/kernel/-/merge_requests unused for three years
* https://gitlab.freedesktop.org/drm/amd/-/merge_requests 12 MRs merged
* https://gitlab.freedesktop.org/drm/msm/-/merge_requests/?state=merged only used by maintainer, presumably to trigger CI
* https://gitlab.freedesktop.org/drm/nouveau/-/merge_requests/ unused for years
* https://gitlab.freedesktop.org/drm/nova/-/merge_requests the newest driver, but it doesn't use MRs either
How do you think that would work with hundreds of maintainers?
I dont understand what you think the difficulty is
The kernel did have a Bugzilla instance but it was more or less unused.
Still does. https://bugzilla.kernel.org/
Bugzilla also simply sucks. In fact AIUI thats a big reason for the move to gitlab.
The scale of the kernel's operation makes any attempt to do more than sheer archival a losing enterprise.
I dont understand how you think mailing lists dont have the same problem then.
But as to PRs (or MRs as gitlab calls them), looking at them shows that development is still mailing list based:
Thats the beauty of using git hosts instead "The pure mailing list approach" i said. They, even github, still support text/email based operations. You can adopt them piecemeal, and only the parts that work, instead of all or nothing.
The ability to reliably track issues with all the modern amenities is valuable. Many issues also serve the function of helping ensure patches dont get forgotten, because you can browse open issues, post patches to users to test, and users will ping if its been forgotten, in ways they won't or can't on the mailing list.
They are using gitlab and it is for a reason. I decried the pure mailing list approach. It is also my understanding that many groups in the kernel are investigating expanding their use of git forge features, but its slow and incremental work, and some tooling needs to be made to bridge between the traditional mailing list that patches ultimately end up in.
I feel like the email workflow is much worse for fast-moving projects, but to be fair I have precisely zero experience using it for fast-moving projects.
One of the points Greg makes in the talk is that the forges don't handle fast-moving projects well while email does. The kernel moves incredibly fast, getting 7-8 change requests per hour every hour.
That's a fair point, but the kernel is also extremely large. Most individual components of it aren't particularly fast moving, and the email paradigm specifically works on the component level.
That is part of the point though. In his talk, Greg mentions projects like Kubernetes / Docker splitting up their projects into subprojects to keep the amount of PRs manageable. mailing lists with a shared repo make this pattern quite natural; it also makes it very easy to only get notified of the parts of the project you're interested in.
Yes this is possible on GH using tags / subprojects, but it requires more effort.
I don't think they are going to be a amenable to adopt the practices of other projects that are far less capable and successful then their own. Even if it is 'modern' in any case.
their development practices and conventions would have grown to match that platform, and they would have just as long of a list of reasons why it's superior to email.
This is literally the case for pretty much all software. What makes things work isn't the adoption of a new different process or adapting to 'modernisms'. It is a long slow grind of making it actually work.
It takes competency and hard work; adopting your workflow to the "correct methodology" or platform or abstraction isn't probably in the top 10 most important things to get right.
It takes competency and hard work; adopting your workflow to the "correct methodology" or platform or abstraction isn't probably in the top 10 most important things to get right.
What? This is ridiculous. of course having correct abstractions and methodology is in the top 10 most important things, thats the foundation of any and everything else. What do you mean you dont think the foundation is important?
with enough effort they could use smoke signals, massive networks worldwide of em, but that wouldnt make it a good idea or mean that migrating to a sane methodology or abstraction wouldnt be objectively better in every way.
And no, light signals, despite being faster, would not be better, the fundamental abstraction has to change.
This also includes visually impaired devs, who have a hard time using web-based tools but can work using emails just fine.
You don't have to use web based tools directly. Most of the forges do support alternatives. I personally couldn't deal with only interacting with say github via the web, so I use the cli most of the time.
Maybe I'm too spoiled by enterprise but every tracker worth its salt can be worked through email. That includes sending new stuff and forwarding stuff to people who want to read it all in email
I think the GitHub model is worse than the email model. I use both regularly. GitHub has no concept of versioned PRs, so reviews stack up in a linear fashion. It doesn't work well for contributions that go through multiple interations. It also leads people to do stupid commits like "cargo fmt" instead of using fixup commits and squashing them. GitHub truly deemphasizes the commit. It puts complete emphasis on the PR in totality.
Not defending GH by any means, but I don't see how GH is preventing people from doing fixups or squashing commits. I'm part of a few FOSS projects that use GH and we enforce sane PRs, commit structure, and commit messages.
What re you talking about, its git, so yes it absolutely does? When you force push github shows both commits and you can view before and after just fine. AIUI they'll last at minimum 90 days, the default git reflog expiration time, if they're unreferenced.
And on github they're only unreferenced if no forks exist, so in practice for popular/large projects they will usually be referenced somewhere indefinitely, as all forks share the same object pool; This is why you can use a commit ID on any fork URL and view it fine, but not non-forks.
They are just as capable of preserving pre-rebase commits as they as preserving emails. There is literally no question or challenge here and your insistence this would somehow be a problem just highlights how little you understand pretty much everything involved.
I do understand everything involved. I've been upstreaming patches for decades.
Git isn't the issue here.
There's a lot needed to copy something the lkml implements.
The main being fully decentralised history and also decentralised patchset submission. If you look at how the lkml works, you'll see people submitting v1 of a patchset and then people replying in-line to everything.
They then submit another patch set and label it v2. It'll have the same number of commits but the commit messages have different text and the actual diff is different.
Now can you do that with GitHub?
Not really.
It's assumed you're not opening a new PR for every new patchset. The UI doesn't even have any support for diffing PRs.
Everything is on GitHub to host and somehow make it decentralised through regular backups or some other system which they haven't done.
Not only that but the kernel maintainers at the moment can easily just sit in a terminal and apply/reject/comment on patches. They don't even need any internet at all.
The lkml is actually a super nice workflow and really GitHub and gitlab are a huge downgrade.
You are describing git itself. that is just git. this is literally just git. a canonical repo being on github/gitlab/codeberg vs git.kernel.org are all just as decentralized as the others. what are you talking about.
The UI doesn't even have any support for diffing PRs
PRs are just git. PRs are just branches. You can diff branches just fine. You can even get patch files just fine. Please learn what git is.
The reflogs do still exist and can be fetched after a force push with git. I'm not sure for how long GitHub keeps stale refs available, but it's why you can't force-push your way out of a security key leak.
GitLab is better for this; at least it archives the previous version and makes it easily reachable. It still doesn't have git range-diff though, so if you rebase your PR the diff is unreadable.
Not sure what specifically you are asking here. If a fixup commit is pushed you can see what's in the fixup commit (obviously), and if it's squashed you can see what was changed in the last push.
GitHub is absolutely terrible at handling history rewrites.
If you replace some commits, and then click the button to show what changed, it'll often say "can't find those commits".
It also doesn't use git range-diff, so, when you rebase your branch, and it is able to show changes, it will include all the unrelated changes made to the base branch, and not just what changed in your commits.
That’s the issue. When working on a sizeable feature, this should be split into several commits but treated as one peer review. That’s what patchest process is about.
PRs can have multiple commits. Pull requests are based on the branch that they're created from. Those with write permission to that branch are able to add additional commits at any point. Including rebasing and rewriting the history of that branch. All discussions will remain relevant to that pull request as it evolves.
The maintainer gets to decide how to merge branches in the end. Either manually with git on a terminal, or using one of the available merge options in the GUI. Manual merges and are detected automatically.
Those who want multiple versions of a patchset for historical reasons can create a separate branch with a separate PR that references the previous one to link the discussion and close it.
Yes, but GitHub funnels everyone into making PRs where individual commits don’t make logical sense. What u/tristan957 mentioned with ‘cargo fmt’ commits is an example.
Contributors sends a PR, gets review comments, and then pushes new commits addressing the comments. As a result, you end up with a branch with some incorrect commits followed by fixes to those commits. As a maintainer you either have to spend days explaining proper git workflow, squashing everything into a single functional commit or just doing this yourself.
This is the opposite of mailing lists which force people into doing proper patchsets. And since git provides tools for sending patchsests, maintainer doesn’t need to explain how to fight GitHub to get the proper PR.
Yes, but GitHub funnels everyone into making PRs where individual commits don’t make logical sense.
A PR is just a discussion thread for peer reviewing a git branch that's being proposed for merging from one branch to another. The individual commits in the git branch associated with a PR are no different from a series of patches copied and pasted over email to a mailing list. Those git patches are generated from commits in a git branch all the same. You're simply reviewing those changes through a web UI instead of an email thread.
Contributors sends a PR, gets review comments, and then pushes new commits addressing the comments. As a result, you end up with a branch with some incorrect commits followed by fixes to those commits.
If you have strict policies and guidelines, you'll have to explain that to new contributors regardless of what tool they use. A pull request that contains a chore: cargo format commit is no different from a series of patches by email that also contain the same commits in patch format.
If you want a PR to contain multiple commits that are individually formatted and verified, you can ask the contributor to rebase and format their commits. You could also set up git hooks that enforce this, as well as editor configs to configure code editors to auto format on save. There are many different ways to automate this for new contributors.
Alternatively, since it's mostly a waste of time for a maintainer to nitpick and demand that every commit in every PR to be perfect, you could ask them to split their changes into separate PRs so that they can be reviewed separately. If you like the structure of their commits, choose the rebase merge option. If you want to squash it into one commit and write the title and description yourself, use the squash merge option.
A PR is just a discussion thread for peer reviewing a git branch that's being proposed for merging from one branch to another. The individual commits in the git branch associated with a PR are no different from a series of patches copied and pasted over email to a mailing list.
I understand all that, but the problem is that GitHub workflow is geared towards treating PR as a single atomic change. This is fine if the change is very simple. It’s terrible if it’s a sizeable feature.
If you have strict policies and guidelines, you'll have to explain that to new contributors regardless of what tool they use.
I’m not talking about explaining policies and guidelines. I’m talking about explaining the proper git workflow and how you have to fight GitHub to use it.
A pull request that contains a chore: cargo format commit is no different from a series of patches by email that also contain the same commits in patch format.
It is different. If my patchset consists of a commit which introduces formatting violation and is followed by a commit which fixes that violation, this is a deficit patchset and should be rejected.
patchsets make you review a series of commits, and not just the diff of HEAD. People submitting patchsets don't want to submit it all as one big commit instead, nor do they want to just review the entire change series at once. They want to review each commit individually, and land the entire feature at once. If any commit is bad, they fix that one commit, and send another patch series down the line.
That doesn't help if you review individual commits at all, though. If I made a commit, and amended that commit, how do you view the prior version of that commit to see what's changed? Not the entire branch. Just the previous version of that commit. The closest thing would probably be to make a new PR, and have it reference the old PR whenever you want to send a new version for review.
Reviewers would then review each commit one by one, and either Ack or Nack them. When they all get ack'd, you can merge the branch in however you deem fit.
My "take" is referring to the current process mixed with the recent-ish Rust in kernel drama and contributors leaving.
But you're right, of course - you're the only one on the internet who knows the kernel development process. And I'm also glad I don't contribute with my noobish kernel development skills.
Calls for using a forum instead often end in something awful like discourse or some worse re-imagined version of that though. You'd think a simple phpbb one would be right up there given the relative simplicity and rigid code.
I wonder if it has to do with a different mindset when working in lower levels of machines that anything more than just text causes fatigue. Just a theory tho.
I haven't actually run it myself but enough projects I follow or participate in have unfortunately switched to it that I feel confident saying that it's not possible to make it behave like a standard hierarchical forum the way anyone who spent years on phpbb, vbulletin, etc. would expect a forum to work.
I have yet to figure out why so many projects have chosen Discourse in recent years, but the vast majority of the places I've seen it have been open-core projects like FreePBX where there's a corporate backer so I have to assume there are some features that appeal to suits.
I wasn't trolling and wanted to have something concrete. Do you have specifics about things you're not happy with? I've been using "classical" forums and newsgroups since the 90s, and I'm quite happy with Discourse across multiple non-corporate backed FOSS projects I'm part of.
I'm not trying to convert anybody and I'm not affiliated with Discourse.
Have you ever used an old style forum? That were so much easier to use.
Discourse is modern, bit a discussion forum doesn't need modern or fancy effects or whatever. It needs to organize discussions well and have robust search wkth easy to parse results.
phpbb really is peak in that regard. It does what it is supposed to do and if you need anything you can just change it to fit without having to fuzz with javascript in frontend and backend.
Most importantly though it works on widescreens, which anything designed with mobile usually doesn't do well. Why waste half the screen on blank space just for that feed-like design... bad enough reddit does that. Same problem. old.reddit so much friendlier, much less wasted space.
The funny thing is they are, the mailing list is a relic that a lot of major development doesnt happen in. Subsystem maintainers have full control over how their subsystems are managed, and a lot of big ones have moved to gitlab for PRs, reviews, etc, and only send the finished product up to the mailing list for inclusion by Linus.
For example much of drm works this way https://gitlab.freedesktop.org/drm/kernel
Considering the latest news about Gitlab, and just the general community squashing retcon bug/issue cycling around their development...I would say to not use gitlab if possible.
That is part of it. They are basically doing the "lay off a bunch of people. Replace them with AI." Dance. This is on top of their already pretty toxic attitude towards the gitlab community. Basically, unless you are paying them a ton of money, don't expect them to fix any bugs or performance issues. It is all about shiny features and AI slop.
It is hard because while I understand they want more developers the areas that they want to push Rust is mainly in the area of driver engineering but that can be quite limited in scope since you can only really do work and test if you have available hardware, so either you are doing it because you already have the hardware and it isn't working or you work at that company. so it is kind of a chicken and egg problem. The bit that Rust helps with though is that it lowers the barrier when someone does want to do that work.
Yeah thats the problem. Its trivial to write clean looking, compiling, warning free code... thats extremely wrong and broken. It is very simple to write use after frees, double frees, memory leaks, out of bound accesses, over/under-flows, etc.
goto is relatively easy to avoid. What's worse is pointers. They are pretty much core features used everywhere, and can easily be null when you `(*ptr)++`. Unlike C++, C has no references, which is much better to say how much null is not expected.
(Rusts pointers are also shit, sometimes worse than C, but in code they are not used that much, which surpisingly makes them shittier -- they can work in debug but break in release as these builds handle resources very differently, so during development you will not even notice the problems)
Yeah C is pretty clean but note that it is the kernel so any memory issue or performance issue is going to be a big issue. Rust isn't just type safety, it also has one of the best toolsets of any language like with linting and formatting baked into everything, great testing workflows, great error messages from the compiler. It is super easy to use Rust, I find Rust easier to use than Java or JS.
the actual bottleneck has never been people who know rust — its the mailing list workflow being genuinely alienating for new contributors. you have to already know the culture to navigate a patch review without getting demoralized by someone who responds in three words and moves on. github prs arent happening but at least a modern patch tracker would reduce the friction significantly. the rust devs who show up and bounce usually arent bouncing because of the technical challenge
So do you want JS in the kernel or something? I think the reason "most" don't work on the kernel is because they have little reason to do so. It sits at a fairly lower layer than what most of us care about. It's the same reason I'm not typically reading and commenting on RFCs for network protocols. It's beyond me and I don't really care.
Unfortunately nowadays this is also a dinner bell for slopcoders trying to make a name for themselves off the back of something that doesn't get context very well.. euhg I hate the 2020s
Why would someone want to use a language like Rust that looks very much like a toy language when they can use a professional looking language like C or C++ which go back to the very roots of UNIX.
Because, unlike you, the kernel developers actually know what theyre talking about and choose Rust for good reasons that they have explained very well over the years many times, and which you dont care at all about because you're simply a troll pretending as if it isnt a question the kernel developers discussed and answered ages ago.
UNIX first appeared in 1969. The C language first appeared in 1972. The C++ language first appeared in 1985.
Quote from the Wikipedia page on UNIX.
In 1973, Version 4 Unix was rewritten in the higher-level language C, contrary to the general notion at the time that an operating system's complexity and sophistication required it to be written in assembly language.The C language appeared as part of Version 2.
Because C abd C++ are dirtier and unsafe, the roots made a good job but it's time to move beyond them, tying Linux to old UNIX will simply limit it, additionally Linus himself said that he would personally hunt whoever suggests using C++ in Kernel space while he approves Rust because of how it tackles memory
Also the distinction of "toy" and "profisional" language is dinnering, childish and uninformed, C was considered a toy language that never would replace ADA, LISP por Haskell but now the C family reigns, Python has replaced Perl aa the favorite glue language and yet it uses tabs for nesting while also giving MATLAB a run for its money, the Java Virtual Machine itself had managed to be considered by itself an alternative to traditional native apps in some archutectures and the list goes and goes, Languages evolve, programmers should too
To all these people talking about forums, github/gitlab, etc.:
I'd like to remind that the kernel development is quite decentralized, which some might not remember/understand here. Independent of the specific software that is used, this requirement won't go away anytime soon.
PerkyPangolin@reddit
For me fighting with gatekeepers over email is not worth it when there so many other FOSS projects that have modern contributing processes. But I'm glad they are trying to attract more Rust developers.
Sentreen@reddit
There are many reasons Linux still uses the patch workflow, here's a talk from Greg that gets into why.
TL;DW:
Personally, I think email seems to be the right tool for the workflow the kernel uses. Yes, there is a barrier to entry but I think that is not that big of a deal if makes kernel development (including the review process) better in the long run.
Schneestecher@reddit
What kind of trash argument is that? Who can‘t access a website?
Sentreen@reddit
Several kernel developers cannot use git, since it is blocked by their company's firewall. This is not even an argument, this is a fact stated in the presentation.It is also not uncommon for sites to be blocked by corporate firewalls, for whatever reason.
Schneestecher@reddit
Why should it be? You know git works over http just fine right?
Sentreen@reddit
I don't know, my company does not block git, github, or any other git forge. But if one of the lead kernel developer states that this is an issue for several reviewers/contributors, I tend to believe him.
Schneestecher@reddit
I don‘t. These people are stuck in their ways and have shown disdain for change of any kind many times in their past. Linus obviously isn‘t, but he has many insanely stubborn people working below him.
Sentreen@reddit
Sure, I agree with that. I do think there is a good reason why they are difficult, but I guess that is where we disagree.
Regardless, it is one thing to say they're being difficult about changing tools and another to imply they're making false claims to do so.
turdas@reddit
It's the right workflow for the kernel, but not really because of email's intrinsic superiority, but rather because they've spent over 30 years building up that workflow. Migrating to an entirely new platform would be a massive undertaking.
Had the kernel started on some more modern platform (or more likely a set of platforms), their development practices and conventions would have grown to match that platform, and they would have just as long of a list of reasons why it's superior to email.
CrazyKilla15@reddit
This also overstates how well it "works" for the kernel, for example recent CVE CVE-2026-46333, the ptrace vulnerability, was explicitly known about in 2020,very soon after the introduction of pidfd_getfd which made it exploitable.
The patch died to useless bikeshedding and then got forgotten entirely. The kernel has no systems beyond "author spams the mailing list with pings every now and then" to actually track patches or make sure they go anywhere.
How many other nicely described security fixes are sitting forgotten, unfixed and unmerged, just waiting for some LLM to find? I can't remember the other one off hand right now but i believe this exact scenario has happened twice recently.
matjoeman@reddit
I mean, there are lots of Github projects where PRs end up forgotten and sit around for years.
CrazyKilla15@reddit
Sure, but thats project management, and its possible to see them at a glance, you can just look at open PRs.
With email the only way to know is to read every email thread in its entirety to find out if it got merged, moved to another patch series(and check manually if that got merged!), etc.
You know there are a lot of github projects with forgotten PRs because you can see it. There is no way to see that on the mailing list, and thats a problem.
Get even a little better project management and use labels and you suddenly have a lot more information than just "open", "closed", and "merged" for PRs too, PRs can be categorized, trivially and discoverably marked for stable rebasing in a way that maintainers can exhaustively see, marked as postponeed until the next cycle, etc.
The pure mailing list approach lacks all of this.
All of this is, of course, why in reality a lot of complex kernel development doesnt happen on the list anymore, but on freedesktop's gitlab, and has for awhile.
Just check out the drm gitlab project, you can see its very active https://gitlab.freedesktop.org/drm?sort=updated_desc including the kernel repo they use as the "Root DRM kernel tree, use this to create gitlab forks. Otherwise for subsystem maintainers only, issues/pr go to subtree repos."
This fact is very conveniently missing from many of the redditor discussions around the mailing lists efficacy, because none of them have ever cared enough to even look into how things work in practice, let alone contribute. They just spout their memes about how perfect it is and how the kernel could never use anything else, ignorant of the fact they do.
bonzinip@reddit
How do you think that would work with hundreds of maintainers? The kernel did have a Bugzilla instance but it was more or less unused.
The scale of the kernel's operation makes any attempt to do more than sheer archival a losing enterprise.
Note development is still mailing list based: * https://gitlab.freedesktop.org/drm/i915/kernel/-/merge_requests empty * https://gitlab.freedesktop.org/drm/xe/kernel/-/merge_requests unused for three years * https://gitlab.freedesktop.org/drm/amd/-/merge_requests 12 MRs merged * https://gitlab.freedesktop.org/drm/msm/-/merge_requests/?state=merged only used by maintainer, presumably to trigger CI * https://gitlab.freedesktop.org/drm/nouveau/-/merge_requests/ unused for years * https://gitlab.freedesktop.org/drm/nova/-/merge_requests the newest driver, but it doesn't use MRs either
CrazyKilla15@reddit
I dont understand what you think the difficulty is
Still does. https://bugzilla.kernel.org/
Bugzilla also simply sucks. In fact AIUI thats a big reason for the move to gitlab.
I dont understand how you think mailing lists dont have the same problem then.
Thats the beauty of using git hosts instead "The pure mailing list approach" i said. They, even github, still support text/email based operations. You can adopt them piecemeal, and only the parts that work, instead of all or nothing.
The ability to reliably track issues with all the modern amenities is valuable. Many issues also serve the function of helping ensure patches dont get forgotten, because you can browse open issues, post patches to users to test, and users will ping if its been forgotten, in ways they won't or can't on the mailing list.
They are using gitlab and it is for a reason. I decried the pure mailing list approach. It is also my understanding that many groups in the kernel are investigating expanding their use of git forge features, but its slow and incremental work, and some tooling needs to be made to bridge between the traditional mailing list that patches ultimately end up in.
tav_stuff@reddit
As someone who uses both all of the time, I very much prefer the email workflow. Not only that, but all the others I work with feel the same way too
turdas@reddit
I feel like the email workflow is much worse for fast-moving projects, but to be fair I have precisely zero experience using it for fast-moving projects.
Sentreen@reddit
One of the points Greg makes in the talk is that the forges don't handle fast-moving projects well while email does. The kernel moves incredibly fast, getting 7-8 change requests per hour every hour.
turdas@reddit
That's a fair point, but the kernel is also extremely large. Most individual components of it aren't particularly fast moving, and the email paradigm specifically works on the component level.
Sentreen@reddit
That is part of the point though. In his talk, Greg mentions projects like Kubernetes / Docker splitting up their projects into subprojects to keep the amount of PRs manageable. mailing lists with a shared repo make this pattern quite natural; it also makes it very easy to only get notified of the parts of the project you're interested in.
Yes this is possible on GH using tags / subprojects, but it requires more effort.
natermer@reddit
I don't think they are going to be a amenable to adopt the practices of other projects that are far less capable and successful then their own. Even if it is 'modern' in any case.
This is literally the case for pretty much all software. What makes things work isn't the adoption of a new different process or adapting to 'modernisms'. It is a long slow grind of making it actually work.
It takes competency and hard work; adopting your workflow to the "correct methodology" or platform or abstraction isn't probably in the top 10 most important things to get right.
CrazyKilla15@reddit
What? This is ridiculous. of course having correct abstractions and methodology is in the top 10 most important things, thats the foundation of any and everything else. What do you mean you dont think the foundation is important?
with enough effort they could use smoke signals, massive networks worldwide of em, but that wouldnt make it a good idea or mean that migrating to a sane methodology or abstraction wouldnt be objectively better in every way.
And no, light signals, despite being faster, would not be better, the fundamental abstraction has to change.
Business_Reindeer910@reddit
You don't have to use web based tools directly. Most of the forges do support alternatives. I personally couldn't deal with only interacting with say github via the web, so I use the cli most of the time.
Hot-Employ-3399@reddit
Maybe I'm too spoiled by enterprise but every tracker worth its salt can be worked through email. That includes sending new stuff and forwarding stuff to people who want to read it all in email
(And also helps a lot with search.)
tristan957@reddit
I think the GitHub model is worse than the email model. I use both regularly. GitHub has no concept of versioned PRs, so reviews stack up in a linear fashion. It doesn't work well for contributions that go through multiple interations. It also leads people to do stupid commits like "cargo fmt" instead of using fixup commits and squashing them. GitHub truly deemphasizes the commit. It puts complete emphasis on the PR in totality.
PerkyPangolin@reddit
Not defending GH by any means, but I don't see how GH is preventing people from doing fixups or squashing commits. I'm part of a few FOSS projects that use GH and we enforce sane PRs, commit structure, and commit messages.
Early_Tap_1008@reddit
GitHub doesn't keep a copy of previous versions of commits if you're doing a force push Email does.
And part of the kernel workflow is explaining what you changed in each patchset version
CrazyKilla15@reddit
What re you talking about, its git, so yes it absolutely does? When you force push github shows both commits and you can view before and after just fine. AIUI they'll last at minimum 90 days, the default git reflog expiration time, if they're unreferenced.
And on github they're only unreferenced if no forks exist, so in practice for popular/large projects they will usually be referenced somewhere indefinitely, as all forks share the same object pool; This is why you can use a commit ID on any fork URL and view it fine, but not non-forks.
For example this torvalds/linux commit is also viewable on AsahiLinux/linux, but it is not viewable on torvalds/GuitarPedal
Github will helpfully warn "This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository."
Early_Tap_1008@reddit
90 days is completely unacceptable for a 30 year project though. You can go back 20 years on lore.kernel and see every version of a patchset
And the GitHub workflow that most people follow is pushing new commits instead of amending commits. Which is just a huge mistake imo
CrazyKilla15@reddit
They are just as capable of preserving pre-rebase commits as they as preserving emails. There is literally no question or challenge here and your insistence this would somehow be a problem just highlights how little you understand pretty much everything involved.
Early_Tap_1008@reddit
I do understand everything involved. I've been upstreaming patches for decades.
Git isn't the issue here.
There's a lot needed to copy something the lkml implements. The main being fully decentralised history and also decentralised patchset submission. If you look at how the lkml works, you'll see people submitting v1 of a patchset and then people replying in-line to everything.
They then submit another patch set and label it v2. It'll have the same number of commits but the commit messages have different text and the actual diff is different.
Now can you do that with GitHub? Not really.
It's assumed you're not opening a new PR for every new patchset. The UI doesn't even have any support for diffing PRs. Everything is on GitHub to host and somehow make it decentralised through regular backups or some other system which they haven't done.
Not only that but the kernel maintainers at the moment can easily just sit in a terminal and apply/reject/comment on patches. They don't even need any internet at all.
The lkml is actually a super nice workflow and really GitHub and gitlab are a huge downgrade.
CrazyKilla15@reddit
You are describing git itself. that is just git. this is literally just git. a canonical repo being on github/gitlab/codeberg vs git.kernel.org are all just as decentralized as the others. what are you talking about.
PRs are just git. PRs are just branches. You can diff branches just fine. You can even get patch files just fine. Please learn what git is.
Early_Tap_1008@reddit
Git doesn't have the cover letter, rejected PRs, pr comments. All of this is on lore.kernel and the mailing list history
mmstick@reddit
The reflogs do still exist and can be fetched after a force push with git. I'm not sure for how long GitHub keeps stale refs available, but it's why you can't force-push your way out of a security key leak.
gmes78@reddit
The point is that GitHub's UI is unable to do so, it just gives up instead of displaying previous versions of a PR.
bonzinip@reddit
GitLab is better for this; at least it archives the previous version and makes it easily reachable. It still doesn't have git range-diff though, so if you rebase your PR the diff is unreadable.
Zaphoidx@reddit
The patchset workflow is effectively opening a new PR each time for each new version.
I think that flow just doesn't translate to the PR world. It's not as simple a change as people are trying to make it
LEpigeon888@reddit
Can you easily get a diff of before fixup and after fixup ?
mitch_feaster@reddit
Don't think so. I use reviewable.io for this exact reason (and others, it's a great tool)
PerkyPangolin@reddit
Not sure what specifically you are asking here. If a fixup commit is pushed you can see what's in the fixup commit (obviously), and if it's squashed you can see what was changed in the last push.
gmes78@reddit
GitHub is absolutely terrible at handling history rewrites.
If you replace some commits, and then click the button to show what changed, it'll often say "can't find those commits".
It also doesn't use
git range-diff, so, when you rebase your branch, and it is able to show changes, it will include all the unrelated changes made to the base branch, and not just what changed in your commits.ABotelho23@reddit
You can absolutely enforce single commit PRs in tools like GitHub and GitLab.
mina86ng@reddit
That’s the issue. When working on a sizeable feature, this should be split into several commits but treated as one peer review. That’s what patchest process is about.
mmstick@reddit
PRs can have multiple commits. Pull requests are based on the branch that they're created from. Those with write permission to that branch are able to add additional commits at any point. Including rebasing and rewriting the history of that branch. All discussions will remain relevant to that pull request as it evolves.
The maintainer gets to decide how to merge branches in the end. Either manually with git on a terminal, or using one of the available merge options in the GUI. Manual merges and are detected automatically.
Those who want multiple versions of a patchset for historical reasons can create a separate branch with a separate PR that references the previous one to link the discussion and close it.
mina86ng@reddit
Yes, but GitHub funnels everyone into making PRs where individual commits don’t make logical sense. What u/tristan957 mentioned with ‘cargo fmt’ commits is an example.
Contributors sends a PR, gets review comments, and then pushes new commits addressing the comments. As a result, you end up with a branch with some incorrect commits followed by fixes to those commits. As a maintainer you either have to spend days explaining proper git workflow, squashing everything into a single functional commit or just doing this yourself.
This is the opposite of mailing lists which force people into doing proper patchsets. And since git provides tools for sending patchsests, maintainer doesn’t need to explain how to fight GitHub to get the proper PR.
mmstick@reddit
A PR is just a discussion thread for peer reviewing a git branch that's being proposed for merging from one branch to another. The individual commits in the git branch associated with a PR are no different from a series of patches copied and pasted over email to a mailing list. Those git patches are generated from commits in a git branch all the same. You're simply reviewing those changes through a web UI instead of an email thread.
If you have strict policies and guidelines, you'll have to explain that to new contributors regardless of what tool they use. A pull request that contains a
chore: cargo formatcommit is no different from a series of patches by email that also contain the same commits in patch format.If you want a PR to contain multiple commits that are individually formatted and verified, you can ask the contributor to rebase and format their commits. You could also set up git hooks that enforce this, as well as editor configs to configure code editors to auto format on save. There are many different ways to automate this for new contributors.
Alternatively, since it's mostly a waste of time for a maintainer to nitpick and demand that every commit in every PR to be perfect, you could ask them to split their changes into separate PRs so that they can be reviewed separately. If you like the structure of their commits, choose the rebase merge option. If you want to squash it into one commit and write the title and description yourself, use the squash merge option.
mina86ng@reddit
I understand all that, but the problem is that GitHub workflow is geared towards treating PR as a single atomic change. This is fine if the change is very simple. It’s terrible if it’s a sizeable feature.
I’m not talking about explaining policies and guidelines. I’m talking about explaining the proper git workflow and how you have to fight GitHub to use it.
It is different. If my patchset consists of a commit which introduces formatting violation and is followed by a commit which fixes that violation, this is a deficit patchset and should be rejected.
ABotelho23@reddit
In what way is email going to prevent people from putting in junk commits?
IAm_A_Complete_Idiot@reddit
patchsets make you review a series of commits, and not just the diff of HEAD. People submitting patchsets don't want to submit it all as one big commit instead, nor do they want to just review the entire change series at once. They want to review each commit individually, and land the entire feature at once. If any commit is bad, they fix that one commit, and send another patch series down the line.
mmstick@reddit
It's possible to specify a commit range in the URI
IAm_A_Complete_Idiot@reddit
That doesn't help if you review individual commits at all, though. If I made a commit, and amended that commit, how do you view the prior version of that commit to see what's changed? Not the entire branch. Just the previous version of that commit. The closest thing would probably be to make a new PR, and have it reference the old PR whenever you want to send a new version for review.
Reviewers would then review each commit one by one, and either Ack or Nack them. When they all get ack'd, you can merge the branch in however you deem fit.
mmstick@reddit
You may comment on an individual commit. GitHub provides references to the previous refs if you force-push them.
Zaphoidx@reddit
Am I going mad or do each of the replies have no correlation to the post before then?
edparadox@reddit
If your take about contributing over email, is "fighting with gatekeepers", I am glad you don't.
But avoid such baseless accusations which only reflect how little you know of it.
PerkyPangolin@reddit
My "take" is referring to the current process mixed with the recent-ish Rust in kernel drama and contributors leaving.
But you're right, of course - you're the only one on the internet who knows the kernel development process. And I'm also glad I don't contribute with my noobish kernel development skills.
EveryZookeepergame98@reddit
They should absolutely stop using mailing lists. It just sucks hard.
TampaPowers@reddit
Calls for using a forum instead often end in something awful like discourse or some worse re-imagined version of that though. You'd think a simple phpbb one would be right up there given the relative simplicity and rigid code.
I wonder if it has to do with a different mindset when working in lower levels of machines that anything more than just text causes fatigue. Just a theory tho.
PerkyPangolin@reddit
How is Discourse awful?
barfightbob@reddit
Basically anything that locks scrolling behind javascript is automatically awful in my book.
w0lrah@reddit
How is Discourse NOT awful?
I haven't actually run it myself but enough projects I follow or participate in have unfortunately switched to it that I feel confident saying that it's not possible to make it behave like a standard hierarchical forum the way anyone who spent years on phpbb, vbulletin, etc. would expect a forum to work.
I have yet to figure out why so many projects have chosen Discourse in recent years, but the vast majority of the places I've seen it have been open-core projects like FreePBX where there's a corporate backer so I have to assume there are some features that appeal to suits.
PerkyPangolin@reddit
I wasn't trolling and wanted to have something concrete. Do you have specifics about things you're not happy with? I've been using "classical" forums and newsgroups since the 90s, and I'm quite happy with Discourse across multiple non-corporate backed FOSS projects I'm part of.
I'm not trying to convert anybody and I'm not affiliated with Discourse.
TeutonJon78@reddit
Have you ever used an old style forum? That were so much easier to use.
Discourse is modern, bit a discussion forum doesn't need modern or fancy effects or whatever. It needs to organize discussions well and have robust search wkth easy to parse results.
Zaphoidx@reddit
But the old forums handled quoting replies terribly and forced you to navigate through several pages of comments to track a thread.
Modern solutions like Discourse make this so much easier to comprehend.
The search is also brilliant.
CrazyKilla15@reddit
The features of discourse but a more minimal/functional UI that doesnt depend on javascript so much or even at all could be pretty good
Thats a lot of what people are conflating, the features and technical enhancements with the interface
TampaPowers@reddit
phpbb really is peak in that regard. It does what it is supposed to do and if you need anything you can just change it to fit without having to fuzz with javascript in frontend and backend.
Most importantly though it works on widescreens, which anything designed with mobile usually doesn't do well. Why waste half the screen on blank space just for that feed-like design... bad enough reddit does that. Same problem. old.reddit so much friendlier, much less wasted space.
Hail_CS@reddit
god, i miss phpBB forums, it was so easy to navigate. there are still a few popular hacker forums that use it/something really similar
TeutonJon78@reddit
Ha, that was the exact one I had in mind.
VayuAir@reddit
Agreed, I wish they move to something more modern like gitlab
mmstick@reddit
Self-hosted forgejo would be better. Codeberg is a public insurance, but federation would be better than centralizing around a single host.
spazturtle@reddit
The Linux kernel breaks Codeberg's TOS so it couldn't be hosted there anyway.
mmstick@reddit
Not sure why they'd use it over self-hosting as they already do. It's for small projects that aren't going to run their own forgejo instance.
CrazyKilla15@reddit
The funny thing is they are, the mailing list is a relic that a lot of major development doesnt happen in. Subsystem maintainers have full control over how their subsystems are managed, and a lot of big ones have moved to gitlab for PRs, reviews, etc, and only send the finished product up to the mailing list for inclusion by Linus.
For example much of drm works this way https://gitlab.freedesktop.org/drm/kernel
Preisschild@reddit
Gitlab has been enshittified the last few years. Just more llm slop integrations while the CI suffers.
aeropl3b@reddit
Considering the latest news about Gitlab, and just the general community squashing retcon bug/issue cycling around their development...I would say to not use gitlab if possible.
gburgwardt@reddit
Are you thinking of github?
What's the scuttlebutt with gitlab
aeropl3b@reddit
GitHub has all of its own issues, the largest being Microsoft. Similarly though, pushing development to prioritize AI slop over good software.
thefossguy69@reddit
https://about.gitlab.com/blog/gitlab-act-2/
aeropl3b@reddit
That is part of it. They are basically doing the "lay off a bunch of people. Replace them with AI." Dance. This is on top of their already pretty toxic attitude towards the gitlab community. Basically, unless you are paying them a ton of money, don't expect them to fix any bugs or performance issues. It is all about shiny features and AI slop.
Zaphoidx@reddit
Seems like realistic changes for the modern development world to me
thefossguy69@reddit
I don't disagree. The person whom I replied to wanted to know what's going on with GitLab.
FlukyS@reddit
It is hard because while I understand they want more developers the areas that they want to push Rust is mainly in the area of driver engineering but that can be quite limited in scope since you can only really do work and test if you have available hardware, so either you are doing it because you already have the hardware and it isn't working or you work at that company. so it is kind of a chicken and egg problem. The bit that Rust helps with though is that it lowers the barrier when someone does want to do that work.
alonjit@reddit
Does it though? I get the memory safety aspect, but lowering the barrier of entry? C is as simple as it gets....
CrazyKilla15@reddit
Yeah thats the problem. Its trivial to write clean looking, compiling, warning free code... thats extremely wrong and broken. It is very simple to write use after frees, double frees, memory leaks, out of bound accesses, over/under-flows, etc.
Used_Ad_5831@reddit
Coincidentally why goto is in violation of a lot of style guides.
lifeeraser@reddit
Linux codebase is notably not one of them—they use
gotoa lot.FlukyS@reddit
goto is super good for performance in certain situations.
bonzinip@reddit
It doesn't use goto for performance, 99% of the time it's for resource cleanup. These days
guard()replaces it.Hot-Employ-3399@reddit
goto is relatively easy to avoid. What's worse is pointers. They are pretty much core features used everywhere, and can easily be null when you `(*ptr)++`. Unlike C++, C has no references, which is much better to say how much null is not expected.
(Rusts pointers are also shit, sometimes worse than C, but in code they are not used that much, which surpisingly makes them shittier -- they can work in debug but break in release as these builds handle resources very differently, so during development you will not even notice the problems)
paholg@reddit
C may be small, but it certainly is not simple. Example: https://blog.regehr.org/archives/213
asmx85@reddit
C might be simple but it is by no means trivial/easy. Please don't confuse this. Chess has simple rules but it's not trivial to play.
FlukyS@reddit
Yeah C is pretty clean but note that it is the kernel so any memory issue or performance issue is going to be a big issue. Rust isn't just type safety, it also has one of the best toolsets of any language like with linting and formatting baked into everything, great testing workflows, great error messages from the compiler. It is super easy to use Rust, I find Rust easier to use than Java or JS.
sjepsa@reddit
C++ would have been the natural solution
john_crimson81@reddit
the actual bottleneck has never been people who know rust — its the mailing list workflow being genuinely alienating for new contributors. you have to already know the culture to navigate a patch review without getting demoralized by someone who responds in three words and moves on. github prs arent happening but at least a modern patch tracker would reduce the friction significantly. the rust devs who show up and bounce usually arent bouncing because of the technical challenge
mrlinkwii@reddit
i think part of the reason most wont work on the kernal is because its out of touch what modern devs use
sylvester_0@reddit
So do you want JS in the kernel or something? I think the reason "most" don't work on the kernel is because they have little reason to do so. It sits at a fairly lower layer than what most of us care about. It's the same reason I'm not typically reading and commenting on RFCs for network protocols. It's beyond me and I don't really care.
NotQuiteLoona@reddit
Jarvis, sort by controversial.
Oh, wait, not enough comments so far.
sylvester_0@reddit
Jarvis, give me guaranteed karma and bring me to orgasm.
QazCetelic@reddit
Just visit the Phoronix comment section
NotQuiteLoona@reddit
"Lunduke will kill himself"
🥰 4Looks nice as of now.
Infinity-of-Thoughts@reddit
That sounds like an ok thing.
adi8888@reddit
https://pbs.twimg.com/media/Fm1CIcZaYAMv96e.jpg
GodsBadAssBlade@reddit
Unfortunately nowadays this is also a dinner bell for slopcoders trying to make a name for themselves off the back of something that doesn't get context very well.. euhg I hate the 2020s
bobbie434343@reddit
Groah....
FortuneIIIPick@reddit
Why would someone want to use a language like Rust that looks very much like a toy language when they can use a professional looking language like C or C++ which go back to the very roots of UNIX.
CrazyKilla15@reddit
Because, unlike you, the kernel developers actually know what theyre talking about and choose Rust for good reasons that they have explained very well over the years many times, and which you dont care at all about because you're simply a troll pretending as if it isnt a question the kernel developers discussed and answered ages ago.
toikpi@reddit
Please do some research in future.
UNIX first appeared in 1969. The C language first appeared in 1972. The C++ language first appeared in 1985.
Quote from the Wikipedia page on UNIX.
Sources
https://en.wikipedia.org/wiki/History_of_Unix
https://en.wikipedia.org/wiki/C_(programming_language)#History
https://en.wikipedia.org/wiki/C%2B%2B#History
dkopgerpgdolfg@reddit
Get your facts right. Unix started off while C didn't exist yet. And mentioning C++ at all is just funny.
Thank you for your opinion.
Admirable-Safety1213@reddit
Because C abd C++ are dirtier and unsafe, the roots made a good job but it's time to move beyond them, tying Linux to old UNIX will simply limit it, additionally Linus himself said that he would personally hunt whoever suggests using C++ in Kernel space while he approves Rust because of how it tackles memory
Also the distinction of "toy" and "profisional" language is dinnering, childish and uninformed, C was considered a toy language that never would replace ADA, LISP por Haskell but now the C family reigns, Python has replaced Perl aa the favorite glue language and yet it uses tabs for nesting while also giving MATLAB a run for its money, the Java Virtual Machine itself had managed to be considered by itself an alternative to traditional native apps in some archutectures and the list goes and goes, Languages evolve, programmers should too
dkopgerpgdolfg@reddit
To all these people talking about forums, github/gitlab, etc.:
I'd like to remind that the kernel development is quite decentralized, which some might not remember/understand here. Independent of the specific software that is used, this requirement won't go away anytime soon.
powerslave_fifth@reddit
But has he consulted with larpers about Rust though?
rumblpak@reddit
Best we can do is AI devs. Take it or leave it.