Kees Cook cleared of malicious git shenanigans
Posted by MatchingTurret@reddit | linux | View on Reddit | 79 comments
The incident reported in Well...well....what you know! Kees pissed off Linus again! ....meh on r/linux has been resolved:
Linus, this is accurate and I am 100% convinced
that there was no malicious intent. My apologies for being part of the mess
through the tooling.
I will reinstate Kees's account so he can resume his work.Linus, this is accurate and I am 100% convinced
that there was no malicious intent. My apologies for being part of the mess
through the tooling.
I will reinstate Kees's account so he can resume his work.
reini_urban@reddit
But it doesn't look like the broken b4 tool will get fixed to avoid the author, committed and date changes. I would throw it away completely. I have written a similar tool, and this would never do such stupidities.
FryBoyter@reddit
Based on the two posts by Konstantin Ryabitsev in the discussion in question, I actually assume that b4 will be modified in such a way that this will not happen again.
reini_urban@reddit
Based on the two posts by Konstantin Ryabitsev in the discussion in question, I actually assume that b4 will be NOT modified in such a way that this will not happen again.
JackXDangers@reddit
That original Reddit post was…something. Lots of professional Linux consoomers speculating on stuff they have no idea about
rebbsitor@reddit
That's every post on Reddit/the Internet basically.
NoTime_SwordIsEnough@reddit
I pretend to speak with authority, so my conjecture is therefore expertly-informed truth.
fractalfocuser@reddit
As someone who's chatted with Kees a couple times I was pretty saddened but not surprised with the response. Dude is a really chill guy and genuinely cares a ton about Linux, he's done a lot not just in commit count but in outreach too. For people to be trashing him so quickly and easily really says a lot. My friends and I knew it had to be some weird mistake. Linux core devs deserve more respect IMO
TracerDX@reddit
Consider the implied natural negativity of the Internet commenter a salve to this. Most of us admire and respect these devs. Shitty people are loud.
idontchooseanid@reddit
That's just entire /r/Linux. People who did 0 systems programming nor studied other systems bash anything but Linux with various levels of copyleft software and demand difficult stuff without seeing the technical and economic reasons behind/against/for it. Some even call themselves "developers" and earn money from various tasks jumping from one framework to another.
Good stuff takes a lot of time and mishaps happen in engineering. An early mishap that's resolved is way better than deploying it.
turdas@reddit
It was so obvious most commenters in that thread didn't actually click through the mailing list reply chain to see Kees's response.
Vynlovanth@reddit
At the time it was posted, Kees hadn’t responded yet. That was like 6 hours later.
ImpossibleEdge4961@reddit
Wouldn't that get filed under "If there is no response then aren't you making decisions based off incomplete information?"
A reasonable baseline expectation for adults should be that they have some sort of rough idea of when they don't know enough to come to a reasonable decision by that. I know many supposed adults will try to use "but I didn't know that" as a categorically good excuse for jumping the gun (when the situation doesn't call for it) but that's an explanation rather than justification.
If you're older than 15 years old then you're old enough to know that if someone central to understanding what happened hasn't commented and it just now happened then maybe just wait? If he didn't respond for a day or two yeah it would make sense to interpret the silence as the response and then proceed but that's not what happened here.
But I suspect the people who do that are used to just hiding behind "well I didn't know that" and people letting them off the hook so they never feel the need to update their habits.
solid_reign@reddit
What is the purpose of a discussion forum if we're not going to talk about things we know nothing about? And I say this as an expert on discussion forums.
Business_Reindeer910@reddit
to discuss things you do know about :)
solid_reign@reddit
I'm going to file a complaint to have your internet usage license revoked.
MichaelTunnell@reddit
If everyone followed the mantra of “if there is no response then aren’t you making decisions based off wildly incomplete information” then there likely wouldn’t be any drama about anything or at least a minuscule fraction of the amount of drama in the world but alas the mantra of humanity is usually closer to “jump to conclusions because patience is overrated”
Dalnore@reddit
You are right, but jumping in on drama without any delay and attempts to figure out what actually happened is always very good for clicks and karma.
turdas@reddit
Fair enough. I don't know if that makes it better or worse.
nialv7@reddit
people really should stop idolizing Linus' bad behavior...
Business_Reindeer910@reddit
and you got downvoted for this one.. it's really sad.
meatpops1cl3@reddit
just-world fallacy in effect...
77slevin@reddit
That was a whole lot of drama that should have been kept indoors. Linus really can be a drama queen.
senectus@reddit
Nah man, his reaction was on the ball.
It's a high stakes role they all have. this would have been a mild shock, but keeys will be fine.
Juts@reddit
Eh, they keep things open. He could probably be less confrontational but its probably a survival tactic when you are managing a project this large. No space in kernel development land for thin skin. If they fuck up and let something malicious through it would be catastrophic. They submitted janky shit, they get yelled at.
mikeymop@reddit
Personally I don't blame him after seeing a lot of attacks on OSS supply chains. XZ being an example.
PDXPuma@reddit
I do blame him, though. He immediately assumed his tool was not the problem, even though Kees said he has no idea how it happened. He could have looked at the trees and pulled diffs to see it made no sense. Instead, he immediately attacked because it couldn't have been git that was the problem.
It took K recreating the issue, and proving it, almost twice, before this got fixed.
natator99@reddit
Git wasn't the issue. A tool ON TOP of git WAS. (b4)
PDXPuma@reddit
Fair, but that's still part of the git workflow they use. I was speaking in the broader sense there since Linus has made it clear numerous times that the git binary is but one part of the whole workflow they use.
nextized@reddit
The worst thing in this discussion was that the assumption that it was malicious was never in question. I saw multiple instances (for example YouTuber who reported this as an attempted supply chain attack). Never was there any proof provided but the conclusion was clear. Even without Kees actually attempting any sort of injections as the commits were still left the same and only the commit metadata altered.
mok000@reddit
I considered the possibility that Kees' computer could have been hacked by malicious actors, perhaps agents of a state, trying to get compromised code into the Linux code base. I wasn't convinced Kees was doing something bad on purpose, on the other hand, sometimes people you think you know well do bad things. Linus was correct in pushing to get an explanation. In the end, it's about millions of peoples' computers, and not hurt feelings. I'm sure they'll patch it up.
tonymurray@reddit
People can't separate the fact that the abnormalities absolutely must be assumed malicious for security reasons.
But this does not mean we are assuming the developer was being malicious.
EODdoUbleU@reddit
That was my assumption reading Linus' original message. Step 1: lock the account; Step 2: explain. A rebase mistake and compromised credentials were equally as likely.
It's not like Linus came out the gate with "F this guy and F the plane he flew in on", even though that could easily be inferred to be the tone. This is Linus after all.
singingboyo@reddit
Yeah if you read it closely, it’s closer to “what the fuck is going on/what the hell did you do? How did we even get here unless you got compromised?” tone and appropriate follow up by assuming it was compromised/malicious, as opposed to “screw this guy for being malicious.”
JockstrapCummies@reddit
The quicker these "Linux Youtubers" die off, the better it is for the whole Linux community. They peddle so much bullshit that newcomers eat up it's tragic.
MichaelTunnell@reddit
I’d appreciate an additional qualifier added to that statement such as “Sensationalist Linux YouTubers” because not all of us participate in that sort of silliness 😎
lottspot@reddit
It never made a bit of sense that someone would try to sneak malicious changes into the kernel by changing hundreds of sha1s. I feel that some of the people reporting on this knew better but played up the negative angle for clicks.
BinkReddit@reddit
We live in a world where people profit from controversy. 😞
rien333@reddit
JusticeForKees
RedditNotFreeSpeech@reddit
Git is hard even if you are an expert. I'm not an expert but I'm generally proficient and I've run into some sticky situations even doing some things that should have been basic. Glad they got it sorted.
Business_Reindeer910@reddit
the fact that so many people here assumed it is the real problem.
deja_geek@reddit
How about Linus assuming it was something malicious? Linus’ message specifically says it looked malicious.
Dalnore@reddit
Linus is known for his rather harsh style of communication. He made the right decision to request immediate access revocation before figuring out what happened, but people trusting his assumptions (and thinking he can't ever make mistakes) without waiting for the story to develop is a problem.
For example, there are a lot of responses in this thread which go along the lines "He created git, so he definitely knows better than everyone else", which is a crazy way of thinking. What I get from the exchange so far, there seems to have been some quirk in the git helper tool called b4, which was written mostly by this Konstantin Ryabitsev from the thread, and Linus didn't know about this behavior. Git and especially the way it's used in the kernel development are complex enough to make it impossible for one person, even of Linus's caliber, to know absolutely everything.
steak4take@reddit
You mean he's an arrogant prick. Which he is.
washtubs@reddit
Also the amount of people who know that Linus created git and don't know who Junio Hamano is despite using git every day is so sad to me.
ryobiguy@reddit
I'd love to hear a Linus style rant about how Linus didn't know that.
PassionGlobal@reddit
If I saw something like that...I wouldn't send the message he did, but I would definitely lock the account pending an investigation
hackingdreams@reddit
r/linux is not a healthy linux community in the state it is in. It's why I have more or less abandoned this subreddit - I'll read through it from time to time, but there are some really, really bad entities in this subreddit that... don't need to be here. It's criminally undermoderated, and the moderators have some... interesting biases towards what they deem to be acceptable behavior from known trolls.
Misicks0349@reddit
I don't know if there is a good linux fourm nowadays tbh, somehow phoronix is even worse.
not_a_novel_account@reddit
Linux isn't a topic worthy of heavy discussion. Development discussion is focused in the LKML, if you care you're already there, and the interfaces for user-space developers move slowly. There's certainly no daily, reddit-friendly churn.
A real Linux technical forum would be very slow moving, like /r/cpp, where there are maybe 2-5 real posts a day and dozens of quickly removed posts that have no technical content in them (questions, show&tell, drama-posting, etc).
/r/linux is mostly for non-technical users who view an operating system as a lifestyle choice and want to do lifestyle posting.
Business_Reindeer910@reddit
IN the kernel sure, but this subreddit isn't just about the kernel. There are plenty of other things changing all the time in the ecosystem that are worth talking about almost every single day.
not_a_novel_account@reddit
Not really, even if you take the full scope of user space, systemd, pipewire, wayland, the various layers built on top of these, the changes happen in the scales of weeks and months not days and hours.
And anyway the point is moot, /r/Linux will never have the kind of moderation necessary to focus it on the technical developments of kernel space or user space. The posts that get upvoted here are "look what I installed Ubuntu on" and "First time Linux user, Windows sucks!"
You'll never see "Understanding the latest Wayland protocol extensions", "In-depth on the Pipewire API, Advantages and Disadvantages", or "An introduction to completion-based asyncio with io_uring", the kind of technical content that is actually useful for building things on Linux.
Business_Reindeer910@reddit
I wasn't even talking about going that deep, although that'd be nice.
There's stuff that's more high level like new compositor releases with new features or new updates in shells to adapt to.
Misicks0349@reddit
the LKML is pretty much exclusively about the kernel, there doesn't seem to be a good place for the broader linux ecosystem/userspace.
CrazyKilla15@reddit
Its a top down issue, /r/linux is a grainy reflection of the LKML community. After all, what started this whole thing was Linus going straight to unambiguously malicious action by kees, and everyone else just repeated that.
Theres a world in which the exact same actions were taken, but not painted as obviously and unambiguously malicious by kees. Something like "this is weird, this looks like potential compromise, disable kees account until this can be investigated, just in case."
Business_Reindeer910@reddit
Good point. I should have jumped over to see what they were saying in places like lobst.rs or whatever
kevkevverson@reddit
It’s mostly just circlejerking desktop screenshots with the caption “I uninstalled windows today”
mazarax@reddit
If there is a (small) chance it is malice, then treat it as such.
That is how the project remains secure.
washtubs@reddit
Someone on the last thread said Linus was wrong for reacting the way he did, and that he should have just said "hey this is sus" and suspend the account to investigate further. That guy was downvoted to oblivion.
This is how security should be conducted: you immediately close the account and investigate, all the while shutting the fuck up.
Foul play does not necessarily imply malice. It could be Kees' account was compromised for example.
Jumping to the conclusion that these security engineers that you've formed a working relationship over time are suddenly corrupt was totally unnecessary and premature, both on the part of Linus and also this community. Y'all need to own up.
Business_Reindeer910@reddit
i think that was me (the same person)
washtubs@reddit
Oh, nice! I didn't notice 😄
Business_Reindeer910@reddit
thus you shut down the account immediately and let your processes play out. No accusations are necessary. (which is what i said in the the first place)
ThomasterXXL@reddit
Or you steer your entire organizational culture towards infighting and self destruction.
Einaiden@reddit
I'll remember this next time I struggle with git and have imposter syndrome.
admalledd@reddit
The instant I read Linus's rant, I was 90% sure it was human+tooling error, and 10% "could it be a compromise?". Anyone who has had to do large and wild rebasing in anything close to the Kernel's email based patch flow should have horror stories of screwing everything up. Normally things are screwed to the point of throwing it away and trying again, but here it seems it worked just enough to slip past a simple "does my code look right, does my short-log look right-ish?".
Further, this is all where the process is working as intended: something "funky" happened and so someone (Linus in this case) called it out to have the admins revoke access in case something nefarious afoot. Further investigation found it to be a tooling error, access is restored.
Granted, what a wild tooling error, but makes sense in retrospect, and surprising it hasn't happened before yet.
anomalous_cowherd@reddit
If you have a reasonable suspicion of malicious intent from a specific user then it makes total sense to block their access while it is investigated, then unblock them quickly if it proves to be unfounded.
The idea of waiting until they wake up and get a chance to explain themselves many hours later seems polite but is asking for trouble. I'm looking at it from my cyber security background - you can't know if that account has been compromised somehow and is going to carry on being malicious before that real user comes back to you, if they even can respond with their compromised account.
You have to act quickly to be better safe than sorry, but the investigation has to also be quick and fair to avoid acrimony. Users also need to understand this and drop their ego if they are on the end of it, we all know there are plenty of big egos out there, it comes with the territory!
Business_Reindeer910@reddit
Did you see anyone suggesting that?
anomalous_cowherd@reddit
It feels like it, but it's possible every instance of it is actually about waiting to comment until there has been a response rather than waiting to take action.
Business_Reindeer910@reddit
that's because the only person who has to take action is the people actually involved in the project rather than a bunch of folks in the peanut gallery on reddit!
Megame50@reddit
I consider myself pretty skilled with git, but I don't think I could have provided a detailed annotated account of my reflog after a complex rebase the way Kees did in his reply.
ContagiousCantaloupe@reddit
Kees Cook is the goat
cookaway_@reddit
Can we expect the message where Linus apologizes for jumping to conclusions or are we just pretending he's "more mature" and "welcoming" because he censors a couple bad words.
MatchingTurret@reddit (OP)
Well...
There was an actual problem and I think it was Linus' obligation to call that out and ask for an explanation. That explanation was provided and accepted.
This weren't wild, unfounded accusations but something based on really suspicious evidence. "Better safe than sorry" isn't wrong. Linux is important enough to be a potential target for malicious actors, so, overall I don't think Linus owes a public apology. I don't know his personal relationship with Kees, so I don't know whether this warrants a personal one.
mcj@reddit
"You're being charged, sir."
"Oh, yeah? What are the charges?"
"Shenanigans, sir."
solid_reign@reddit
Linus, I'm really sorry, I used this stupid tool called "git" and couldn't get it to work right. Maybe we should move to mercurial. Git sucks.
lerliplatu@reddit
The message that OP quoted from without linking.
turdas@reddit
The post title links to the email. Reddit UI confusion. Used to be you couldn't have a link and text in the same post.
SquareWheel@reddit
I don't even know how people create these hybrid link/text posts, but they trip me up every time. I never notice the title is also a link when there's a text post.
sequentious@reddit
It's annoying as an old-reddit user. You need to switch to new-reddit to get some of these post features.
FryBoyter@reddit
You may be able to create such posts using the function described at https://old.reddit.com/r/modnews/comments/vj4evp/text_now_available_on_all_post_types/.
However, it may be that you need the Reddit app for this. At least I can't create such a post with a browser.