The title is a bit misleading because stenberg isn't claiming curl is near zero bugs, he's more describing the philosophy and process of taking each bug seriously rather than treating them as inevitable noise.
I feel that it's like that old adage - the more time you have to do something, the more the work expands to fit the limit. As it gets easier to write correct code, we set our sights higher, demand more from the code, put it in harder situations.
There used to be a time where nobody cared about anything other than ASCII. Then we decided 'no, our code should be usage by everyone' and invented a new category of bugs for ourselves. Ditto with accessibility, or distributed systems.
Bugs are defined by deviance from the correct model of operation for a piece of software, and the notion of a 'correct model' is ultimately derived from transient human values. The computer doesn't know if a piece of code is a bug or not, it's just zeroes and ones.
Can you explain what you mean exactly? Are you saying it's a bad idea?
It's a very valuable feature for certain use cases. Like geological surveying in remote areas with shitty internet that needs to upload tens of Gigabyte of already highly compressed data.
In TCP we already have a feature where packets are split into smaller packets and transmitted as fragments. Problem is that packets can come in out of order and the system has to wait for all of them before reassembling the original structure that was sent by the sender and handing it off to the server application that was listening for it.
In http application land we already implemented this at application level with a fancy flow where
You signal how much you're going to upload
You get back a token/ticket/cookie/identifier that can group your chunks/blobs/parts/packets
You upload each chunk individually with the ticket within the expected time constraints
Server reassembles back the uploaded chunks once total upload size matches the original signaled upload size
There's NO FUCKING REASON to have this at protocol level in http. Even in your usecase HTTP is the wrong protocol. What you want is the torrent protocol or something that was designed around out of order chunk transfer and resuming.
Thanks for the explanation, that's a very good point, they should be using torrents tbh. In the case I'm referring to, I think they are just in the mindset that uploading zip-archives to cloud servers is the logical choice.
The most undersold use of LLMs is analyzing code for bugs or improvements. All the energy is going to code production even though that is a hot mess. My projects are moving much faster than they were before even though I'm writing all the code, simply because we run into fewer traps. It's going to be a long time before we have a healthy relationship with LLMs but it is out there.
I think a challenge here is something I've observed that I don't have a name for so I call it "the critic fallacy" or "bias". Basically, if you ask an expert to review something, especially publically, you're putting their reputation on the line to find something "interesting" to say. They're very unlikely to just say, "it's very good, no further comments." No, their going to think and think about something critical they can say that will make them sound observant and smart. So it can be very difficult to get an easy "green light" in situations where your solution is pretty much fine and adequate, if you bring in an expert to advise. That's not to say you shouldn't, or course you want an expert opinion, but I think it's important to be aware of this bias: if you task someone to find a problem, 99% of the time they're going to find a problem.
I suspect that a similar thing happens with AI review. If there are real problems hopefully they find and surface them. But they're also likely to find "problems" that don't really deserve the attention they might draw to them. Having an AI reviewing PRs doesn't mean you don't need a real person to review the reviews, is what I am saying -- and hopefully management trusts the humans in this respect. Otherwise they might find this kind of "quest for 0% bugs" to have no real end...
I often use AI to proofread my writing, asking it to find typos, grammatical errors and the like. I get its review, fix the errors, then re-submit it in a fresh chat. You can very clearly observe that when it runs out of 'real' mistakes to find, it starts to simply come up with fake ones - either things that aren't really problems, things which redefine the terms of the argument, or simple fabrications. If you ask for a list of five items, it will give you a list of five items, come hell or high water.
In the sense that a user or product manager might call something a "bug", I'm not sure it's really possible to approach zero bugs, practically. A "bug" doesn't have to be a math or logic error, or a typo, etc. Some bugs occur when there's a mismatch between "intent" and "reality". But the issue is that "intent" is often less sharp than the reality implied by the code. Large systems have lots of interactions and considering all of them a priori isn't really tractable. Any time some obscure combination of factors isn't considered opens the door to a case where the full precision of "intent" isn't realized. In any case, to avoid getting too philosophical, I think bugs are a fact of life and whenever there are only a finite number of resources chasing a moving target (so always, in real life), bugs may practically emerge.
one man's bug is another man's intended workflow (link xkcd here)
I agree with the sentiment, but some things are very obviously bugs - regardless of intention. taken to its extreme - if somebody presses the power button their computer and the OS doesn't boot up, is that a bug? by any standard definition, yes... but what if they only want their computer to turn on some LEDs that are connected to the motherboard?
business forwards some weird, but completely expected behavior to bsa. bsa forwards email to devs "seems like a bug". devs create ticket to "fix the bug." tale as old as time
Bugs are ultimately a social construct. As the old saying goes "It's not a bug, its a feature."
Obviously there's program behavior that is virtually universally undesirable (any thing that leads to elevated privilege or remote code execution), but beyond that bugs are defined by the spec. You can create and fix any amount of new bugs instantly by simply altering the program spec, without ever touching the code. The author even implicitly suggests this, they only track data for vulnerabilities, not for the other kinds of business logic errors.
So given that, its probably better to narrow the question to "Approaching zero vulnerabilities?" and for that I don't know, but I don't think its just a matter of tooling. The amount of bugs (vulnerabilities and logic) in software correlates with the size of its code. Not just the source code, but the dependencies, compiler/interpreter, and even OS syscalls. If we have a large surface area of code for bugs to manifest on, then it seems likely that even with near perfect tooling, you'd just just end up in a game of bug whack-a-mole, or with bug fixes that are mutually exclusive with intended behavior and other bug fixes.
If we want to approach zero vulnerabilities (and zero logic bugs, since that is still desirable), then we need to apply advanced tooling to new projects with smaller designs. We can't just have software code bases balloon forever and expect tooling to pick up the slack and fix all the bugs for us at scale.
Let's be utopic for a sec. Please understand that I don't "expect" this to happen, just trying to imagine a good equilibrium, even if it's not generally possible.
The problem with a lot of tools is that they spread out in feature-space much faster than their bugs can be found and fixed. For minimizing bugs, it'd be better not to evolve feature-complete packages.
Could we say that LLMs might help with that? I'd say yes, for a very simple reason. LLMs make it possible for the end users to rely less on the additional features. If you have a basic LLM, even local one, you might be able to do more with less features from your dependencies. We don't need to support every fancy weird use-case, every little CLI or UI affordance, because many of the users can figure them out on their own with their chatbots, as long as we give them load-bearing, well-tested basic functionality in good packages.
Of course there's a force that's pulling in the other direction. As features seem to many people to be "cheaper" to implement and maintain, their open-source packages spread out and do more and more. That's a counter-productive disaster that is absolutely pointless, as nobody really needs all of them. I think we can only hope that a taste will develop to avoid tools, packages and libraries with excessive features like the plague.
liampolemarch@reddit
The title is a bit misleading because stenberg isn't claiming curl is near zero bugs, he's more describing the philosophy and process of taking each bug seriously rather than treating them as inevitable noise.
araujoms@reddit
No, he is not. Did you even read the article? Or are you just a bot?
Kok_Nikol@reddit (OP)
I took the title directly from the article, there's a "?" for a reason in the end.
frakkintoaster@reddit
Approaching 0 bugs zeno’s paradox style
Sability@reddit
Except every now and then Heracles takes half a step that sends him twenty full paces back
trejj@reddit
The number of bugs is not proportional to how good the software is: it is proportional to the number of users your software has.
Perfect software will never have zero bugs left when it is complete, because the world it is applied to is imperfect.
Saint_Nitouche@reddit
I feel that it's like that old adage - the more time you have to do something, the more the work expands to fit the limit. As it gets easier to write correct code, we set our sights higher, demand more from the code, put it in harder situations.
There used to be a time where nobody cared about anything other than ASCII. Then we decided 'no, our code should be usage by everyone' and invented a new category of bugs for ourselves. Ditto with accessibility, or distributed systems.
Bugs are defined by deviance from the correct model of operation for a piece of software, and the notion of a 'correct model' is ultimately derived from transient human values. The computer doesn't know if a piece of code is a bug or not, it's just zeroes and ones.
So we beat on, boats against the current...
matthieum@reddit
I mean, people in those other countries -- like us Europeans, you know -- didn't wait for English-speaking folks. We just went ahead without you...
OMGItsCheezWTF@reddit
curl is also in a place where it's not directly in control of its scope. The scope is defined by advances in the HTTP space.
HTTP/2, HTTP/3 etc all added new requirements.
The IETF are looking at various extensions to QUIC like OHTTP, resumable uploads, multipath QUIC, FEC etc.
These will all change the scope of what curl needs to support and be potential vectors for new bugs being introduced.
Worth_Trust_3825@reddit
oh boy. lets introduce fragmentation into application layer.
CramNBL@reddit
Can you explain what you mean exactly? Are you saying it's a bad idea?
It's a very valuable feature for certain use cases. Like geological surveying in remote areas with shitty internet that needs to upload tens of Gigabyte of already highly compressed data.
Worth_Trust_3825@reddit
In TCP we already have a feature where packets are split into smaller packets and transmitted as fragments. Problem is that packets can come in out of order and the system has to wait for all of them before reassembling the original structure that was sent by the sender and handing it off to the server application that was listening for it.
In http application land we already implemented this at application level with a fancy flow where
There's NO FUCKING REASON to have this at protocol level in http. Even in your usecase HTTP is the wrong protocol. What you want is the torrent protocol or something that was designed around out of order chunk transfer and resuming.
CramNBL@reddit
Thanks for the explanation, that's a very good point, they should be using torrents tbh. In the case I'm referring to, I think they are just in the mindset that uploading zip-archives to cloud servers is the logical choice.
Worth_Trust_3825@reddit
It's fine to upload archives but in your case they should be chunked due to possible intermittent network failure.
CramNBL@reddit
winrar? Most of the data is massive blobs of scale offset compressed HDF5 files.
I don't think they're interested in implementing their own cloud infrastructure either, but thanks for the input.
_chookity@reddit
I love this take, and I think it’s a more positive way to think about job security than “ai could never do what I do”.
bwmat@reddit
How dare you denigrate my SIGILL-handler-based programming style
TrespassersWilliam@reddit
The most undersold use of LLMs is analyzing code for bugs or improvements. All the energy is going to code production even though that is a hot mess. My projects are moving much faster than they were before even though I'm writing all the code, simply because we run into fewer traps. It's going to be a long time before we have a healthy relationship with LLMs but it is out there.
Kok_Nikol@reddit (OP)
I think using LLM as evaluators of any kind is pretty well used.
Sisaroth@reddit
So what are the actual tools they are using? I only heard about claude mythos. Is there anything free available?
Kok_Nikol@reddit (OP)
That depends on the people submitting the reports, most likely all of the tools
You can probably get more details in the actual reports - https://hackerone.com/curl/hacktivity
radarsat1@reddit
I think a challenge here is something I've observed that I don't have a name for so I call it "the critic fallacy" or "bias". Basically, if you ask an expert to review something, especially publically, you're putting their reputation on the line to find something "interesting" to say. They're very unlikely to just say, "it's very good, no further comments." No, their going to think and think about something critical they can say that will make them sound observant and smart. So it can be very difficult to get an easy "green light" in situations where your solution is pretty much fine and adequate, if you bring in an expert to advise. That's not to say you shouldn't, or course you want an expert opinion, but I think it's important to be aware of this bias: if you task someone to find a problem, 99% of the time they're going to find a problem.
I suspect that a similar thing happens with AI review. If there are real problems hopefully they find and surface them. But they're also likely to find "problems" that don't really deserve the attention they might draw to them. Having an AI reviewing PRs doesn't mean you don't need a real person to review the reviews, is what I am saying -- and hopefully management trusts the humans in this respect. Otherwise they might find this kind of "quest for 0% bugs" to have no real end...
Saint_Nitouche@reddit
I often use AI to proofread my writing, asking it to find typos, grammatical errors and the like. I get its review, fix the errors, then re-submit it in a fresh chat. You can very clearly observe that when it runs out of 'real' mistakes to find, it starts to simply come up with fake ones - either things that aren't really problems, things which redefine the terms of the argument, or simple fabrications. If you ask for a list of five items, it will give you a list of five items, come hell or high water.
nirreskeya@reddit
https://en.wikipedia.org/wiki/Bugzilla#Zarro_Boogs
1vim@reddit
Fewer bugs means simpler code. Complexity is where bugs hide.
OffbeatDrizzle@reddit
A implies B does not mean B implies A
complexity is sometimes needed for obscurity / security, but for actual business logic, yes
PortiaLynnTurlet@reddit
In the sense that a user or product manager might call something a "bug", I'm not sure it's really possible to approach zero bugs, practically. A "bug" doesn't have to be a math or logic error, or a typo, etc. Some bugs occur when there's a mismatch between "intent" and "reality". But the issue is that "intent" is often less sharp than the reality implied by the code. Large systems have lots of interactions and considering all of them a priori isn't really tractable. Any time some obscure combination of factors isn't considered opens the door to a case where the full precision of "intent" isn't realized. In any case, to avoid getting too philosophical, I think bugs are a fact of life and whenever there are only a finite number of resources chasing a moving target (so always, in real life), bugs may practically emerge.
OffbeatDrizzle@reddit
one man's bug is another man's intended workflow (link xkcd here)
I agree with the sentiment, but some things are very obviously bugs - regardless of intention. taken to its extreme - if somebody presses the power button their computer and the OS doesn't boot up, is that a bug? by any standard definition, yes... but what if they only want their computer to turn on some LEDs that are connected to the motherboard?
shill_420@reddit
business forwards some weird, but completely expected behavior to bsa. bsa forwards email to devs "seems like a bug". devs create ticket to "fix the bug." tale as old as time
Norphesius@reddit
Bugs are ultimately a social construct. As the old saying goes "It's not a bug, its a feature."
Obviously there's program behavior that is virtually universally undesirable (any thing that leads to elevated privilege or remote code execution), but beyond that bugs are defined by the spec. You can create and fix any amount of new bugs instantly by simply altering the program spec, without ever touching the code. The author even implicitly suggests this, they only track data for vulnerabilities, not for the other kinds of business logic errors.
So given that, its probably better to narrow the question to "Approaching zero vulnerabilities?" and for that I don't know, but I don't think its just a matter of tooling. The amount of bugs (vulnerabilities and logic) in software correlates with the size of its code. Not just the source code, but the dependencies, compiler/interpreter, and even OS syscalls. If we have a large surface area of code for bugs to manifest on, then it seems likely that even with near perfect tooling, you'd just just end up in a game of bug whack-a-mole, or with bug fixes that are mutually exclusive with intended behavior and other bug fixes.
If we want to approach zero vulnerabilities (and zero logic bugs, since that is still desirable), then we need to apply advanced tooling to new projects with smaller designs. We can't just have software code bases balloon forever and expect tooling to pick up the slack and fix all the bugs for us at scale.
va1en0k@reddit
Let's be utopic for a sec. Please understand that I don't "expect" this to happen, just trying to imagine a good equilibrium, even if it's not generally possible.
The problem with a lot of tools is that they spread out in feature-space much faster than their bugs can be found and fixed. For minimizing bugs, it'd be better not to evolve feature-complete packages.
Could we say that LLMs might help with that? I'd say yes, for a very simple reason. LLMs make it possible for the end users to rely less on the additional features. If you have a basic LLM, even local one, you might be able to do more with less features from your dependencies. We don't need to support every fancy weird use-case, every little CLI or UI affordance, because many of the users can figure them out on their own with their chatbots, as long as we give them load-bearing, well-tested basic functionality in good packages.
Of course there's a force that's pulling in the other direction. As features seem to many people to be "cheaper" to implement and maintain, their open-source packages spread out and do more and more. That's a counter-productive disaster that is absolutely pointless, as nobody really needs all of them. I think we can only hope that a taste will develop to avoid tools, packages and libraries with excessive features like the plague.