This makes intuitive sense. It's the classic Waymo vs GoogleMaps dichotomy: Google Maps offers routes that are actually faster, but Waymo feels faster. That is because Google Maps will pull you through traffic, and you will have to stop at key points, but it's still the fastest route overall. Waymo tries to avoid this frustrating experiences that make you feel slow but actually are the setup needed to go as fast as possible.

BTW I really appreciated that the article has a table specifying what they are not claiming, and the real scope of the context. It's so important (especially in ML research) that I want to quote it here:

We do not provide evidence that: | Clarification AI systems do not currently speed up many or most software developers | We do not claim that our developers or repositories represent a majority or plurality of software development work AI systems do not speed up individuals or groups in domains other than software development | We only study software development AI systems in the near future will not speed up developers in our exact setting | Progress is difficult to predict, and there has been substantial AI progress over the past five years ^3 There are not ways of using existing AI systems more effectively to achieve positive speedup in our exact setting | Cursor does not sample many tokens from LLMs, it may not use optimal prompting/scaffolding, and domain/repository-specific training/finetuning/few-shot learning could yield positive speedup

That's just so nice that I now wish many papers, and every scientific article, had a table like this at some point shortly after the introduction/abstract.

Also lets be clear (in the same spirit as the table above) that this post is just speculations and intuitions on my part, none of this should be taken as true from this.

It makes sense though. AI speeds you through a lot of things, and if you have a good enough idea of what you want, it will give you a good enough solution. I feel that seniors sometimes lack the vision that when they give stuff out to mid and especially junior engineers, it already has patterns and references that can help create a mental model that the engineers can follow when they make their own thing. It may look differently but it still fits within the same model. LLMs are the opposite, they only make things that look the same, even when they don't fit within the model at all. To compound issues engineers are throwing LLMs to write code that is still too early to make it work. You have to go back and fix this things. The conventions and tricks earned to guide it just won't work with LLMs.

And honestly anyone whose gone to a serious programming language discussion, you learn that what really matters is not the syntax, but the semantics, the meaning of things. LLMs understand language at a syntactic level perfectly, but not semantic. They don't understand what the word on its own means, but rather the relationship it has with the words around it and what goes next.

Now I think that agentic AIs need a lot of work to get good and useful. They are too mediocre and dumb, and you're better off doing it yourself many times. Ultimately it's the same balance of automation we've had before just tweaking the prompt rather than the script.

And I do think that agentic AIs have their value. I think that as code analyzers (what static analyzers do nowadays) which is the obvious. Less obvious I believe is automated code improvers. So whenever I do a change in my library (be it an open source library, or one used by others) which has deprecated code, or now prefers something is done in a new way vs the old, I include a small documentation on how change the old way of doing code to the new one, as part of release documentation/notes/commit description. Then an agent on a downstream library can pick up on this, and create its own PR updating the downstream library's use of your stuff for you. Sure the library author would have to care to make sure that the code changes are easy for an LLM to do, but this isn't new. I tend to write code changes in a way that is awk-friendly so that it's easy to do automated changes on downstream libraries as a janitor.

But that kind of hints at the thing. None of those things "speed up developers" as the idea goes. Rather they simply free up time from developers who are valuable but struggle to explain that value (yet companies who lack these developers struggle really bad).