Dialectics, AI, skepticism

Posted by messedupwindows123@reddit | ExperiencedDevs | View on Reddit | 30 comments

I've always been very skeptical of tools like Copilot. At first, I would say things like "a good type system, with a good IDE, gets you most of the way there". I would also point out that the code generated by these tools is often wrong/bad.

But I think I figured out what really bugs me about these tools. For me, writing software contains a super important feedback loop. I have ideas, and I write them down. Then, by engaging with the written representation of my ideas (and by looking at how the computer reacts to these written representations), my own ideas can evolve. And then I write down the new ideas. And I run/inspect the code, and this feedback loop continues.

So, for example, I might have an elegant idea, and this idea doesn't pass the type-checker. And the type-errors will raise really interesting conceptual issues that I may have initially overlooked. And, by harmonizing these two things (my own ideas, and compiler-feedback), I'll get to something I'm really happy with.

So, loosely speaking, the ideas do not stand on their own (they have to be validated by being materialized into code). And the code does not stand on its own (you have to actually fully understand it, and you have to contextualize this understanding in a knowledge of how your ideas evolved over time). So the "code" and "concepts" are really intertwined. Sort of in the way that the sun and the moon orbit one another.

By leaning heavily on code-generation tools, this feedback loop gets broken. And I can't help but think that it will be really easy to wind up in really nasty situations where you're stuck at a "local maximum", and you don't have the tools to break out and find the real solution that you're really hunting for.

Does this resonate with anyone else? Are you able to maintain this feedback-loop while using these new tools?