I think your argument relies on stuff that is unproven. For instance, it takes as a given that AGI not only is possible to build (and it behooves to remember that we don't actually know that it is), it will inevitably turn hostile (again, unproven), and then proceeds to kill/enslave humans. This kind of stuff has very low predictive power, because it is contingent on an if-on-if-on-if. You either see this or you don't.

Firstly, AGI may be impossible to build. Now, this is on its face probably not a very compelling starting point, but it needs to be stated. Most people seem to assume that technology marches ever forwards, and have literally no conception of limits of technology, and so it doesn't seem a stretch to simply assume that an omnipotent AI will one day exist. But AI is constrained by the physical realities of our finite planet: access to minerals and energy is limited on our planet. This prevents covering the whole planet with solar panels or wind turbines, or similar rollouts that have scale that exceeds the rate at which sufficient materials can be mined, transported and refined, and the level of energy that is available on this planet.

I work in technology, though not AI. I use AI tools. Machine learning as it stands today is really synonymous with statistics. If you have lots of data, you can fit a predictive model that learns the features of the data and predicts outcomes based on variables. In the simplest versions of "machine learning", you just fit a linear regression and then the machine, having "learnt" parameters a and b, applies y = ax+b to your input x, and that is the "prediction". In case of today's neural networks, the networks learn not only the "parameters" for the best fit, but also the "formula", by using the weights and biases of the network together with the network's nonlinear elements to find ways to learn the data in order to make predictions later.

LLMs are famously text completion engines. The text arrives in some kind of thousands of dimensions long vectors that are processed by mindnumbingly vast matrices that transform these vectors, and then do it again hundreds of times, stacking transformation on top of transformation... Somewhere in there, the meaning of these vectors is encoded and results in prediction of the next word that makes sense to us because it is similar enough to "real" writing the model has been trained with.

AIs have been employed to search for improved architectures, though, as people are trying to get that recursive self-improvement loop going. But even that is not so simple, because this stuff is all based on statistics and it takes a long training run for network to learn statistical properties of language, which start from literally random gibberish to the model until over time the correlations between words begin to influence the model and it gradually learns grammars, facts, concepts, and so forth until it talks almost like us. People tend to assume that AI can rewrite itself in an instant and create a better copy. Maybe so, but it isn't base on the approach we have found most promise with, if so.

Secondly, the alignment/hostility of AGI. You sound like one of those rationalists that made these arguments some years ago, but which either have been silent for some years now, or it's just that nobody is paying any attention to that stuff anymore. I think reality of the situation may have settled in. After watching LLMs struggle to compute simple arithmetic and failing to get the result correct, people have gradually become more realistic about how severely crippled this technology is. However, if it can at least reliably write programs, it can then use the programming environment as tool to solve numerical problems, just like humans. It's just that it seems that humans are now better at simple arithmetic than computer inferring with AI software that simultaneously has some of the most eye-watering computation costs you can possibly imagine, and that is a stark inversion from past half-century.