I'm a cranky old sysadmin, I've seen more dumb IT trends come and go than you can imagine. AI is NOT a trend, and it's not going away. Cynicism is often useful, but what's important to watch is the rate of change. In three years it's gone from a plausibly useful novelty for writing emails to actually taking over major software engineering work. Last week I was at a conference and saw first hand how vendors are already getting real-world results with AI running sophisticated troubleshooting on in-house systems (taking a services syslog errors and going all the way to discovering specific patch sets that need to be applied to resolve them, writing patches for new features, etc). I talked to a dev from Microsoft, and in his words, he hasn't written any code in months now, Claude is writing all of his code (with MS pushing this heavily internally). Humans only do reviews now, and AI is even taking over large parts of review work now. I've heard rumors this was the case, but this was the first time I've heard this in-person.

About a year ago I leaned into using AI as a troubleshooting assistant in some very nasty problems, and it's only getting better. It might take me a few hours to run through some tcpdumps, compare to the RFC, check a few different hosts for clean and bad captures, etc. AI easily digests stacktraces, tcpdumps, logs, etc and can give you line-by-line explanations of what you're seeing and it does it at lightning speed. If I hit a new technology or five while working on a problem, getting to talk to the manual and ask it questions is an order of magnitude faster than me reading the manual and figuring out what parts apply to my problem.

For now, the freely available open weight models out there are good, but not the same as the frontier models. If your company is willing to invest in a small in-house gpu cluster, do not pass go, do it now, you want to be the person with experience with setting this up and running it. The models are only getting better, and having infrastructure in place now will let you drop in updated models as they come out. But, an in-house model allows you to not worry as much about sharing sensitive information with the frontier models that are also scraping your information. At this point, this is a performance increase. Another thing is you can teach your in-house system to know YOUR environment, so it's not just guessing at your environment like when talking to a frontier model.