TLDR; "Harness" seems like a good term for the underlying controller between LLM inference servers and some kind of task, Chat, Assistant, or Coder (which all seem to be in the realm of "Agent" since it can "make its own choices").

This is something I had noticed. People seem to use a lot of words while not fully agreeing on the meanings, a byproduct of this being all so new. This means I've heard things like Claude Code or Cursor called Agentic AI tools or Agentic IDEs, etc., but they've expanded to do other things with tools and MCP servers (such as reading files, making files, sending emails, etc. which is why it seems Claude Coworker is now separate from both Chat and Code now), sometimes being called AI Assistants (such as PicoClaw and OpenClaw) which can and are most often used for coding (as the only users who would use them tend to be more computer literate anyway). It's essentially a program that acts as an autonomous agent with an LLM backed communication interface (or rather, more of the, well, harness to allow you to do those things instead of being the agent itself), but the problem is that interface may or may not be a CLI (you can use a GUI these days too), so I've been struggling to figure out what the best word is.

Other programs that run the LLMs themselves like Ollama and llama.cpp or LMStudio, etc., which only run the LLM inference itself, so they're often called "Providers", "LLM Servers", or "Inference Servers", or LLM Engines, but an engine sounds like it could apply to agentic AI too, like an AI Engine (as AI doesn't refer to just language models, hence why we don't use the more generic "AI" that everyone knows to refer to LLM chat interfaces like ChatGPT or Gemini).

From my understanding with what "Test Harnesses" are in Software engineering,

LLMs are more like the "engine", the lifeblood, if you will, and AI/agent harnesses are the wrapper systems that control and make it ready for use by either an actual AI Agent or a Human who provides input. LLM -> Provides either just inference of intent or actually automated reasoning to decide what tools to use and how to meet an user's goal, and a harness is the one that controls that, defining limits and security and capabilities.

Basically, LLMs seem to have evolved from NLP for natural interfaces with computers to being "brains" for agentic AI as well, but harness seems to be a good term for both use cases, whether a chat interface or an actual autonomous agent you could just let run for long periods of times while it figures out how to handle its directives. Getting some Tron vibes here.

In this new landscape, it's really no wonder this is really difficult to explain to the average person other than "The AI."